The Exact Query Complexity of Yes-No Permutation Mastermind

Mastermind is famous two-players game. The first player (codemaker) chooses a secret code which the second player (codebreaker) is supposed to crack within a minimum number of code guesses (queries). Therefore, codemaker's duty is to help codebreaker by providing a well-defined error measure between the secret code and the guessed code after each query. We consider a variant, called Yes-No AB-Mastermind, where both secret code and queries must be repetition-free and the provided information by codemaker only indicates if a query contains any correct position at all. For this Mastermind version with n positions and $k\le n$ colors we prove a lower bound of $\log_2(k+1-n)+\log_2(k+2-n)+\dots+\log_2(k)$ and an upper bound of $n\log_2(n)+k$ on the number of queries necessary to break the secret code. For the important case $k=n$, where both secret code and queries represent permutations, our results imply an exact asymptotic complexity of $\Theta(n\log_2(n))$ queries.


Introduction
Mastermind is a popular two-player board game invented in 1970 by Mordecai Meirowitz. Its idea is that a codemaker chooses a secret code of fixed length n, where each position is selected from a set of k colors. The second player, codebreaker, has to identify the secret code by a finite sequence of corresponding code guesses (queries), each of which is replied with the number of matching positions and the number of further correct colors. The original game is played by picking pegs of k = 6 different colors and placing them into rows with n = 4 holes, where the number of rows (allowed queries) for codebreaker is limited.
Generalizing the situation to arbitrary many positions and colors, codemaker selects a vector y ∈ [k] n and codebreaker gives in each iteration a query in form of a vector x ∈ [k] n . In the original setting, codemaker's reply is the so called black-white error measure, consisting of a pair of numbers, where the first number, black(x, y), is the number of positions in which x and y coincide and the second number, white(x, y), is the number of additional colors which appear in both x and y, but at different positions. In this paper, we consider a variant, called Yes-No AB-Mastermind, which is defined by the following properties • Both secret code and queries must be repetition-free. This property is indicated by the prefix AB and stems from the AB game, better known as "Bulls and Cows", which was even known prior to the commercial version of Mastermind with color repetitions.
• The provided information by codemaker only answers the question whether or not a query contains any correct position at all. This property is introduced by us and referred to by the term "Yes-No".
Related Works: Mastermind and its variants have been analyzed under different aspects. One of the first analysis of the commercial version with n = 4 positions and k = 6 colors is due to Knuth [12] and shows that each code can be cracked in at most 5 queries. Even before the appearance of Mastermind as a commercial game Erdős and Rényi [8] analyzed the asymptotic query complexity of a similar problem with two colors in 1963. After Knuth's analysis of the commercial game, many different variants of Mastermind with arbitrary code length n and number of colors k have been investigated. For example, Black-Peg Mastermind restricts its error measure between two codes x and y to the single value black(x, y) (i.e., the exact number of positions where both codes coincide). This version was introduced in 1983 by Chvátal [3] for the case k = n, who provides a deterministic adaptive strategy using 2n⌈log 2 k⌉ + 4n queries. Improved upper bounds for this variant and arbitrary n and k where given by Goodrich [10] and later by Jäger and Peczarski [11] but remained in the order of O(n log 2 n). Doerr et al. [4] provided a randomized codebreaker strategy that only needs O(n log log n) queries in expectation and showed that this asymptotic order even holds for up to n 2 log log n colors, if both black and white information is allowed. A first upper bound for AB Mastermind was given by Ker-I Ko and Shia-Chung Teng [13] for the case k = n, i.e., secret code and queries represent permutations of [n]. Their non-constructive strategy yields an upper bound of 2n log 2 n + 7n queries. A constructive strategy by El Ouali and Sauerland [6] reduced this upper bound by a factor of almost 2 and also included the case k > n of Black-Peg AB-Mastermind. The term Black-Peg labels the situation that the error measure between secret code and queries is only the "black" information, i.e. the number of coinciding positions, while the "white" information (see above) is omitted. El Ouali et al. [5] combined the upper bound with a lower bound of n queries, which is implied by a codemaker (cheating) strategy. It improved the lower bound of n − log log n by Berger et al [2]. However, a gap between Ω(n) and O(n log 2 n) remains for this Mastermind variant. Some facts indicate that closing this gap means to improve both bounds. On the one hand, a careful consideration of the partition of the remaining searchspace with respect to all possible codemaker replies might yield a refined codemaker (cheating) strategy and possibly increase the lower bound. On the other hand, overcoming the sequential learning process of the codebreaker's binary search strategy might decrease the upper bound. The latter presumption is reinforced by results of Afshani et al. [1], who consider another permutation-based variant of Mastermind. There, the secret code is a combination of a binary string and a permutation, (both of length n), queries are binary strings of length n, and the error measure returns the number of leading coincidences in the binary string with respect to the order of the permutation. For this setting, which is also a generalization of the popular leading ones test problem in black box optimization, the authors prove an exact asymptotic query complexity of Θ(n log n) for deterministic strategies but a randomized query complexity of Θ(n log log n).
One of the ultimate goals in the analysis of Mastermind variants is to prove the exact asymptotic query complexity. As mentioned above, closing the asymptotic gap between the lower Ω(n) bound and the upper O(n log 2 n) bound is an unsolved problem for Black-Peg AB Mastermind. A related open question is whether the same asymptotic number of queries is required for both, (Black-Peg) Mastermind with color repetition and (Black-Peg) AB Mastermind.
Our Contribution: We consider a new variant of AB-Mastermind which is more difficult to play for codebreaker since the error-measure provided by codemaker is less informative. Here, for a secret code y the answer info(σ, y) to a query σ is "yes" if some of its positions coincide with the secret code, otherwise the answer is "no". We first analyze the worst-case performance of query strategies for this Mastermind variant and give a lower bound of k j=ℓ log 2 j queries for k ≥ n, which becomes n log 2 n − n in the case k = n. The lower bound even holds if codebreaker is allowed to use repeated colors in his queries. We further present a deterministic polynomial-time algorithm that identifies the secret code. This algorithm is a modification of the constructive strategy of El Ouali et al. [5]. It returns the secret code in at most (n − 3) log 2 n + 5 2 n − 1 queries in the case k = n and in less than (n − 2) log 2 n + k + 1 queries in the case k > n. For the important case k = n, our results imply the exact asymptotic query complexity of Θ(n log 2 n). Since the considered "Yes-No" error measure implies a new variant of AB-Mastermind, there is no previous reference to compare our results to.

Lower Bound on the Number of Queries
To simulate the worst case, we allow codemaker to "cheat" in a way that after every query he may decide for a new secret code concerning the answers given so far. Proof. We give a codemaker (cheating) strategy that implies the lower bound. For i ∈ N let M i denote the set of secrets that are still possible after the i-th query has been answered, starting with M 0 := {y ∈ [k] n | ∀i = j ∈ [n] : y i = y j }. Let M yes i ⊂ M i be the set of secrets that lead to a yes-answer to the (i + 1)-th query and M no i := M i \ M yes i the set of secrets that lead to a no-answer. The strategy of codemaker in round i + 1 is as follows: and give the answer yes) • Otherwise pick a secret from M no i (and give the answer no) By using this strategy, codemaker achieves for every round i that which means that there are still at least two possible secrets left. Since we obtain the claimed lower bound.
Corollary 2. Every strategy for Yes-No Permutation-Mastermind (the case k = n) needs at least n j=1 log 2 j ≥ n log 2 n − n queries in the worst case.

Upper Bound on the Number of Queries
Theorem 3. Let k, n ∈ N, k ≥ n and ℓ := k + 1 − n. For k = n, there is a strategy for Yes-No AB-Mastermind that identifies every secret code in at most (n − 3) log 2 n + 5 2 n − 1 queries and for k > n, there is a strategy that identifies every secret code in less than (n − 2) log 2 n + k + 1 queries.
Corollary 4. The exact asymptotic query complexity of Yes-No Permutation-Mastermind is Θ(n log 2 n).
The proof of Theorem 3 resembles the proof of a corresponding result concerning Black-Peg AB-Mastermind [5], except that the information whether a given query contains a correct but unidentified position is not derived directly but requires special querying outlined by Algorithm 1 below. In a nutshell (summarizing with regard to both cases k = n and k > n), the strategy consists of k initial queries which are the first n positions of shifted versions of the vector (j) j∈ [k] . From the answers of the initial queries, we will be able to learn the secret code position-wise, keeping record about the positions that have already been identified. As long as there are consecutive initial queries a and b with the property that a coincides with the secret code in at least one yet unidentified position but b does not, we can apply a binary search for the next unidentified position in a, using O(log 2 n) further queries. Such initial queries a and b exist ever after one (usually after zero) but not all positions of the secret code have been identified.
Proof of Theorem 3. The case k = n: We give a constructive strategy that identifies the positions of the secret code y ∈ [n] n one-by-one. In order to keep record about identified positions of the secret code we deal with a partial solution vector x that satisfies x i ∈ {0, y i } for all i ∈  Figure 1 illustrates the four distinct cases that are considered by infoP. In the first and easiest case (panel (a)) the actual query σ does not coincide with the partial solution x. Thus, σ contains a correct unidentified position if and only if it contains a correct position at all, i.e., infoP(σ, x, y) = info(σ, y). In the second case (panel (b)), σ and x coincide in more than one position, namely the positions with colors 3, 9 and 10. The modified query ρ is obtained from σ by rearranging these positions in a way that all identified positions get a wrong color while leaving all open positions of σ unchanged. This implies that infoP(σ, x, y) = info(ρ, y). Panels (c) and (d) deal with the case that σ and x coincide in exactly one position, say i. If x already contains a further non-zero position j, we obtain ρ from σ by swapping positions i and j in σ (the positions with colors 3 and 5 in panel (c)). Again, we obtain that infoP(σ, x, y) = info(ρ, y). Finally, if position i is the only yet identified position of the secret code we have to ask two different modified queries to derive infoP(σ, x, y) (panel (d)). We obtain the two queries ρ 1 and ρ 2 , each by swapping the identified position (here 3) with another position in σ, (here with 1 and 2, respectively). While the color of the identified position is wrong in both modifications ρ 1 and ρ 2 , every other position of σ coincides with the corresponding position of at least one modification. Therefore, infoP(σ, x, y) = no if and only if info(ρ 1 , y) = info(ρ 2 , y) = no.
(a)  Exactly one position is identified and appears to be correct in σ; two modified queries ρ 1 and ρ 2 must be defined, each by swapping the identified position with another one; infoP(σ, x, y) = no if and only if info(ρ 1 , y) = info(ρ 2 , y) = no.
The codebreaker strategy that identifies the secret code y has two phases. In the first phase codebreaker guesses an initial sequence of n queries that has a predefined structure. In the second phase, the structure of the initial sequence and the corresponding information by codemaker enable us to identify correct positions y i of the secret code one after another, each by using a binary search. We denote the vector x restricted to the set {s, . . . , ℓ} with (x i ) ℓ i=s , s, ℓ ∈ [n].
Phase 2. Now, codebreaker identifies the values of y one after another, using a binary search procedure, that we call findNext. The idea is to exploit the information that for and σ n n = σ 1 1 . findNext is used to identify the second correct position to the last correct position in the main loop of the algorithm.
After the first position of y has been found and fixed in x, there exists a j ∈ [n] such that infoP(σ j , x, y) = no. As long as we have open positions in x, we can either find a j ∈ [n − 1] with infoP(σ j , x, y) = yes but infoP(σ j+1 , x, y) = no and set r := j+1, or we have infoP(σ n , x, y) = yes but infoP(σ 1 , x, y) = no and set j := n and r := 1. We call such an index j an active index. Let j be an active index and r its related index. Let c be the color of some position of y that is already identified and fixed in the partial solution x. With ℓ j and ℓ r we denote the position of color c in σ j and σ r , respectively. The color c serves as a pivot color for identifying a correct position m in σ j that is not fixed, yet. There are two possible modes for the binary search that depend on the fact if m ≤ ℓ j . The mode is indicated by a Boolean variable leftS and determined by lines 5-9 of findNext. Clearly, m ≤ ℓ j if ℓ j = n. Otherwise, codebreaker guesses σ j,0 := c, (σ j i ) By the information σ j i = σ r i+1 we obtain that (σ j i ) . We further know that every open color has a wrong position in σ r . For that reason, infoP(σ j,0 , x, y) = no implies that m ≤ ℓ j .
In the first case, the first ℓ − 1 positions of σ j,ℓ coincide with those of σ j . The remaining positions of σ j,ℓ cannot coincide with the corresponding positions of the secret code if they have not been fixed, yet. This is because the ℓ-th position of σ j,ℓ has the already fixed value c, positions ℓ + 1 to ℓ j coincide with the corresponding positions of σ r which satisfies infoP(σ r , x, y) = no and the remaining positions have been checked to be wrong in this case (cf. former definition of leftS in Algorithm 2: Function findNext input : Secret code y, partial solution x = 0 and an active index j ∈ [n] output: A correct open position in σ j 1 if j = n then r := 1; 2 else r := j + 1; 3 Choose the color c of some non-zero position of x; 4 Let ℓ j and ℓ r be the positions with color c in σ j and σ r , respectively; 5 if ℓ j = n then leftS := true; 6 else 7 σ j,0 := c, (σ j i ) if leftS then σ j,ℓ := (σ j i ) Example 6. Suppose, that for n = 10 the secret code y and the partial solution x are given as in the top panel of Figure 2 and that we have first identified the position with color 1, such that 1 is our pivot color. The initial 10 queries σ 1 , . . . , σ 10 together with their current infoP measures are given in the mid panel of Figure 2. We see, that the highlighted queries, σ 4 and σ 5 , can be used for the binary search with findNext, since σ 4 has a correct not yet identified position but σ 5 has not. So the active indices are j = 4 and r = 5 and the corresponding pivot color positions in σ 4 and σ 5 are ℓ j = 4 and ℓ r = 5. The first query of findNext (cf. lower panel of Figure 2) is σ a . It begins with the pivot color, followed by the first 3 positions of σ 4 (positions 2 to 4 of σ 5 ) and positions 5 to 10 of σ 4 (cf. line 7 of findNext). Since infoP(σ a , x, y) = yes, the left most correct but unidentified position in σ 4 is none of its first 4 positions. Thus, the binary search is continued in the interval [5,10]. It is realized by queries σ b , σ c , and σ d , which are composed according to line 20 of findNext (in this case), and finally identifies position 8 with color 5 of (a)  in the beginning, we first identify the correct position in σ 1 (which is unique in this case) by ⌊ n 2 ⌋ + 1 queries (each swapping two positions of σ 1 ) and update x and v, correspondingly. After this step, there will always exist an active index. Every call of findNext in the main loop augments x by a correct solution value. One call of findNext requires at most 1 + ⌈log 2 n⌉ queries if the partial solution x contains more than one non-zero position, and at most 2 + 2⌈log 2 n⌉ queries (two queries for each call of infoP) if x has exactly one non-zero position. Thus, Algorithm 3 does not need more than (n − 2)⌈log 2 n⌉ + 5 2 n − 1 queries to break the secret code inclusive the n − 1 initial queries, ⌊ n 2 ⌋+1 queries to find the first correct position, n−3 calls of findNext and 2 final queries.
Algorithm 4: Function findNext for k > n input : Code y, partial solution x = 0 and an active index j ∈ [k] output: A position m that is correct in σ j 1 if j = n then r := 1; 2 else r := j + 1; 3 a := 1, b := n; 4 while b > a do variant is that codemaker's answers are restricted to the information whether query and secret code coincide in any position. A bigger challenge with AB-Mastermind is that no color repetition is allowed in a query whereas most strategies for other Mastermind variants exploit the property of color repetition. While for most Mastermind variants there is a gap between lower and upper bounds on the worst case number of queries to break the secret code, our results imply that this number is Θ(n log n) for the most popular case k = n of Yes-No AB-Mastermind, which is also referred to as Yes-No Permutation-Mastermind. To our knowledge, this result is a first exact asymptotic query complexity proof for a multicolor Mastermind variant, where both secret code and queries are chosen from the same set, here [n] n .
A future challenge will be studying the static variant of Yes-No AB-Mastermind (where codebreaker must give all but one queries in advance of codemaker's answers). Lower and upper bounds for static Black-Peg AB-Mastermind were provided as Ω(n log n) and O(n 1.525 ), respectively [9].
Codeavailability: We provide Matlab/Octave implementations of the codebreaker strategy via GitHub, a permanent version of which is archived in a public zenodo repository [7].