The human data for the estimation set was made available to researchers ahead of time. They were allowed to analyze the data, study the observed behavior, and build their own models. After the submission deadline, the competition set was run using the same behavioral procedures used in the estimation set. The competition set involved the selection of 40 games different from those used in the estimation set, but the games were also determined by the same selection algorithm (Appendix 1 in [1]). The MEC focused on the models' predictions of the six statistics described above in the new 40 competition set problems. Currently, both the estimation and competition studies are publicly available, posted by the organizers in the MEC web page (http://sites.google.com/site/gpredcomp/study-results/competition-study).

Twenty-five models were submitted to the competition and each of the models was evaluated using the mean squared deviation (described in detail in Erev et al. [1]) between the models' predictions and the observed performance in the competition set. Each model obtained a final score, the nMSD (normalized mean squared deviation). The nMSD is the mean of the six normalized mean squared deviations (MSD) for each of the six statistics described above. Each of the MSDs for each of the dependent measures was calculated in the following three steps: (1) compute the squared deviation between the model's prediction and the observed statistic in each of the 40 games; (2) compute the mean squared deviation over all the 40 games; and (3) normalize each score by the variable's estimated error variance. The model with the lowest nMSD score won the MEC.

IBL models are particular representations of IBLT for specific tasks. Many IBL models have been developed in a wide variety of tasks, including dynamically-complex tasks [2-5], training paradigms of simple and complex tasks [6,7], simple stimulus-response practice and skill acquisition tasks [8], repeated binary-choice tasks [9,10] among others. Although most of the IBL models developed are task specific, a recent IBL model showed that it generalizes well to multiple repeated-choice tasks that share the same task structure. The IBL model reported in Lejarraga, Dutt, and Gonzalez was built to predict performance in individual binary-choice tasks, and generalized accurately to choices in a repeated-choice task, probability-learning tasks, and repeated-choice tasks with changing probability of outcomes as a function of trials [10]. The IBL model for the market entry task is an extension of the model reported in Lejarraga et al. [10].

Instances in the repeated choice and MEC model are much simpler than in other IBL models, as the structure of these tasks is simple. Each instance consists of a label that identifies an alternative in the task and the outcome obtained (i.e., a button label and its observed outcome). For example (Enter, $4), is an instance in which the decision was to enter the market and the outcome as a result of that choice was $4. In the MEC, since participants also observed forgone payoffs, another similar instance was created for the obtained foregone payoff in each decision made, for example (Stay Out, $3).

In each trial t of a market-entry game, the process of selecting alternatives in the model starts with an inertia rule (Equation 1 below), which determines whether the previous choice in the task is repeated according to the surprise-triggers-change hypothesis by Nevo and Erev [12]. If the previous choice is not repeated, then the alternative with the highest blended value is selected (Equation 5 below). The blended value of an alternative is calculated in each trial t of the game and it is a transient value that depends on the outcome stored in an instance and the probability of retrieval of that instance from memory (Equation 6 below). Furthermore, the probability of retrieval of an instance from memory is a function of its activation in memory (activation is a function of the recency, frequency and noise in the retrieval of an instance) (Equation 7 below).

Erev et al. [1] report several behavioral regularities found in the estimation set in the MEC: Surprise-triggers-change from choosing one alternative to the other, and the presence of strong inertia in repeated choice. These two effects relate to sequential dependencies between choices in human data, measured with the Alternation rate. These sequential dependencies in choices over time have also been demonstrated by Biele, Erev, and Ert [11] and Nevo and Erev [12].

Our model in Lejarraga et al. [10] was not built to account for alternations, but rather to account for the proportion of risky choices in repeated-choice tasks. Given that many influential models of repeated choice have found a weak relationship between the generic measures of performance and sequential dependencies [13,14], we investigated how IBL models are able to account for sequential dependencies and the tradeoffs between the proportion of risky choices and the proportion of alternations in repeated-choice tasks [15]. To capture sequential dependencies in the MEC data, we built on the surprise-triggers-change hypothesis by Nevo and Erev [12] and proposed a new inertia mechanism that considers blended values instead of running averages. This mechanism is determined at the moment of making a choice in trial t+1 by a simple rule:

If the draw of a random value in the uniform distribution (1) U ( 0 , 1 ) < ( pInertia ) Surprise ( t )

Then

Repeat the choice as in the previous trial

Else

Select the alternative with the highest blended value as per Equation 2 (below)

The pInertia or the Probability of Inertia is a free parameter between 0 and 1 and initially defined at 0.30, according to Biele et al. [11]. The value of the Surprise(t) is assumed to depend on the gap (absolute difference) between an expectation of the outcome and the outcome actually received. In our inertia mechanism this is the absolute difference between the observed outcome and the blended value for that alternative. Since forgone payoffs are observed in the market entry games, the gap is calculated for the two alternatives, enter and stay out, as follows: (2) Gap ( t ) = ½ × ( | outcome Enter ( t − 1 ) − V Enter ( t − 1 ) | + | outcome Stay out ( t − 1 ) − V Stay out ( t − 1 ) | )

The outcome _Enter (t–1) is the observed or foregone outcome obtained upon entering the market in the last trial and outcome _{Stay out} (t–1) is the observed or foregone outcome obtained upon staying out. The V _Enter (t–1) and V _{Stay out} (t–1) are the blended values of the two alternatives obtained in the last trial (the calculation of blended values is defined below).

The surprise in trial t is normalized by the mean gap (in the first t−1 trials): (3) Surprise ( t ) = Gap ( t ) / [ Mean ( Gap ( t ) ) + Gap ( t ) ]

The Mean(Gap(t)) is defined over 50 trials of a market entry game as: (4) Mean ( Gap ( t ) ) = Mean ( Gap ( t − 1 ) ) ( 1 − 1 / 50 ) + Gap ( t ) ( 1 / 50 )

Erev et al. [1] justified the gap-based formulation of surprise by the observation that the activity of certain dopamine-related neurons is correlated with the difference between average past payoff (or the blended value) and the present outcome. This assumption is the only extension to the model reported in Lejarraga et al. [10] needed to account for the sequential dependencies reflected in the proportion of alternations in the market entry games. Naturally, the higher the value of pInertia, the more the IBL model will repeat its choice.

In making a choice, IBLT selects the alternative with the highest blended value, V [16] resulting from all instances belonging to an alternative. The blended value of alternative j is defined as

Where x_i is the value of the observed (obtained or foregone) outcome in the outcome slot of an instance i corresponding to the alternative j, and p_i is the probability of that instance's retrieval from memory (for the MEC as noted in Equation 2, the value of j is either to enter or to stay out). The blended value of an alternative (its utility) is the sum of all observed outcomes x_i in the outcome slot of corresponding instances in memory, weighted by their probability of retrieval. In any trial t, the probability of retrieval of instance i from memory is a function of that instance's activation relative to the activation of all other instances corresponding to that alternative, given by

Where τ is random noise defined as = σ × 2, and σ is a free noise parameter (more details below). Noise in equation 2 captures the imprecision of recalling instances from memory.

The activation of each instance in memory depends upon the Activation mechanism originally proposed in the ACT-R architecture [3]. A simplified version of the activation mechanism that relied on recency and frequency of use of instances in memory was sufficient to capture human choice behavior in several repeated-choice and probability-learning tasks [10]. For each trial t, Activation A_i,t of instance i is: (7) A i , t = ln ( ∑ t i ∈ { 1 , … , t − 1 } ( t − t i ) − d ) + σ . ln ( 1 − γ i , t γ i , t )

Where d is a free decay parameter, and t_i is the time period of a previous trial where the instance i was created or its activation was reinforced due to an outcome in the task corresponding to the instance in memory. The summation will include a number of terms that coincides with the number of times that an outcome has been observed in previous trials and that the corresponding instance i's activation has been reinforced in memory. Therefore, the activation of an instance corresponding to an observed outcome increases with the frequency of observation (i.e., by increasing the number of terms in the summation) and with the recency of those observations (i.e., by small differences in t - t_i of outcomes that correspond to that instance in memory). The decay parameter d affects the activation of the instance directly, as it captures the rate of forgetting. In ACT-R, the d parameter is almost always set to 0.5. The higher the value of the d parameter, the faster the decay of memory, and the harder it is for the model to recall distant memories of its instances with outcomes.

The γ_i,t term is a random draw from a uniform distribution bounded between 0 and 1, and the σ . ln ( 1 − γ i , t γ i , t ) term represents Gaussian noise important for capturing variability of human behavior. The σ is a free noise parameter that has no default value in ACT-R, but that it has been found to have a mean of 0.45 in many ACT-R studies [17]. Higher σ values imply greater noise in the retrieval of instances from memory.

In the first trial of a game, the model has no past instances in its memory from which to calculate blended values. Therefore, the model makes a selection between two instances in memory for the first trial by assuming some initial blended values. Each initial blended value corresponds to one of the two alternatives, entering or staying out. The blended value of the pre-populated instances may represent the expectations that participants bring to the laboratory [10]. The choice of the blended values in the two instances was motivated from the observed entry rate in the first trial of the estimation set. We found that the observed entry rate was about 73% (>50%) in the first trial of the estimation set, i.e., more than 50% of the participants entered the market in the first trial. We speculate that one reason for the higher entry rate could be that the experiment was framed as a market entry competition. Due to this observation and the fact that the ratio of the blended values assigned to the two instances determines the entry rate in the first trial in the model, we assigned a +94 value as the blended value of the instance corresponding to the enter alternative and a +34 value to the blended value of the instance corresponding to the stay out alternative. As seen, the ratio of the value assigned to the instance corresponding to the “enter” alternative to the sum of the values assigned to both instances, i.e., 94/(94 + 43) computes to 73%, i.e., the observed entry rate in the first trial in the estimation set. In the first trial, a decision to enter or stay out is also based solely upon these blended values for each pre-populated instance. The inertia mechanism is used from the second trial onwards. As the value of V _Enter (0) and V _{Stay out} (0) do not exist, Gap (1) is calculated by replacing V _Enter (0) and V _{Stay out} (0) by 94 and 34 in Equation 2. Also, the Mean(Gap(1)) is taken to be a very small number close to 0 (= .00001). Our assumption on Mean(Gap(1)) is similar to that by Erev et al. [1].

The parameter values of the IBL-different and IBL-same models were obtained through an optimization process that maximized the fit of the model's data to human behavior in the estimation set. The process involved an optimization of the parameter values for each model involving a genetic algorithm. The goal of the generic algorithm was to find the set of parameters that minimized the error between the model's predictions and the observed behavior. Specifically, the genetic algorithm attempted to minimize the normalized mean squared deviation (nMSD).

The genetic algorithm tests different combinations of parameters in a model to minimize the nMSD between predictions and observed behavior across the 40 problems in the estimation set of the MEC. First, different parameter combinations (N) are selected and tested. This first group of parameters combinations is the referred to as the first “generation.” Each test of a combination of parameters involves running the model multiple times (i.e., multiple simulated participants) and obtaining the mean prediction, which is then compared to the mean behavior across the six measures that determine the nMSD. The parameter combinations are then ranked from best fitting to worst fitting based upon the calculated nMSD values. After ranking, the best half of the parameter combinations are kept (N/2), and the rest (N/2) are discarded. The parameter combinations that are kept are then duplicated, bringing the number of parameter combinations back to the original amount (N). Then, the N parameter combinations are paired off with each other at random (thus forming N/2 pairs). Now, each parameter combination exchanges some of its adjustable parameter values with the corresponding parameter value of its partner (this is called “reproduction”). For example, suppose the following two three-parameter combinations have been paired off: (a1, b1, c1) and (a2, b2, c2). Then, due to the exchange of the adjustable parameter values a1 and a2 in the pair, the resulting parameter combinations will be (a2, b1, c1) and (a1, b2, c2). The exchange of parameters defines a new generation that is different from the previous generation but maintains characteristics (i.e., the parameter values) of the previous generation's best cases. After the exchange, a new set of N parameter combinations is tested in the model and new nMSDs are obtained. The process is repeated for 10,000 generations. This value is extremely large and thus ensures a very high level of confidence in the optimized parameter values obtained. We simulated 100 four-participant teams for each combination of parameters in the model during optimization to derive an nMSD. Once the optimization was completed we increased the number of four participant teams in the model from 100 to 1,000. This value ensures that a model's prediction for the dependent measures is stable and does not change much from generation-to-generation for the same parameter combination used in the model.

For the purpose of optimizing the two IBL models using the genetic algorithm, the d and σ parameters were varied between 0.0 and 10.0, and the pInertia parameter was varied between 0.0 and 1.0. The assumed range of variation of d and σ parameters in the models is large and ensures that the optimization does not miss the minimum nMSD value on account of a small range of parameter variation.

The fitted values of the three parameters for the IBL-same model were:

d = 1.97, σ = 1.17, and pInertia = 0.23

The fitted values of the three parameters for the four simulated participants in the IBL-different submission were:

Player 1: d = 3.00, σ = 1.08, and pInertia = 0.13

Player 2: d = 1.73, σ = 1.44, and pInertia = 0.63

Player 3: d = 1.22, σ = 1.16, and pInertia = 0.02

Player 4: d = 2.93, σ = 1.26, and pInertia = 0.22

Because a larger number of free parameters allows models greater flexibility, the IBL-different model fitted the human data on the estimation set slightly better (nMSD = 1.153) than the IBL-same model (nMSD = 1.308).

These predictions of the IBL models were obtained for their best fitting parameters (determined above) for a set of 1,000 simulated four-player teams. The choices of all participants were averaged to obtain the six dependent measures used to evaluate the competing models: Entry rate, Efficiency, and Alternation rate in the first block (B1, the first 25 trials) and the second block (B2, last 25 trials) of each game. The MSDs and the nMSD value of each model were obtained by the three-step procedure detailed above.

Table 1 summarizes the MSDs for both models in the estimation set. Detailed values of the six statistics for the IBL-same, IBL-different, and observed values per problem are included in Table A1 in the Appendix.

Our principal motivation for the submission of two identical models that differed only in how the value of parameters for each simulated participant was treated (same or different per participant) was to explore the tradeoffs of complexity (in terms of number of parameters) and generalization [18].

Generalization is the process of predicting new findings from an existent model [19]. The MEC focused on the prediction of the six statistics a priori, implying that researchers could not use any information concerning the observed behavior in the competition set, since this was unavailable at the time of submission of models.

Although generalizing ability of a model was rewarded in the MEC, as well as in other recent model competitions [20], parsimony was not. For example, in a recent modeling competition of binary choice (i.e., Technion Prediction Tournament, TPT), the winner model in the “sampling paradigm” was a complex model made of 4 sub-models and 40 different free parameters [20]. Similarly, the winning model in the MEC involves 7 parameters where 6 out of the 7 parameters take a uniformly distributed range of values around a parameter mean. Both the MEC and the TPT assumed that by following the Generalization Criterion Method [19] as an evaluation procedure, parsimonious models (i.e., with a lesser number of parameters) would rank high. The authors of the Generalization Criterion Method suggest that, because the estimation and generalization sets are different “conditions,” simple models would generalize better than complex ones, an advantage that is absent in other evaluation methods like cross-validation which uses the same dataset split in different ways for calibration and testing of a model. We believe, and show evidence, that the estimation and competition sets in the MEC are not sufficiently different to produce the effect observed by Wang and Busemeyer [19].

Evidence suggests that models tradeoff complexity—which leads to accurate fit in the estimation set—with generalizing capacity (i.e., accurate predictions) [18], and often these dimensions tradeoff in non-linear ways [21,22]. Therefore, comparing models across these two dimensions is challenging.

Jae Myung, Mark Pitt, and colleagues have studied these tradeoffs in many different ways [21-23]. Some of the main conclusions that we can summarize from their work are that:

A complex model with many parameters can fit data better than a simple model with fewer parameters through over-fitting. Over-fitting occurs when a model captures not only the underlying phenomenon but also the noise and variability of a particular dataset. A model that captures noise of a data set would make poor predictions in unknown, generalization conditions.

Given that the IBL-same and IBL-different models were exactly the same on their cognitive assumptions and principles of how decisions makers learn and make decisions in a market entry game, the only difference in the models is that different participants may recruit cognitive processes differently, rather than being homogenous participants. Because the IBL-different model treated the four simulated participants in a market-entry game as different individuals (with different parameters), it is possible that this model captured not only the underlying learning and decision-making process, but also the noise and variability of different individuals in the estimation set. Thus, it was expected that IBL-different model would fit the estimation set better than IBL-same model.

However, our main question was whether the generalization procedure used in the MEC would favor the simple IBL-same model or the more complex IBL-different model. On the one hand and given the diverse values of the parameters per player, the IBL-different model may have over-fitted the estimation set and thus predict the data in the competition set worse than the IBL-same model. On the other hand, the estimation and competition sets were similar in many aspects that would question how challenging a generalization would be: The problems in the competition set had the same structure as those in the estimation set; the problems were obtained with the same selection algorithm in both studies; and the participants, although different, were drawn from the same population in both studies. Thus, it is likely that there exist systematic sources of variation and correlated noise across the estimation and competition set, which would favor the more complex IBL-different over the parsimonious IBL-same model in the competition set.

The observation that the parsimonious IBL-same model was outperformed by the more complex IBL-different model suggests that the competition set was not sufficiently different to the estimation set to favor parsimony over complexity (or flexibility). We argue that the generalization condition of the MEC had the characteristics of a traditional cross-validation rather than a generalization as in Wang and Busemeyer [19].

According to the cross-validation method [24], a dataset is divided into two samples, one sample is used as the estimation set and the other set is used as the prediction (competition) set. The MEC did not follow this procedure but rather ran two different studies, at different times, with different participants, and thus, in this sense, the MEC competition set is not strictly cross-validation. The MEC competition set, however, is not strictly a generalization set either. A generalization test indicates that the sampling for the calibration (estimation) set should be restricted to exclusively new experimental conditions [18,19]. As discussed above, both the estimation and competition sets followed the same problem structure, same problem selection algorithm, and similar population of participants. In fact, upon analysis of human data in the MEC, we found no differences in the entry rate between the estimation set (55%) and the competition set (54%) (t(78) = 0.225, ns, r = 0.03). Similarly, we found no differences in the efficiency between the estimation set (−0.17) and the competition set (−0.21) (t(78) = 0.099, ns, r = 0.01). Furthermore, the alternation rate between the estimation set (22%) and competition set (25%) did not differ (t(78) = –1.479, ns, r = 0.17). These results suggest that the MEC's estimation and competition sets were similar on all three dependent measures. We, therefore, extend the tests of the IBL-same and IBL-different models with a more challenging generalization process: We use problems that are structurally different from the problems used in the MEC.

The similarity of the problem structure between the MEC and TPT allows us to generalize the models from the MEC to the TPT without significant changes in the working of the model. In order to execute both versions of the IBL model in the TPT, we adapted the number of problems (from 40 in the MEC to 60 in the TPT per set), the number of trials (from 50 in the MEC to 100 in the TPT per game in each of the two sets, estimation and competition). Although the problems in the MEC involve foregone outcomes, and these are absent in the TPT, our IBL remains unchanged: Outcomes in IBL are processed in the same way, whether obtained or forgone. The difference in the availability of foregone outcomes in MEC and TPT is captured by the IBL model in the following way. Simulated participants for TPT generate fewer instances—and reinforce them fewer times—than simulated participants in the MEC. Thus, behavior emerging from the IBL model comes from the same cognitive processes assumed in information processing. The only adaptation necessary to evaluate the IBL model of the MEC in the TPT was the calculation of the gap in the inertia rule (Equation 2). Given the absence of forgone payoffs in the TPT, the calculation of the Gap(t) in Equation 2 changes to:

Equation 2—used in TPT: Gap ( t ) = ( | outcome Enter ( t − 1 ) − V Enter ( t − 1 ) | )

There are other ways in which we could modify the model to fit the TPT data better. For example, the propensities used in the MEC for the first choice would not apply to TPT where the games involved no market context (see section 3.4). Similarly, the inertia mode could be modified to capture the sensitivity of inertia to the lack of foregone payoffs in the TPT. However, the nature of generalization involves testing models in new problems without modifications. Thus, although we acknowledge that our IBL model could perform better in the TPT if we made some changes, we only pursued those changes that were strictly necessary to run the model in the TPT.

We ran both versions of the IBL model in the TPT problems, pooling the 60 problems in the TPT's estimation set and the 60 problems in the TPT's competition set. The IBL-same and IBL-different models were run using the same parameters as the MEC estimation set (detailed above). The IBL-different model was run four times on each problem, i.e., each time with a specific set of parameters representing each of its 4 players.

We compared the two versions of the IBL model using two different statistics across the 120 problems: the average proportion of risky choices (R-rate) across the 120 problems and the average proportion of alternations (A-rate) across the 120 problems (the average in both the measures was taken over all trials and participants in each of the 120 problems). The R-rate and A-rate in the TPT are similar to the “Entry rate” and “Alternation rate” in the MEC.

Table 3 shows the MSD values obtained by the IBL-different and IBL-same models using the R-rate and A-rate measures across the 120 problems in the TPT. The table also shows the values of the MSDs obtained for each set of parameters representing each of the four players in the IBL-different model. The Total MSD is the sum of the MSD determined upon the R-rate and the MSD determined upon the A-rate. The “average of all four players” is not the average of the MSDs calculated for the four players in the first four rows of the table; rather, it is the MSD obtained by averaging the R-rate and A-rate over the four players and then calculating the MSD between the averaged R-rate and A-rate and the corresponding R-rate and A-rate observed in human data. Results shed light onto the parsimony-flexibility trade-off. The simpler IBL-same outperformed the more complex IBL-different in this generalization: The MSD upon the R-rate, MSD upon the A-rate, and the Total MSD for IBL-same are smaller than the corresponding MSDs for each of the players of the IBL-different model and also lower than the average MSD values of the four players. Detailed values of the R-rate and A-rate statistics for the IBL-same, IBL-different, and observed human values per problem are included in Table A3 in the Appendix.