Edinburgh Research Explorer Whither Intelligence Research?

: Today we have many exciting new technological tools that allow us to observe the brain and genome and lure us into new kinds of studies. I believe, however, that we will not be able to make truly effective use of these tools until we understand better what it is we mean to measure when we measure intelligence, how it develops, and the impact of the clear presence of gene-environment correlation on its development.


Introduction
I feel honored to have been asked by the Editor, Paul De Boeck, to write a feature article for the Journal of Intelligence articulating my views on the most important outstanding questions and promising new directions in research, attempting to understand and harness the construct commonly recognized as human intelligence.It creates a welcome obligation to find the time to put my own over-cluttered and disorganized house of thoughts on the topic in better order, with the bonus that I receive some feedback on those thoughts from colleagues in the field, as well as possibly the opportunity to influence their thoughts and activities.I begin with some ruminations about the challenges the topic presents for coherent research, and follow these with some suggestions of specific topics on which we critically need further work in order to advance our understanding.We live in an era of rapid advances in technological capacity and we are dazzled and fascinated by what it produces.With respect to intelligence research, our rapidly accumulating abilities to mine large quantities of data, and to observe our own genomes and the structures and activities of our brains, are particularly relevant.I fear, however, that we may be too much like a six-year-old eager to drive a car: unable to see over the dashboard or reach the brakes, limited in use of peripheral vision, and lacking the prefrontal cortex development to maintain the necessary focus of attention and impulse control.In other words, we lack the necessary conceptual foundations to make use of all our new technological tools wisely.

Challenges Inherent in Intelligence Research
Intelligence is a construct.That is, it is an idea we hold in our minds about a driver of observable behavior rather than a tangible, fungible property we can observe directly.It is also a complex idea, about a property of our minds that contributes not only to our behavior on a day-to-day basis, but also, somehow, to our potential for future behavior.This combination of complexity and inability to observe directly has two major implications that I want to stress, without meaning to disregard or deny a host of other implications.First is that, though there is general agreement that intelligence is multifaceted in the sense that it is manifested in many different ways, each of us holds individual and no doubt slightly different definitions of it.Jensen infamously said that "intelligence, by definition, is what intelligence tests measure" [1] and took quite a bit of justified flak for this lapse into circular reasoning.But at some level he was right: we decide what we want intelligence to mean when we decide how we are going to measure it and the uses to which we are going to put those tests.There are, however, lots of tests and they do not measure exactly the same things.Again to generalize, there are two main approaches to measurement.One is to test intelligence by aggregating results from several to many rather diverse but specific tasks; the other to make use of a single highly complex task.Even researchers who take the same approach, however, do not make use of the same tasks for aspects of intelligence to which they assign the same labels.Persistently, none of this really matters: all the tests correlate, indicating they are tapping into the same property.But they never correlate perfectly, and their correlations are low enough to be sure that measurement error in each test is not the only factor making the correlations less than complete.At the same time, if you aggregate two separate bunches of intelligence tests together and consider the variance common to each bunch, the correlations between those two common-variance bunches are effectively complete [2][3][4].So we are all 'sort of' talking about the same thing when we speak of intelligence, but the devil is in the detail of the 'sort of' i.e., those differences in our individual definitions.This is probably especially the case when it comes to the parts of our individual definitions that relate to how intelligence, measured like anything else only at discrete points in time, contributes to our potential for future behavior.This brings me to the second major implication of the complexity and only indirect measurement of intelligence: its strong, though never perfect, predictive validity is one of the hallmarks of the construct, one of the major indicators that at some level we have 'gotten it right'.However, if something is to have predictive validity for future highly variable behavior, there has to be some sense in which what is here today will also be here again tomorrow, or next year, or 20 years from now.We can only measure intelligence indirectly through performance on some kind of task, which means display of some kind of skill: skill at working through tasks requiring application of logic, skill at making a perception and quickly pressing the right button somewhere, skill at drawing meaning from printed material, skill at storing actual facts or even nonsense in memory, whatever.But skills are always built through exposure and practice, making use of course of underlying abilities or capacities, but heavily reliant on opportunity for exposure and practice.Some skills are more helpful than others in building other skills, but each skill improves with specific practice in that skill and each skill deteriorates when practice in that skill stops, independently of changes or stabilities of underlying abilities or capacities.We are never testing raw intelligence; we are always testing some combination of prior exposure to the specific task material, prior exposure to other relevant material and situations, ability to retain, and ability to make the practice opportunity offered by the current testing situation 'count'.This is where things get really messy.Newborn babies have few if any skills and essentially nothing of what we later test as their intelligence, but they begin to build skills immediately.This initial absence of skills, coupled with immediate and constant acquisition of new skills and subsequent dropping off of unused skills, is another prominent feature of intelligence: anything we might use to measure it develops throughout the lifespan, growing rapidly in youth and falling off in old age, but not remaining static even in that middle period of adulthood.We can see this clearly in particular skills, but all the tests we use most often to test what people call 'general intelligence', which I will at least here equate with IQ, have been calibrated carefully and specifically to avoid showing how the actual skill sets change with age, that is to have the same means and variances at all ages.In other words, the scoring hides the changes with age in the actual behavioral manifestations of intelligence.This is intended because the tests are designed as if they can capture some unobservable yet fixed capacity, the one we call intelligence.For some people, this idea of fixed capacity as one of the components of its definition is inherently repugnant and inaccurate; for others the fixed capacity idea is just as inherently fundamental to the definition.The fur can only fly fast and furiously when this topic comes up for discussion because we really do not know to what degree either of these camps is correct and those in between can speak only too vaguely to satisfy anyone.
Those who see the fixed capacity property as essential to the definition of intelligence point to the robust evidence for substantial genetic influences on mental test scores of all kinds, the strong correlations that the scores show across long spans of time, and the lack of success of even very intensive intervention programs in raising IQ scores with any degree of permanence.Those who see the fixed capacity idea as inaccurate point to the facts that people who tend to score poorly on cognitive tests also tend to have experienced socioeconomic deprivation, that all scores benefit from prior exposure to the material that is often relatively lacking in those who score poorly, and that test anxiety and 'stereotype threat' impede immediate performance.
But just how do babies, and the rest of us too, build new mental skills?We can say many superficial things about this process, but nothing very concrete.At some level, building new mental skills requires focus of attention to the appropriate learning stimuli, shaping some kind of response to those stimuli, receiving feedback of some sort on the success of that response in generating the intended outcome, and subsequent tries at remedying whatever was not so successful in the initial response attempt until we get it 'right'.From the get-go, this means access to the appropriate learning stimuli for sure, but it also means some kind of motivation to focus on those stimuli and the ability and motivation to make effective use of whatever feedback on response success is available.Access to more rather than less applicable feedback helps of course, but most of this motivation and use of feedback stuff comes from somewhere inside the individual and it too shows genetic influences.Add to this that, beginning at ever-younger ages, we basically force certain kinds of skill learning on young children, in the form of 'working and playing with others' and academic schooling.To some the contents of this material are apparently inherently engaging in their own rights and they seek out additional related stimuli even after the school bell rings for the day.For others, however, this is far from the case, and they escape as fast and furiously as they can as soon as they're dismissed.It's not at all clear just how intelligence-ascapacity (whether fixed or not) is involved in this, but motivation's involvement is clear for sure.It is also clear that unfortunate correlations among stimuli opportunities, task performance success, and motivation develop rapidly in young children.
Neither intelligence nor educational research has recognized the conundrums inherent in this collection of observations and ruminations, and I do not think the field will make much progress in understanding much of anything about intelligence until we confront these conundrums head-on.What follows are some, at least I hope, more specific observations about what will be necessary in order to do so.I consider early development and its individual differences, the role of feedback about task success in fostering motivation to pursue intellectual engagement broadly defined, the extent to which task practice is specific to task, gene-environment correlation, and changes in brain use arising through use of technology.

Early Development and Its Individual Differences
In thinking about early cognitive development, developmental psychologists have tended to think about normative patterns: what kinds of specific cognitive skills do typically developing one-year-olds, two-year-olds, etc., display?How do those skills change as children advance from age one year to age two years, and so on?From this perspective, little consideration has generally been given to the extent to which different performances on different cognitive tasks correlate, and individual differences among apparently typically developing children of any given age are generally considered noise.This stream of research has been very separate from that of researchers attempting to develop early measures of IQ, who typically look for early measures that correlate well with later-administered IQ and achievement test scores standardized to the same scale for each age.Neither of these approaches address, or even recognize, a fundamental question.This is: how does variance in cognitive skills change with age?The few studies out there that have addressed this question [5,6] suggest that children with better skills at any given age tend to show greater skill development from one year to the next than do children with poorer skills at that age, so that variance in skill performance increases sharply from one age-year to the next.If generally correct no matter what kind of educational exposure or social structure, this has huge implications for educational policies mandating equality of student performance across groups of students, as well as for understanding the kinds of educational programs and resources that will be necessary to meet the omnipresent but never clearly stated twin goals of public education systems.These twin goals are to inculcate the basic cognitive skills needed to function as a productive and self-sufficient citizen in society to all children, and, at the same time, to allow all to reach their intellectual potentials.
At present, however, this whole topic has been given so little research consideration that few even are aware of the question of how variance in cognitive skills tends to change with age, never mind the absence of any clear answer and the importance of the implications if the shreds of evidence we do have are robust.Much could be done observationally to change this, but, unfortunately, the kinds of experimental studies needed to disentangle the involvement of imposed social structure from that of genetically influenced selection into social structure are very difficult to conduct from ethical and political perspectives (not to mention time-consuming and expensive), and their difficulties are only compounded when considering the kinds of experimental studies needed to evaluate effectively whether forms of educational streaming could help.Simply addressing the question fully through observational studies would help considerably though.
A related important developmental question that has rarely been addressed is to what extent changes in children's apparent cognitive capacities from one year to the next are associated with direct exposure to new and/or accumulation of material rather than to some kind of inherent brain maturation.Most researchers studying increases in children's cognitive performance with age tend to assume that the increases can be attributed to inherent brain maturation, but this is not necessarily the case.Studies of expertise have made clear that experts can hold much larger volumes of information in working memory at a time than can people who are not experts.For example, chess experts can hold whole boards and even series of moves on those boards in working memory using the same working memory capacity that non-experts require to hold even the positions of a few pieces.This has made clear that as we become more familiar with material we are able to store it in meaningful chunks rather than bit by bit.This additional chunking capacity that comes with greater familiarity with material has not generally been taken into consideration in studies of children's growing working memory capacities with age.This applies as well to individual differences in apparent working memory capacity within ages: some of what may appear to be differences in raw working memory capacity should possibly be attributed instead to differences in familiarity with, and thus chunking of [7], the material involved in the task: vocabulary words, numbers, shapes, etc.
Similar points need to be raised and explored about cognitive decline at the other end of the lifespan.

How Does Feedback about Task Performance Affect Motivation and Development?
Intelligence tests were developed originally to measure educability.Though the limitations of thinking of intelligence solely in these terms have long been acknowledged, we have yet to create valid measures of intelligence that do not correlate rather strongly with academic achievement, and access to higher education is often rather directly dependent on performance on tests that correlate highly with overtly intended measures of intelligence.Our conceptualization of intelligence is thus tightly intertwined with, though not equivalent to, academic success.Schooling is required of children throughout the economically developed world, and has been now for several generations.Society in general holds it to be important, and children pick up this message quickly.Many, but not all, parents strongly reinforce it.Moreover, from early ages children are expected to sit quietly, listen to and follow instructions, and carry out academically oriented tasks.However teachers and parents phrase their responses to children's performances on those tasks, I suspect children learn quickly not just how they performed on the specific tasks, but what the adults around them think about their performances.Do the adults seem impressed and pleased with children's speed of success, or do they seem frustrated and to be forcing themselves to be patient with lack of success?Children also learn quickly about the patterns of their successes: do they succeed quickly and easily, often even on the first tries, or do they tend to struggle and experience many failures before eventually succeeding?And as soon as they go to any kind of school with other children (which is very early for many), they see not only their own patterns of success and the responses they get from adults, but those of other children and begin to compare.I suspect that these patterns build quickly into children's self-concepts about their academic abilities, which may as well read intelligence.I also suspect that, whatever what is stable and inherent about lifespan intelligence might actually be, the self-concepts about it that children form early tap into it [8].At the same time, the existence of those very self-concepts may be reinforcing and accentuating those stable and inherent individual differences.How this is involved in the development of what we call intelligence is something we need to explore and understand much more thoroughly.
Carol Dweck (e.g., [9]) and her colleagues have written much about 'fixed' and 'incremental' personal theories of intelligence.They maintain and have demonstrated experimentally that people who believe that intelligence (as they call it; they really mean task achievement, but then so always do people purporting to test intelligence) is incremental, that it improves with effort and application, tend to achieve more, persist longer in learning new skills, and be more willing to take on new learning challenges than do people who believe their intelligence is fixed.According to Dweck and her colleagues, these latter poor souls not only achieve less, etc., but tend to stick closely within activities they already know they can perform well both to prove yet again that they are capable and out of fear that failure on some new task will reveal the 'inherent' limitations of their abilities.This sounds like a fine recipe for anxiety to me, for children growing up in a society that expects them to spend half their waking time acquiring and performing new skills.
Dweck is making a very valid point: if we know anything about skill acquisition in psychology it is that time on task, read 'basic practice', leads to eventual mastery, for everyone.Anything about attitude and motivation that contributes to willingness to spend more time on task is going to bring results.What is missing from Dweck's analysis, however, is recognition of those persistent individual differences in rates of skill acquisition, which are just as persistently tied to those robust and quite relatively stable individual differences in level of performance at any one time that we call intelligence.Which children are more likely to develop a belief that their abilities respond well to effort?It seems only too likely that it is those who master assigned tasks quickly relative to the others around them.We need to explore and understand much better how this is involved in the development and maintenance of intelligence throughout the lifespan, which leads us straight back to the need to understand better how variance in mental task performance changes with age.

Can Task-Specific Practice Improve Intelligence?
Questions about belief in the efficacy of effort also lead us straight back to the conundrum that, however hard we try to measure this intrinsic construct of intelligence, we always only measure task performance.Though performances on all tasks robustly show correlations, practice with specific tasks has generally been frustratingly specific to task: it does not tend to transfer to improved performance on other tasks.Such transfer is of course necessary if we are to identify tasks that can be used to enhance or maintain intelligence effectively, as many recent studies have sought to do.These recent studies have tended to focus on working memory tasks, based on the strong correlations between performances on working memory tasks and measures of general intelligence.Results have been, at best, mixed, in part due to the difficulties of establishing clear control groups.I suspect, however, that this general approach is inherently limited by the arbitrary nature of working memory tasks.They always subtly rely on familiarity with the material to be manipulated in working memory, but never in any conceptually coherent way.
I suspect that this lack of conceptual coherence is what limits their potential for transfer.That is, the material to be retained is usually rather generally familiar but otherwise arbitrary, such as configurations of common shapes or the last words of random but otherwise ordinary sentences.Thus, performance level relies on extent of meaningful familiarity with the material to be manipulated to some degree, affecting encoding and maintenance.Improvement with practice, however, mostly involves development of strategies for processing arbitrary material to be repeated later but then purged from memory, as it holds no meaning.A more fruitful approach might be to try to identify the broad principles that draw together bodies of knowledge we want people to be able to use meaningfully over time, without purging [10].This has not generally been explored in much detail, but it does appear to have generated some real success in at least one area: the importance of thoroughly understanding the number line for young children's development of arithmetic skills.This was demonstrated in a series of studies by Robbie Case and colleagues [11], and has been reinforced in many subsequent studies seeking to understand what is amiss in children who struggle to master grade-level arithmetic [12].Understanding just how mastery of a conceptual tool like the number line transfers to practical arithmetic skills (still needed by all despite our modern technological computing prowess) may help to identify more such conceptual tools that can then be taught.
Another study that should be telling intelligence researchers more than I think it has was carried out by Hart and Risley [13].They visited families with different levels of socioeconomic status on several occasions, essentially observing just routine family activities as naturally as possible and counting the sheer numbers as well as variety of words addressed by the parents to the children.Their results were striking in that parents of high socioeconomic status tended to address many more and more varied words to their children than did parents of low socioeconomic status.Subsequent work has indicated that this socioeconomic difference in environment extends to the kinds of topics parents tend to discuss with their young children.Parents of high socioeconomic status also tend to ask their children more exploratory questions, tend to explain concepts and reasoning more thoroughly, and tend to stimulate their children to try to do likewise.Hart and Risley pointed to this richer verbal environment in homes of higher socioeconomic status as one of the major reasons children in such homes tend to develop larger vocabularies and greater intelligence (measured using any recognized intelligence test), and subsequent studies have made similar attributions.It is of course possible, even likely, that the kind of general mental practice this environmental stimulation fosters acts in some way similar to the number line in transferring directly to many more specific tasks, but the authors of these studies have generally ignored another strong possibility.Parents of high socioeconomic status tend to have higher general intelligence, measured however you want, than do parents of lower socioeconomic status, and the evidence for substantial genetic influences on intelligence is strongly robust.This makes it all too likely that the tendency for higher intelligence in children of parents of high socioeconomic status has some genetic roots.Taken together, the studies of environmental enrichment in families of higher socioeconomic status and the studies showing genetic influences on intelligence indicate that genetic and environmental influences on intelligence are correlated [14][15][16][17][18].

What are the Implications of Gene-Environment Correlation?
It is clear that genetic and environmental influences are correlated because brighter parents, who pass their genes for intelligence to their children, also tend to provide them with more intellectually stimulating environments.Behavior geneticists call this kind of gene-environment correlation passive, because the children do not have to take any initiative of their own to experience this environment.But it is also clear that genetic and environmental influences are actively and evocatively correlated: that is, those who are brighter, at least partly due to genetic influences, also tend to seek out more intellectually stimulating environments and activities themselves, as well as to obtain more encouragement to do so through positive feedback from others [19][20][21].It is not at all clear, however, what these correlations mean for development of intelligence in young children, nor for maintenance of cognitive function throughout the lifespan.That is, if we were able to manipulate exposure to the environments that generate these correlations, would distributions of intelligence in the population look different?
Gene-environment correlation is the elephant in the room, the perennial political hot potato, that which shall not be named, because if it matters for the development of intelligence, it violates all our democratic ideals about equality of creation, social mobility, and access to education to equalize opportunities in our societies that strongly reward cognitive skills.We do not understand its effects because we do not study it.Not only do we not study it, but we do not even note that we do not study it.Its omission from consideration is glossed over and minimized in behavior genetic studies in general, and, in particular in studies of moderation of genetic influences by socioeconomic status (e.g., [22]), where it is likely most relevant.Gene-environment correlation is generally not even mentioned in studies seeking specific genetic variants.For example, if there really are genetic variants robustly involved in educational attainment (e.g., [23]), then we have some serious population stratification for those genes, but this implication and its downstream consequences are left unspoken.I suspect that we will not understand too much about intelligence, functionally, within the brain, or genetically, until we investigate the effects and implications of gene-environment correlation head-on.This means investigating the development of intellectually-related interests, motivations to seek intellectually stimulating environments, persistence in the face of learning challenges, etc., as well as 'sheer' intelligence, whatever that is.

Changes in Brain Usage
The Flynn Effect, though robustly observed, remains poorly understood, but I think it should be telling us something of relevance to the implications of gene-environment correlation, along the lines that Dickens and Flynn [24] have proposed.They have been thinking what amounts now to historically: about the observance of the increases in test scores throughout the 20th century, but we are now somewhat past that, and there has been a massive explosion in day-to-day use of technology throughout economically developed societies since 2000.There is also some evidence that, within those societies, the Flynn Effect may be leveling off (e.g., [25,26]).Most studies investigating whether the Flynn Effect applies to general intelligence have concluded that it does not (e.g., [27,28]).This suggests that the Flynn Effect applies only to some aspect of practice with the skills we use to measure intelligence and not to whatever is inherent about intelligence itself.
Since the turn of the current century, it has become a frequently noted source of amusement that young children are much more adept at using technological devices than are older people, even teenagers just ten years beyond young childhood themselves.This should give us a big clue about the source of the Flynn Effect: it arose, and is still arising in economically developing countries, from increasingly direct exposure to the kinds of mental tasks that are used to test intelligence.I suspect this is also the reason it may be slowing or even ceasing in economically developed countries: as we make increasing use of technology for information storage, access and communication, we are changing the ways we use our brains.The new usage patterns come easiest if learned young, when much about specific neurological structure is still under development.What is not at all clear, though, is whether the new usage patterns, and their accompanying neurological structures, actually represent more effective, efficient, read 'intelligent' use of our brains.We cannot tell because the tasks on our intelligence tests do not require these new patterns: they are stuck in what passed for demonstration of intelligence in the twentieth century, not the twenty-first.This warrants further exploration and understanding, but the way forward is not to design tasks that do require the new brain usage patterns, as that would only be to set the definition of intelligence as something like 'ability to tweet.'We will have to come up with an ideal definition of intelligent performance in order to evaluate whether the new technology is good for it or not.
If it is correct that neurological structure develops in childhood in a way that becomes relatively fixed, and the structures in young children growing up with technology are rather different from those of us who learned to use it much later in life, then we have also to accept that there are likely important cohort differences in brain structure and function, as well as important experience-driven individual differences within cohorts.This would help to explain the difficulties we have experienced in 'finding general intelligence' within the brain, as well as the difficulties identifying the genes involved in it.It is also the reason I said at the beginning of this essay that I fear that we lack the conceptual tools necessary to make use of our new technological abilities to observe the brain and the genome, either structurally or 'in action'.I put my own money on understanding the development of what we measure as intelligence, and the role of gene-environment correlation within it, as the most promising way forward.In order to do this, I think we need some more basic external/behavioral observation, and conceptual work that addresses the conundrums and topics I have discussed, before we will be able to make truly effective use of our fancy new brain scanning and genetic tools.