Problems with using “IQ” in…anything

In the interest of actively keeping a blog, I thought I’d elaborate on a topic I touched on in my last post about the potential perils of eugenics.  In particular: inborn ‘intelligence’.  Hopefully in the future I’ll keep these relatively short so people actually care enough to read them.  This one goes on a tad (spoiler alert!).

First off, as a disclaimer, I don’t study intelligence.  I study memory and learning.  I do study cognitive processes though and I guess learning is tied in pretty heavily with intelligence studies.  Anyway, my point is I think my arguments here are ripe to be argued.  And since I am not an intelligence expert, some passersby might find what I say unoriginal and under-cited.  I welcome comments.

Part of what spurred this post was an interesting link to a graph tweeted by my friend and NYU CNS colleague Pascal Wallisch.  I have not discussed this with him yet—so am not actually sure his intent in this case—but suspect HE is a bit suspect (maybe I should use less homonyms in the same sentence for those foreign readers out there?) of what this graph really measures.  I went ahead and embedded it here:

motionquotie

(Citation: http://dx.doi.org/10.1016/j.cub.2013.04.053)

As you can see, this graph shows a clear correlation between people’s ability to suppress background motion and do well on a “standard” IQ test.  The interpretation in the linked write-up is that suppressing background motion is very similar to ignoring distractors, specifically in this case ignoring the world at large around you while focusing on smaller objects in the foreground.  On its face this makes a bit of sense: separating the wheat from the chaff is an important everyday task that successful people are likely to excel at.  But (I should probably just call this blog “but…”) there are so many confounding factors here (as there can be with any correlation without evidence of causation).  I’d like to explain what I think some major ones are as exemplars for why ‘intelligence’ isn’t so easily quantifiable.  I should point out that there’s nothing wrong with this study per se, I’m more using it as an example of the perils of correlating anything to IQ as a “standard” measure.

1.) The obvious one first: measuring one’s IQ.  I haven’t taken any full, true IQ tests, but think I get the gist from my CAPT/PSAT/SAT/GRE days and various fun things I’ve come across online like Wonderlic and IQ test examples.  And I hardly need to have taken any to tell you the problem: they’re written by people just like me.  College-educated fellow native-born Americans.  I’d even venture that they’re largely written by white guys living well above the poverty line.  And since my Venn diagram of experience is bound to overlap quite highly with these test-writing folks, I have a natural advantage on the tests they create.  This makes the claim that doing better on any test not written by someone quite like you (this again) will not help your ability to understand where it’s coming from (are double negatives a sign of intelligence?  Shakespeare used them right?).

A further problem with the IQ-quantification-by-test fallacy: you can get better at them.  This is certainly the reason for SAT/LSAT/all-AT preparatory courses and I have little doubt it’s true for any ‘aptitude’ test.  Practice in the realm of the test—whether spatial or linguistic or mathematical—or even just in the form of similar test questions themselves is a huge component of how well you’ll score.  So, bringing this back to the graph above, simply stating “IQ” is always a rather arbitrary measure, and is likely determined more by environmental factors (like how or where you were brought up) than anything biologically innate.

2.) I get the impression that what the authors of these kind of works are trying to get at though IS this biological innateness.  I think that using a visual task is hardly a way to do this though if your end goal is to inform on intelligence.  You know what makes a great student?  One who isn’t distracted by the background environment when they’re taking a test.  Or when they’re studying for a test.  Or when they’re learning.  Or when they’re trying to enhance how many marshmallows they’re given.  But are people less ‘smart’ because they can’t force themselves to sit down and concentrate on some rather arbitrary measures of their spatial/linguistic/marshmallow-withholding/whatever qualities?  Certainly not.  For whatever reason, biological or environmental (or nonshared environmental—I can think of so many random, unlucky things that could negatively bias people in test-taking situations), concentrating on the specified goal at hand is not one of their stronger abilities.  So for these people, OF COURSE their poor IQ would correlate with an inability to suppress background motion: you’re just seeing the same thing measured twice (or as Ben Gibbard would say, different names for the same thing).

3.) Now, for the speculative (and most fun!) part.  Let’s say on the graph above, instead of “suppresion of large moving objects” on the Y-axis it said ‘reaction time at completing eye fixation task’.  There are certainly studies on this (here’s an interesting recent one from a commenter on the Wiring the Brain blog).  Now we’re getting at something seemingly more innate.  I get the impression what IQ-jockeys want to hold their hat on is the idea that ‘smart people’ are wired better, think quicker and this is where the genetic component of IQ comes into play.  And, for the sake of argument, let’s assume this trend—namely, that reaction times correlate to IQ—exists for people with a similar environmental background.  Like similarly-educated and reared people from the same place.  Or, even better, young babies, wherein you’re hoping to minimize the environmental factors.  In these cases: does an innate (almost wholly genetic) ability to complete a fixation task make them smarter?  Or more prepared for any and all tasks that humans need to accomplish?

I most certainly think not.  Yeah if you’re pushing tin for a living it probably helps to have an innate knack for inputting spatial info a little faster.  But there are so many other things that weigh into how ‘smart’ our decisions are.  It’s entirely possible a faster reaction to such stimuli limits the amount of bandwidth being used for a decision.  Sure for a simple task like looking at where a dot moves this isn’t going to matter.  There are certainly many situations where a too-fast decision without sufficient time for processing could come at a cost though.  In fact, your brain already limits the stream of components that contribute to your consciousness.  It doesn’t separately inform you of what you’re hearing, seeing and feeling in succession.  It accumulates this information, writes a little report on it and then passes it on to upper management only AFTER multiple subsystems pare out what they consider useless information.

A great related example comes to mind: chimpanzees KICK OUR ASS at short-term memory tasks.  This included a test where information was shown very briefly and chimps were able to recall this info way better than some slow-brained university students (of course I’m being facetious, Kyoto University has a rather good reputation).  It took a long time for us to design a task to figure this out—probably because we knew it would hurt our anthropocentric egos—but chimpanzees have brains better wired for such tasks.  We hardly think they’re going to start writing novels, however.  The quick assimilation of visual information is just one of many streams of input information, and just because they answer faster in a relatively complicated task certainly doesn’t give them a leg up on us in a whole bunch of other things.

In conclusion, these are the kind of problems in measuring innate “intelligence” with such tasks as the graph above.  You could just be measuring spurious correlations.  Or you could be measuring something hardly indicative of what we’d consider important for success.  I’d be fascinated to know if ‘successful’ (not IQ-high) people have faster reaction times.  Don’t get me wrong: more data gives us a more complete understanding of our brain, so I hardly think this kind of research is a waste of time (for example, reaction times may prove useful in diagnosing Alzheimer’s).  But did Charles Darwin have enhanced visual reaction times?  (ok maybe—he was a birdwatcher).  Did Albert Einstein?  Did Virginia Wolff (speaking of stream of consciousness)?  I don’t know.  And even if they did: did they also have them as babies?  And most importantly: was this crucial to their success?  My suspicion is what made them geniuses in their chosen fields were brains developed to handle and manipulate multiple high-level processes at once, and not just spit out answers to simple tasks a hair quicker than the average person (but not a chimpanzee).

Advertisements

4 thoughts on “Problems with using “IQ” in…anything

  1. I have read this post, and it seems you don’t know very much about IQ testing. Neither do most people talk about IQ, which is quite infuriating to the seasoned reader.

    There is indeed an environmental component to IQ, but you being a “privileged white male” has little to do with it. If it did, east Asians and Ashkenazi Jews wouldn’t score so high on tests which were supposedly invented to justify white supremacy.

    (BY the way, if you have access to a computer, you have little use for “alternative definitions” of intelligence that are prized by other cultures. Guess which culture pioneered the scientific revolution? Surely not the Aka pygmies.)

    We know little about the genetic architecture of IQ, or of most quantitative traits, in general. But we know what doesn’t contribute much to variation in IQ, once heredity has been accounted for: variation in school quality, income (within a broad range), exposure to toxins in the womb, variation in parenting styles, early childhood stimulation, access to preschool, etc.

    Gee, that leaves… what, exactly? Even being adopted by physicists doesn’t boost IQ, by adulthood! There’s not even much evidence that exposure to crack cocaine in the womb has any long-term impact on IQ, all else held equal. (Where’s the explosion in crime and social decay that criminologists predicted from the cohort of inner city children born during the ’80s? )

    • And before you try and recite any correlation between environmental variable X and IQ (say, income and IQ scores) — well no duh.

      Such is the typical rebuttal that I have come to expect from those who are ignorant of behavior genetics. Correlation is not causation.

    • Nicely stated, misdreavus. I wouldn’t go so far as to say we have “little use for alternative definitions of intelligence”, but it’s certainly worth making use of available evidence about what does/doesn’t influence IQ scores. I don’t see anything wrong with a paper reporting a correlation between scores on a standard IQ test and something else. The correlation is just that: we might even entertain the alternative interpretation that this (perhaps) gives insight into what it is that IQ tests measure.

  2. @misdreavus I think you have a rather narrow definition of what makes a “useful” person. You seem to hold being a physicist in high regard (thank you, I used to be one haha). And yes, IQ tests, because they’re designed to test similar quantitative skill, are likely to correlate highly with physics skill. This is point 1, above. But as jpillow hints at, there are many, many desirable traits in someone and we only need so many physicists. And we have not shown (and I don’t personally believe) that the traits a physicist possesses automatically are transferable to all walks of life. People can also be born with genetic predispositions for good social skills (unlike physicists, trust me) which might be useful in, say, nuclear disarmament.

    @jpillow I was more hoping to use this paper as an example of fallacies with IQ as a strict definition of what makes a person ‘smart’ or ‘useful’ or ‘successful’. I did try and point out there was nothing wrong with this kind of work, even if I think it might be measuring the same thing in two different ways.

    I was also looking to point out my problem with any of these papers that look to relate psychophysical behavior with something vast like ‘intelligence’. What makes the Mischel marshmallow study cool is they followed the cohort across decades and were able to draw significant conclusions from a pretty simple study. Of course, the subjects were tested at 3-4 years old, at which time they had plenty of environmental exposure, making the idea that there is something inborn about delaying satisfaction weak.

    If you did a longitudinal study and looked at the same vast array of traits Mischel did (depression, marrital success, SATs, income, health, etc.) and compared them to something psychophysical from those same people as really young babies: then I’d start believing you’re onto something with major implications. Science obviously works on a grant-scale lifetime, however, making these kind of studies a pipe dream. And personally, as I said at the end, I don’t think they’d show anything.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s