How valid/useful is IQ as a measure of personal intelligence?

Great Debates
21

I’m not certain myself what exactly IQ measures, but whatever it is, it seems to be highly consistent when I consder myself as an example.

The test was administered to me by trained specialists; I was selling my brain to science to make extra cash in those lean post-graduate and grad-student years, and the test was part of a battery of tests to establish baselines for some-or-other study protocol. Good money! I got a PET scan and MRIs for nothing! Plus, free IQ tests.

I scored somewhere above average on the IQ test each time, always within the stated margin of error about my mean score, I came to find. I took the test a few times over the span of about 3-4 years

My test scores always gave the same result: Virtually equal aptitude for verbal and mathematical problems, roughly speaking. As it so happened, my verbal and math scores on the SAT were exactly the same. Before I learned my IQ, I was a humanities and science double-major, graduated with honors, and got about the same GPA for each major (within .1 points).

22

ambushed,

Please do not confuse reliability with validity.

IQ tests are quite reliable. This means that the results are reproducible. What is unclear is their validity. Are they a true measure of the sought after g?

The classic way to illustrate the difference is to think of an archer shoting at a bullseye. If he clusters all his shots tightly but all half a target away from the bullseye then he has good reliability but poor validity. If he is scattered some right around the bullseye, then he may have marginally poorer reliability but much better validity.

Reliability tells us nothing about validity. We do not know the validity of IQ tests for measuring g if such a critter exists or has even been clearly defined.

And btw, I see nothing in that SciAm article that supports your continuous contention that “nearly identical values of g can be derived from just about any test, no matter what type and however culturally biased or unbiased.” Only that it is reliable foe all groups that are English speaking born in the US. Care to point out what I missed? Dare I say it? Cite, please.

That said, I hardly doubt that some test can be devised that is indeed reliable across cultural groups. And that it would be measuring something. That said something might very well correlate somewhat significantly with various outcomes in Western society of today. That said something seems to be genetic to a large degree, and epigenetic to another large degree. Truth is that my role as a parent on my children’s IQs is very slight beyond conception (and thus little at all for my adoptive child) and that it declines as my child reaches adulthood. Peer groups have some influence but genes and random events make up the bulk. But I can influence how they use the tools they have been given. That’s enough responsibilty for me.

My main point remains: g is a naive concept and is so poorly defined that commenting on its validity is impossible. “A general mental ability”? “A broader and deeper capability for comprehending our surroundings”? Which aspects of our surroundings? Our success biologically is both due to what we do percieve and spend energy thinking about, and what do not, what we filter out. Someone who is biologically more focussed on one domain may be potentially superior in another. Definitition of what is salient to the individual is key.

If defined as those aspects that are salient for success in today’s Western culture schools and society then IQ has some predictive value. But only some. Look more carefully at your SciAm reference. Link to the correlation illustration. 50% of the population has an IQ between 90 - 110, and 20% have an IQ 110 - 125. Yet the differences in outcomes for these significant discrepencies is often slight: a few percentage points difference for most outcomes.

And no evidence has been offered that shows that IQ has any validity for predicting performance in non-Western cultures.

23

Together, these statements seem a bit obtuse. You seem to accept the predictive power of IQ tests in Western cultures, and then go on to imply it has not been “validated”. What standard of validation are you after? Universal applicability? If so, I doubt very much that any test could measure up to that criterion. If you chose a validation endpoint that can be assessed in the real world, I fail to see how IQ scores could be called “invalid” or “not sufficiently valid”. They certainly appear to provide a measure of something, reliably and reproducibly, and those measurements do seem to correlate well with socioeconomic success in Western cultures. What’s missing here? Whether you call what the IQ test measures “intelligence” or some other quality is basically a problem of semantics; there is no practical issue if all you are interested in is probabilities of certain outcomes.

In my estimation, the critics of IQ tests use these semantic traps to attempt to discredit what is known empirically about the test, which are associations that ought no to be in dispute. If we were talking about goblins, and we came up with a testing scheme that we claimed would tell us which goblins in Tasmania would likely grow up to be the best hunters of mugwumps, and further demonstrated empirically that the test had predictive power, nobody would complain. They might point out that the test hadn’t shown predictive power in the mugwump hunting patterns of New Caledonian goblins, but that wouldn’t be cause in and of itself to call the Tasmanian case “invalid”. “Not universally applicable” might be a better critique, but then you run into the problem of possibly not ever being able to define what a “universally applicable” test ought to measure.

24

Are you sure? It seems the Neurometrics IQ Cap, mentioned by Tom Wolfe in his essay quoted above, would be universally valid and free of any cultural bias.

25

Beyond the “Your Soul Just Died” article (which seems to be little more than a deliberately inflammatory polemical front for this BrainMachines “biometric kung fu” nonsense), I can find nothing more substantive about this “IQ Cap” thing; hence, I remain highly skeptical. If there exists some more info. about this, perhaps some good data published in a peer-reviewed journal of neuroscience to back up its purported utility, I might give the idea more credence.

26

Linked is an April 2004 paper from two professors, one does his work in economics and the other, psychology. The paper discusses economic correlations with averaged national IQs with a conclusion that national IQs correlate with per capita GDP, they hesitate (sort of) to say why. Read the conclusion. I suppose this is similar to the correlations of individual IQ to economic success within the United States. The text of the paper is 22 pages, with 10 pages of charts and so, I haven’t had time to read the entire thing – only the first nine pages and the conclusion. This is posted here not because this paper indicates a positive correlation between IQ and economic achievement - but because the paper indicates this exists worldwide.

As you can see, the file is .pdf so you’ll need a reader to open.

http://www.siue.edu/~garjone/JonesSchneApr.pdf

27

I did some Googling and could find nothing about the “IQ Cap” or this “Neurometrics” corporation, except for several links to online versions of Wolfe’s article. So I started a GQ thread: “What ever happened to the Neurometrics “IQ Cap” IQ-testing device?” – http://boards.straightdope.com/sdmb/showthread.php?t=278544 We’ll see if anybody knows anything.

28

That IQ cap seems to fall close to what I have been telling people for years. There is such a thing as a universally testable IQ. The test had not been developed yet and yet, while the IQ cap seems to fit the bill, I will stop short of saying that it is the test I had been referring to.

I proposed that an IQ ceiling (IQMax) is genetically set and can not be exceeded. Everything after conception plays a role to either a)lower a person’s IQ in relation to IQMax, or b)maintain the present level of IQ of the individual in relation to IQMax. Under ideal conditions (Whatever they are), which includes diet, upbringing, social interaction, and a host of other factors, a person could theoretically achieve a perfect IQ, meaning that their IQ = IQMax. This person could have an IQ of 85 and would nevertheless be considered the perfect intellectual specimen.

IQ can manifest itself in different ways in different societies, depending on what is valued culturally. However, even though cultures and societies change through time, the g is fixed and cannot be determined by writing a test, although it can be approximated through testing to varying degrees. I agree with DSeid on this point.

I have done quite well on IQ and other aptitude tests, but I am starting to think that I am less and less “gifted”. Rather, I am starting to believe that I am really good at getting tested. You see, I have a photographic memory and the ability to retain vast amounts of information. When I take a test of this type, I can draw upon the hundreds, or even thousands, of similar questions I have encountered in the past. At times, though not too frequently, I come across an identical question. Other times, there is a close similarity between questions so I can arrive at an answer far faster than normal. I think that this is the reason that I score so high on tests.

Am I smart? I think I am smart enough to get by.
Can I solve complex problems? Yes but if I have never seen the problem before, it takes me a while. However, more often than not I will eventually solve it.
Can I inflate my test scores by referring to my “cache” of knowledge testing questions? In my opinion, yes.

The point is that in order avoid any type of skewing, it appears that the measure of intelligence must be taken from the mind directly or from the human genome. Paper testing, in my opinion, can not be universal because people create IQ tests to test for a specific manifestation of intelligence.
Just my uninformed opinion.

29

Loopy,

I’m sorry if it seems obtuse. I’ll try to clarify. It is less semantics than pointing out that the op’s question is not very answerable.

Validity requires a clear understanding of what the item that you are trying to measure is, and an ability to see how accurately your test measures it. To say that a test measures g you must first be clear what g is. Reliabilty has no bearing on the question of validity.

Our current definitions of g are very vague. I would in fact take the position that g is a naive concept poorly defined. It not only fails to correlate with any particular biologic factor, the presumption of its existence without convincing evidence hamstrings the emergence of a more sophistating understanding of brain function.

The brain is not an all purpose device. It works with sensory mechanisms to filter and focus on particular aspects of available input. It is fairly fixed in its processing of some predictable aspects of the environment (hard-wired so to speak which gains processing speed at the expense of adaptability) and very flexible in other aspects where rapid changes of behavior in response to changing situations has been advantagous. From perceptual input to meta-cognitive processes and in reverse we have a variety of cognitive domains that function interdependently.

We must define with precision if we hope to judge validity. The only meanigful definition of intelligence is how well an individual solves novel problems in pusuit of salient goals, and to discuss it precisely you must be clear about what sort of problems and what is salient to the individual in question. It appears that what is of interest is those behaviors that correlate with academic and economic success. Here we can say that IQ tests have validity to the degree of explaining as much of a third of the variance between individuals within Western society (see Tiger’s first post for the cite). The national averages article notwithstanding we have been offered no data to show us a similar validation outside of Western society.

Now within Western society, for which IQ tests have been shown to have very good reliability and some modest degree of validity at measuring the cognitive skills associated with academic and economic success, how useful are IQ tests? (The second portion of the op question.) I would say fairly little. The vast majority (two thirds) of the inter-individual variance is from factors other than IQ, or at least IQ as measured. A test that fails to account for the majority of the variance is of little usefulness for individual decisions, especially when you have past achievement to use as a better predicitor.

To bring this into a practical focus. I’m a doctor and our site has four docs. Within a few years our most senior partner will retire and we will be looking to bring a new doc in. Will I want to hire the applicant with the highest IQ? Would that information be useful to me? No and no. I want someone intelligent enough sure, but I want someone who understands how much they don’t know even more than someone who knows a lot. I care that the individual knows when to ask for help and knows who to ask. I care that (s)he knows how to get along with the rest of us and to communicate well with our patients. A sense of humor and an appreciation of the absurd would help. A superior range (or gifted) IQ is less predictive of a new docs successful integration into our practice culture than a host of other habits of mind and personality traits. And I know that the sorts of intelligence to succeed in a general pediatric practice are not the same as those needed for being a professional quarterback nor a champion chess master nor a physicist. I want the kind of intelligence needed for this particular task.

Does that help clarify?

30

We know who the fastest runner in the World is. There’s an example of a ‘valid test with universal applicability’.

My hypothesis: IQ test measure how good you are at IQ tests.

Your evidence: People’s IQ test scores provide a measure of something, reliably and reproducibly.

Suggested test: Have a group of subjects (of various backgrounds) take IQ tests. Split them into two groups. One group get extensive practice with IQ tests, the (control) group doesn’t.
Retest both groups after the practice.

Predicted result: The practice group will have improved their IQ scores significantly more than the control group.

31

Sorry, but the Rigeur de Science you splashed on this morning has worn off.

I mean, you must be joking. These tests are designed. Then they’re normalized so that people get an average score of 100, because a priori the average IQ is 100! Doh!

So when they give these “unicultural” tests in Africa and the people there (ostensibly with zero test-taking practice, if they are literate at all) get really crappy scores, which do you conclude:

A. Those Africans are mentally inferior.
B. The tests don’t really measure innate intelligence.
C. A mixture of A + C.

Both A and C are politically incorrect nightmares as choices, and you have to accept that that is what you are getting into when you claim that IQ tests “really measure” intelligence. It is the inescapable conclusion.

My take is that the cognitive differences between different ethic groups are so largely influenced by cultural and environmental factors that the genetic component is impossible to measure. But I won’t say that there are respected scientists who think otherwise.

Well, I recommend seeing the movie Anchorman again.

32

Er, C should be,

“A mixture of A + B.”

I hate it when I mess up in Excel…

33

Well, no, it doesn’t. Everyone loves to point out all the things the IQ test can’t do as “proof” of its lack of utility; then whenever someone points out IQ test can do some things, they claim those things are useless. I might point out to someone that hammers are not good tools for tightening screws, but I wouldn’t then expect them to reply that therefore, hammers are useless. If folks have no use for what IQ tests can predict (things which I think can only achieve reasonable power when applied to populations), then state IQ test are useless for that application. If the test is misapplied, of course it will be useless; but I don’t think it can be said that there are no good applications of the IQ test. Unfortunately, the entire issue has been muddled by politics.

34

Hmmm Loopy, I’m not sure I can make it much clearer. You are no longer questioning the point on validity vs reliability, so I’ll assume that you’ve got that now. Limited validity for intelligence as narrowly defined with good reliability.

Therefore I’ll limit my comments to the usefulness question. I did not say that IQ tests are “useless”. Just of little usefulness.

What potential use are they?

Perhaps solidifying the concept of some general all purpose intelligence, g would be useful from the perspective of understanding brain function? Nope. There is no evidence that there is any such factor. Instead we have solid evidence for multiple cognitive and emotional faculties that are interdependent and competitive with each other at the same time. Understanding those complex interactions is not served by the simplistic concept of a general all purpose intelligence. Better to study how the brain functions performing different tasks at different ages and figure out from the data how the brain divides up the tasks when, then to name a skill and try to identify what part of the brain does it. Or to presume that it all reflects some unidentifiable g. Opening up the mind requires an open mind.

Perhaps they can help in an admission or employment decision? Nah. Not predicitve enough and too general. Better to test for more specific skills and achievements and traits and to judge by the past record. Together these are likely to be more predictive than an IQ test. Sure, if all I had to go on was IQ test results, I’d hire the verbal IQ gifted person for the writing job, and the performance IQ gifted one for the engineering position. But in real life I have better information to judge on.

How about in helping decide on educational interventions? Yes, here IQ tests can be useful as part of a picture. They can help identify a significant discrepency between types of intelligence and suggest remediation or work around strategies. They can help teachers differentiate the kids who are slackers or underachievers for attentional reasons or those who are bored to tears from those who are low across the board and really have a hard time getting the concepts and thus different additional evaluations and interventions become indicated. So on.

So one real use. Do you have any others to suggest?

35

I could go on all day about uses, hypothetical and real. But even if there is only one, is that not good enough to justify the test’s existence? I’m not sure what point there is enumerating the uses of any standardized test, especially one that never was responsibly applied as a universally-relevant biometric. Simply stating over and over again that it is not such a beast is proof of nothing other than lack of universal applicability. Again, so what? You can’t interview job applicants with an IQ test? Use something else. What is the fundamental problem here? Why must g or any other such quantity be defined for a test to measure a quality with relevant correlates for certain investigations?

In my first post I noted I had participated in clinical studies (at the NIMH, actually) where, as part of a battery of tests, the IQ test was administered. I do not remember precisely why the scientists wanted that data point (in one case for a study of obsessive-compulsive disorder, for instance), but there could be any number of reasons. If I showed a low score, it might imply a neurological disorder, or any number of other problems that might eliminate me from the study. If scores tracked with some ther metric, it might be an interesting correlation worthy of further study. As it turned out, the scientists did tell me that most OCD sufferers had above-average IQ, yet there was tantalizing evidence of certain specific deficiencies among OCD sufferers that might provide clues about the neurological basis of the disease. So there you are, IQ as a diagnostic and a correlate in a study of a psyciatric disorder with hypothetical neurological bases. It’s interesting in and of itself if there are relationships. Might I point out that because some OCD sufferers have high IQ but do worse socioeconomically that this is itself an acceted indicator that the disease is socioeconomically debilitating? Is it necessary to then trivialize the test with its purported uselessness in providing a sufficiently-sweeping overview of human cognitive phenomena and thus rob scientists of a perfectly servicable metric with measurable correlates?

36

Uh Loopy, no one has said that, like, the test should be rounded up and shot, dude.

My comments at least have been basic and answering the op’s question: How valid is the test and how useful? That’s the “what” of the “so what” - it was the op. Once again, my answer: somewhat valid as long as one appreciates the precise meaning given to the word “intelligence” by using the test, but of little usefulness. Used with an appreciation of its limitations is just fine by me.

Your comment raises an interesting aside however. The outcomes studies define success in particular ways. People with OCD might even be quite successful by many of those metrics - see the book Strange Minds for a selection of high IQ/OCD individuals who have been among our most successful scientists, for example - but may do poorly on some metric of contentment or happiness. It would be interesting to see a comparison of correlations between IQ and some measure of EQ and perhaps other particular domains and satisfaction with life (rather than drop-out rate, years of schooling, etc.).

37

BTW, the book is actually Strange Brains and Genius by Roger Pickover. Just in case anyone is interested.

38

Do they do well as detectives? :slight_smile: