Imagine that you have a child, or a student in your classroom or in your life, and something suggests this child has a low IQ. I've worked with children who appear not to understand what people say, who are very forgetful, or who don't appear to be learning to read. Sometimes, parents and teachers just say that they have "strange ways" or something else that they can't really describe. When I talk to their parents, one of the things that has struck me is that the parents have usually already done a little testing whether they realize it or not.
If we imagine a child who appears very forgetful, and he or she must be forgetful to the point that there is a real question about succeeding in school, then most parents investigate. They might ask children whether they remember things that happened to them, or to recall and tell them about what happened in TV shows. In other words, they take a little sample of behavior that can demonstrate intelligence or memory, and they observe whether their child can do it. Other parents are never this formal, but just try to observe and recall little bits of behavior and make conclusions from these. When a teacher says that she's worried because little Abby didn't remember the state capitals she just learned, a parent might reply that everything is fine because little Abby does remember the street where grandma lives. In order that we have something to call it as we compare it to other approaches, let's give this kind of testing a name. After a chat with my mother, I'm going to go with "The Parent Administered Test of Smarts" or "PATS" for short. This kind of informal assessment is fine most of the time, especially since most children are typical and most worries from parents and teachers sort themselves out. However, PATS is not enough for a more serious evaluation that can have significant consequences for someone's future.
The first shortfall of the PATS is that there is no way to compare the results to other kids. Years ago, I lived with a roommate, a dog and a snake. On the Fourth of July one year, my dog did as dogs will do when fireworks started going off and he ran under the table in a panic, hoping to curl into the smallest ball he could. My roommate asked if the dog would be OK, and then (and this was very kind of him) also checked about the snake. He asked if he might also be worried by the noise and if we should open his cage and check on him. I was surprised and replied, "snakes are deaf." I know it's mean, but I was shocked that he could be so dumb. Then of course, I told the story to other people (gossiping is bad, I know) and it turned out that I was the dumb one. Lots of people don't know snakes are deaf (it goes with the territory of most people never having taken care of a snake or being interested in them) so when I would tell people about this thinking that it was a funny story about how my roommate could not have known such a simple fact, everyone just said "oh, I never knew snakes were deaf." I was mistaken because I didn't have a good impression of what people know. This is a normative sample. When an IQ test is made, it's given to hundreds or thousands of kids at different ages so that we can actually say with confidence what percentage of kids at what age can answer what questions and solve what problems. Without this step, PATS testing a child can't really get at whether the questions we ask are something that most kids that age will know, or are too hard or too easy. Even after testing a lot of kids, I still always need to check what is expected at what age. Being sure what to ask or how hard something should be is a very difficult thing and not something to be estimated when it's important to get it right. Once we figure out exactly what to ask at what age, there's also the problem of exactly how to ask it.
It is incredibly difficult to really test a child. If you ever get the chance to sit in a classroom or watch a child and parent interact, a positive interaction will usually have a lot of give and take and lot of teaching. The last time I bought groceries, I saw a mother point to the peaches and ask her daughter (who looked younger than five) if she knew what they were. Her daughter at first said "apples" and so the mother did what good parents do and went through asking her what colors apples were, reminding her of times they'd eaten peaches and otherwise gently reminding her daughter that these were peaches, until her daughter very happily yelled "peaches!" That's wonderful, adorable and a completely appropriate way to interact with a child. It's hard not to want to help when a child doesn't know the answer to a question. It's also hard to know how to ask a question in a way that won't influence whether or not a child with know the answer. When a parent looks back on the PATS testing they've done, there is no reason to expect that he or she has asked a question without a hint, or without giving a little support (it really is an unusual way to talk to a child). I've also observed that parents and teachers will give this little bit of help, or will accept a lot of answers that are maybe a little off and when I talk to them about it, they generally just remember the interaction as "I asked the question and the child answered." This is why it's such a hard skill to administer an IQ test. Training involves how you interact and the tests themselves give very, very specific ways to ask questions, to describe tasks and (when you are allowed to help) the help you can give. Some are as detailed as having you say something like "what do you call this piece of clothing?" which is very different from "what do you call this thing?" or "what do you call this kind of shirt?" Without this level of structure, there is no reliable way to know what exactly you asked.
There is another problem with PATS testing that is one that psychologists have had to work very hard to address. Thankfully we can explore this one with waffles (I love waffles, so I had to include this example). Dr. Jim Laidler was an early proponent of untested Autism treatments such as gluten free diets and chelation (a treatment for heavy metal poisoning). To be clear, Dr. Laidler is a very smart guy (he's an anesthesiologist, and has published in high level academic journals). He also has two sons with Autism Spectrum Disorders. He had both on strict diets and schedules of supplements and kept careful notes of their progress and response to them. Then one day, on a trip to Disney World, the family went to a buffet. Dr. Laidler and his wife brought boxes of special food for their sons, but in a moment away from his parents at a buffet, one of Dr. Laidler's sons ate a waffle. Dr. Lailder has written several places about how he expected to see an awful outcome from his son eating a big pile of sugar and gluten, but nothing happened. I think his wife may have been less surprised because it turned out that she'd been taking the boys off of all of their supplements and untested treatments anyway. Slowly, Dr. Laidler realized that he had been doing careful observations of effects that weren't really there (http://articles.latimes.com/2009/dec/07/health/la-he-autism-parents7-2009dec07). It's just that hard to really carefully record someone's behavior (no matter how many degrees you have). If you aren't incredibly careful and systematic about recording behavior, or if you're motivated to see a particular outcome, you can miss a lot. A test is a sample and not a whole day, but it's one of the very few ways to get a reasonably objective record of behavior. Each test is administered with a "Protocol," which is a form on which every response of a child is recorded and noted. Without it, I can't imagine trying to write out or remember every response. It'd be too easy to remember "oh, he got three of those right" and not think about the four he got wrong. Alternatively, I've been surprised when children who were sluggish, quiet or had teachers really worried, would slowly but surely get a lot of things right.
A lot of people I meet are not thrilled with the notion of "tests." I think high stakes testing in schools has a lot to do with this, or just the idea that some people are bad at tests and there are certainly legitimate concerns there. At the same time, I have a lot of trouble imagining a convincing argument to abandon tests. If you ever want to know what someone else knows, what alternative is there to asking questions or challenging them to puzzles? What an IQ test does, is go about this systematically, with consistent tasks and fair comparisons to a large sample of other people. It's not magic or a secret knowledge some of us have, but it's an acknowledgement of how hard it is to figure people out.