What is `Reading Level?'

Factual Questions
1

This site:

http://www.sph.emory.edu/WELLNESS/reading.html

Shows a method for determining reading level which strikes me as somewhat arbitrary. You count the number of polysyllabic words in 30 sentences, take the square root and add 3.

Even assuming that polysyllable count has anything immediately to do with the grade at which a person could, should or would be reading (which I’m not prepared to grant), why are we supposed to take the square root and add 3? Looks to me like the equation is designed to generate a number within the proper range, which is convenient, but suspect. What has the number generated got to do with anything real? How do they know?

2

Most likely, it’s a statistical technique used to get the data to fit into a linear model (i.e., one which is something like y = a*x + b). Linear models are highly desirable because they’ve been studied at length and we can use them to make predictions. Just a guess.

3

Is there an actual study out there that justifies the particular curve they’re trying to mimic, then?

4

Presumably, although I don’t know of any. My background is in statistics, not education, so what I said above is pretty much all I know.

5

You’re taking the notion of “reading level” more seriously than the people who made up this method. Here’s the sort of discussion that probably went on in creating the method.

Person 1: I need a simple method by which someone can determine the reading level of a written passage. You know, one which will tell us which grade level the passage is appropriate for.

Person 2: Hmm, that’s not going to be easy. At the moment, we do this by having an expert on reading education read the passage. They have a pretty good intuitive knowledge of how difficult a passage is to read and thus what grade level it’s appropriate for.

Person 1: That’s too complicated. I need a way to do this so that anyone with a high school degree can do this with no more than a couple of minutes of looking at the passage.

Person 2: Well, what the experts are doing is looking at the words and the grammar used in the passage. Let’s see, we can create a dictionary of all English words with their reading level and an annotated grammar that would allow us to mark the passage for its . . .

Person 1: No, no, no! That’s too difficult. I don’t want some huge program that only a computer could do. I want a simple system that would allow a person to determine the reading level. It can’t be something complex that would take a long time to learn. The person should be able to learn the system in fifteen minutes.

Person 2: Hmm, well, I suppose you could just use the number of polysyllabic words in the passage.

Person 1: What’s a polysyllabic word? One with more than one syllable?

Person 2: No, it would probably be better to use three or more syllables as the cut-off. I’d have to ask a statistician to figure out the correlation between the number of words and the grade level.

Person 1: Well, do so.

Person 2: Statistician, can you create a formula that relates the number of polysyllabic words and the reading level.

Person 3: Sure. Take a bunch of medium-sized reading passages. Have your experts go through them and use their intuitions to mark their reading levels. Give me the results.

Person 2: O.K., here you go.

Person 3: Well, I looked at them, and it appears that the best thing to do is to count the number of polysyllabic words in any 30 sentences. Call this number n. The reading level is 9.876n + n3.841 + ln(n) + . . .

Person 1: No, no, no, that’s too complicated. You’ve got to simplify it.

Person 3: Well, I can’t use a linear formula. That doesn’t fit the curve at all. I suppose I could just use a second-degree equation. Are you assuming that these people can square a number?

Person 1: O.K., but it’s got to be squaring a whole number and adding a whole number. It’s too complicate to use decimals or fractions.

Person 3: Well, O.K., here’s a formula using just squaring and adding whole numbers. It’s really not very accurate.

Person 1: Who cares about accurate? I just need a method that any bonehead can use. Nobody’s going to question the answers that the formula gives. These people think that any number that comes out of a mathematical formula is obviously accurate. Remember, these are people gullible enough to think that IQ scores are tremendously precise.

6

There are many types of reading levels, but only two basic types of numeric reading levels. The first is grade level, and this is the type being measured by the formula in the web site given. The second type gives gradations within a narrow band of readability, particularly among emergent readers. This type might divide reading materials intended for K-1 students into ten or twelve categories. Since the OP asks about the first type, I’ll address that here.

The most accurate way to gauge reading level (reading ability needed for basic comprehension of material) is to conduct a multi-tiered cross-sectional study. You would give said reading material to a large group of people of varying reading abilities whose reading levels are known, then determine which among the group are able to comprehend the material. The most advanced group in which a statistically significant sampling of readers would be able to comprehend the material would represent the reading level of the material. This would necessitate having a sufficiently large number of readers at each reading level being tested to achieve a statistically significant sample with a small enough standard deviation to be useful.

Put more simply (and I am grossly oversimplifying here), you would need to do this: Give a valid and reliable norm-referenced reading test to a large group of first graders. Take all those who score within one standard deviation of the mean. These are your first grade reference group. Anything that a statistically significant portion of this group can read is first grade reading level or less. Repeat with second graders, third graders, etc. until you have a representative group for each reading level for which you would like to test. (for my thesis, we stopped at eighth grade, but you could theoretically go up to, say senior year of college and call that year 16, although differences in reading ability tend to be greatest in the early years and diminish as students get older. There’s a huge difference between first and second grade reading level, but relatively little between seventh and eighth.) After giving the same reading material to each group in succession, you find the highest group that understands it, and you have your reading level.

This would be highly impractical for anyone who isn’t pursuing a doctoral thesis, so the next best thing would be to find a way of analyzing the text itself to determine reading level. There are certain things we know about what makes text more or less difficult to read through studies such as the one described above.

Words: Longer words are harder than shorter ones. Less common words are harder than more common ones. Words that follow regular pronunciation rules are easier than irregular ones.

Sentences: Simple sentences are easiest, followed by those with compound subjects, compund predicates, compound clauses, complex clauses, and finally compound-complex sentences are most difficult. Even within simple sentences, a large number of modifiers increases reading level, as does the complexity of such modifiers. Non-conventional sentence patterns increase reading complexity. Sentence length matters, but not as much as complexity.

Context: The complexity of the ideas presented, and the manner in which the presentation is made affect readability.

Readability formulas try to take into account some or all of these factors to create a way to numerically analyze the text without testing it on actual readers, and are more of an attempt to predict readability than to accurately measure it. The formula you list is using sentence length and word length, and to a lesser degree sentence complexity (longer sentences are more likely to be compound or complex) and word commonality (most common words are one or two syllables). Basically, what the researchers who develop these formulas do (and again, I am greatly simlifying due to my being a reading specialist, not a statistician) is fool around witht the numbers until they come up with a formula that they think would approximate what the results from a cross-sectional study would be across the broadest possible array of reading levels. Such formulas tend to have a “sweet spot” where they are most accurate, and the further away from this sweet spot you get, the less accurate the prediction.

The formula listed is one of the more simple ones, but should be helpful for determining relative readability, ie whether one set of reading material is more difficult than another, but I would suspect it’s simplicity makes it of little use in determining absolute readability.

Upon preview, I see that Wendell has described one way this formula may have been arrived at. I suspect his description may not be far off.

7

Wendell Wagner wrote:

Actually, my concern is precisely that people do in fact take these reading levels' seriously. I have many times encountered serious discussions based on some notion of reading level’ without any hint that the notion is problematic. For example, magazines like Time are often criticized for being written on an eighth grade reading level. But they’re not. No editor says, `Keep this on an eighth grade reading level.’ Magazine articles are written for concision and clarity, which is a skill kids need to learn, and they can learn it well from good examples. But some yutz decides that it’s on an eighth grade reading level, and implies that this puts it beneath high school students, that’s basically pissing away a valuable tool that can be used to make students into better writers. The system is not only arbitrary, it is taken at face value, and tends to assign normative value to the spurious claims it makes.

Number Six wrote:

Yet, another problem would seem to arise from this, because reading levels' are used as a basis for claims about acceptable reading levels. If that's based merely on norms, with no basis to judge the sufficiency of these norms, then the system begs the question. Older textbooks just on science that I have seen expect a lot more from a reader than modern science textbooks. Were the norms too high in the 30's, or are they too low now? (I realize that norm’ is probably one of those mathematical terms that mathematicians cringe at the sloppy lay usage of, but hopefully you see what I mean.)

8

Johnny, you’ve hit upon one of the thornier questions of educational testing. Should “reading level” be a descriptive term or a prescriptive term? In other words, should we use “third grade reading level” to describe what the average third grade student is able to do, or what the average third grade student should be able to do.

In my doctoral thesis, we were studying the usefulness of subject area textbooks (science, social studies, etc.) by comparing the readinlg level needed to comprehend these books to the reading levels actually present at the grades for which the textbooks were intended, ie, can a typical seventh grader read and understand a typical seventh grade science book. For this purpose, defining reading level by norms was the relevant way to go.

BTW, the study concluded that the average social studies and science text book was written at roughly two grade levels above the grade written for. Fifth grade science books at the time were typically written at the seventh grade reading level.

I agree with much of what you say here. Criticism of Time based on reading level is misplaced. “Written at an eigth grade reading level” means that an eighth grader can understand it. It does not mean “aimed at eighth graders” or useful only to people who read at that level. Presenting complex information in such a way that it can be understood by the lay person is harder than presenting it to a peer group. David MacCauley is a master at this. My fifth graders love The Way Things Work, Building Big and his architechture construction books such as Cathedral and Pyramid. Though aimed at middle schoolers, they are an excellent source of information for any lay person interested in the subjects being discussed. Isaac Asimov’s non-fiction science and history books inevitably required lower reading levels than most of his competition. That doesn’t make them any less useful as tools of study.

Being able to explain complex things clearly lowers the reading level of the written material. It is not necessarily indicative of lack of value to those who read at a higher level.

9

and

(emphasis mine)

Don’t you mean “least” and “lowest” there, Number Six? I can understand The Cat in the Hat, but that doesn’t make it 19th-grade level (and I’m not even the most advanced person who might be reading it!).

10

D’oh. Of course, Chronos is correct.

When testing an individual’s reading level you take the highest, or most advanced, level at which the person is able to comprehend the material, and that is the individual’s reading level.

When testing reading material, you take the lowest, or least advaced, group that can read the material, and that determines its reading level.

I somehow confused the individual testing with the reading material testing. My bad. Thanks for the correction, Chronos.