I went through four years of college and took maybe two tests with multiple choice questions. Tests typically consisted of a fairly short question and then a lot of room to work out the answer. In physics you actually did physics, not have 3 near variants of F=ma.
But besides doing it, essay type questions are better at seeing if a student understands a concept. When I taught data structures, multiple choice worked well in seeing if the students understood the differences between various sorting algorithms. I felt I needed an essay type question to see if they understood the difference between a variable and a type. My class, at a not really great college, got freaked out by this, since it seems they had never seen this type of question before.
You can’t really ask people to do computer science in a test, but you can see if they know how to do it.
I’m a teacher, and I’d agree wholeheartedly with Karweenie that there is a contingent of whiny, lazy teachers. It’s nothing personal.
Except that when you’re running a school system, as opposed to one classroom, they can’t just take the teacher’s word for it. There are many, many thousands of school teachers and administrators (and parents) who are entirely happy to “pass” kids who have not in fact mastered the skills. We cannot just trust teachers and schools: there must be some kind of empirical, external, objective-as-we-can-make-it yardstick.
Personally, this is one of the the main reasons I support a free-market reform of the school system; we need to make it easier for parents to pick and choose the school their kids go to, so they can pick one thy do trust to teach the kinds of things they (the parents) care about, rather than having to deal with external mandates from faraway bureaucrats.
Speaking as someone who has taught writing to college freshmen for the last decade, and has had to deal on a daily basis with the shitty basic skills produced by our school system: I would fucking love to God if the schools taught sentence diagramming. Sweet Jesus, I wish they would teach it. No, it doesn’t have much school-external value, but it is an excellent tool for a) helping students think through the logic of grammar, and b) checking that they do have such knowledge and understanding. Especially for boys, it would beat the living fuck out of the airy-fairy personal-expresssion writing that so many middle-school English teachers would rather do.
Standardized testing was the fallout of the grading curve. With such a curve curve it is possible for a child in an educationally challenged area to complete 12 years of instruction with the skills of a third year student (as measured without a curve).
It’s not a function of “teaching to the test”, it’s a recognition of lowest allowable standards. The “test” represents the bare minimum of skill and should not be viewed as the bar of achievement.
When I was growing up, 90% of my teachers were drooling morons, and the other 10% were some of the smartest, funniest, most dynamic, interesting, and amazing people I’ve ever met. (And that’s a way better ratio than among the general population :p:D. I’m kidding, I’m kidding half kidding . . . )
I kind of liked sentence diagramming, too. My use of it as an example wasn’t meant to disparage its use in the classroom.
But you would agree that sentence-diagramming is just a tool intended to help illustrate and teach basic grammar, right? It isn’t the goal in and of itself. When you graduate, nobody cares if you know how to diagram a sentence, they’ll only care that you can speak and write correctly.
Similarly, nobody cares what grade you got on some stupid test in grade school so long as you are competent at those skills you were supposed to have learned. A test is a tool, not the master. We should tailor the test to fit the class and material, not tailor the students and teacher to fit the test.
One big problem that I’ve seen with standardized testing is the fact that multiple choice is such a silly way to approach understanding of a topic. It’s not all that useful later in life as a basis for solving problems. I recall being in high school when Arizona was attempting to institute a standardized test (AIMS or AMES or something) to ensure that students who were graduating knew basic math and english (i think). After a few trial tests where people bombed the mathematics section, the teachers started showing how to just plug in all of the possible solutions until you found the right answer. I could understand showing estimation as a sort of trial and error process, but just trying four random numbers until one worked? How on earth is that an accurate test of someone’s math skills?
Maybe it wasn’t such a bad idea, because once I got to college there was always a slew of multiple choice questions. Again, not that useful as an assessment of the topic at hand. The tests were either ridiculously easy, simple trial and error, or ambiguous and vague in an effort to make them more challenging. I understand why they had to test via scantron (efficiency and the like), but I don’t think that made the tests any more effective as an evaluation tool.
I would argue that these two things don’t follow one another. I’m sure pilots and neurosurgeons take some ‘standardized’ tests, but I wouldn’t ever claim the a neurosurgical residency or flight school was guilty of ‘teaching to the test’. Especially in these two examples there is a lot of practical skill demonstration that either takes precedent or works in conjunction with other forms of testing.
Rather than re-hash the novel I wrote in a previous thread on this matter, thought I’d just link to my own post here. If you wander on to the thread that post is in, you’ll see much more discussion further down.
Essentially my point is: I work for one of the companies that make these standardized tests. For Florida and Texas (I can’t speak for other states), the test is based upon standards that are written by educators in those states.
In other words, Florida teachers come up with Florida’s standards based upon what they want to teach on the ground, every day, in Florida classrooms. Those standards are adopted by the State’s school board and put up on the web for god and everyone to offer opinions before the standards are adopted. The test companies merely come up with tests – that include short-response, extended-response, essays, and multiple-choice items – that measure achievement of the skills laid out in the State-adopted standards.
AFAIC, if Florida teachers want to teach to the test, it shouldn’t make a damn bit of difference. Their curriculum is based on the Sunshine State Standards and the test is based on the Sunshine State Standards, so teaching to the test is really just teaching the State-approved curriculum. More on this from me in the other thread. Enjoy.
My wife has to teach retards long division for two weeks until the ISAT test is over. Then they go back to learning how to count and add. Most of them actually perform worse on the test than blind chance would indicate. Then they go home to get yelled at by their illiterate, alcoholic parents. This is the same test they give to honors students in the wealthiest suburbs of Chicago. I find it extremely difficult to believe it is possible to write a test that is relevant to the educational progress of every single child in a populous state like Florida or Texas, or even a majority.
Sure, you tailor your test to the approved curriculum, teachers and students be damned. If you then pressure teachers to base their teaching styles on that test, you are tailoring the students and classroom to the test, not vice versa. Florida may be able to dictate what students must learn, but the teachers and students themselves are the only ones in a position to figure out how.
Dr. Cube, did you follow my links to the other thread from January?
The curriculum, and the standards, are written by committees of Florida teachers, who take a few days out of the classrooms to serve on the committees so they can come up with reasonable standards, and reasonable curriculum. There’s no “teachers and students be damned.” The teachers are the ones who write the curriculum and the standards and committees of active, licensed Florida teachers review each and every item for bias, sensitivity, grade-level appropriateness, and appropriate measurement of the standard. It takes two years of review and analysis before any single item ever makes it on the test.
The people doing all that reviewing and committee meeting? Florida teachers.
If you follow the links to the previous thread, and follow the links I posted there, you can go straight to the FL DOE web site and download released tests, the Sunshine State Standards, and a document that lays out the entire process from initial item writing through scoring and reporting.
Yeah, but the teachers aren’t writing the test for their own students. That’s the whole point. The teacher knows her students and Florida Dept of Education doesn’t. In making a generic test that will apply to a million individuals, you are, it seems to me, necessarily making a test that is not very accurate at a level below that of perhaps an entire school district. Certainly, the opposite is true. A teacher can make a test that will accurately determine the knowledge of her small class, but that test will not be well suited to any random classroom you handed it to. I contend that the following is true: you can’t make a test that will simultaneously and with equal accuracy measure the aptitude of a single student, teacher or classroom and also the quality of an entire statewide educational system. There are too many variables involved at the school, classroom and family level.
You are absolutely correct; you cannot make a test that measures the aptitude of a single student. That’s precisely why Florida’s test doesn’t measure aptitude, which is a prediction of potential. Florida’s test measures achievement of the standards.
For example, from this site, (This is the site containing all of the Sunshine State Standards) here is a third grade math standard,
Now, I don’t think anyone here can argue that this is an unreasonable expectation, that a third grader should be able to solve multiplication and division problems. It is entirely possible that any third-grade math teacher in this state can approve an item that is designed to measure that standard. Here, kid, what is 12 x 3? If the kid bubbles in “36,” then the kid has achieved mastery of that particular standard. Teach to the test, teach to the standard. Either way, if you teach a kid how to multiply and divide, then there shouldn’t be any problem with veering off curriculum nor success on the test. The state-level tests are not IQ tests and they are not aptitude tests. They are tests that measure mastery of standards, and there is nothing unreasonable about presenting a concept and then measuring if the students in a given classroom mastered that concept. If your state’s assessment isn’t aligned with reasonable standards, and those standards aren’t aligned with the actual curriculum taught in classrooms, then you have a valid concern.
And I can’t understand how you followed all my links, read everything I posted and still claim that teachers don’t write the test. They don’t write the items. But they review every single one. If an item is unrealistic, or the material isn’t taught in the schools, or is biased toward high-performing students, or biased against minorities, then that item doesn’t go on the test. Teachers vetted and approved the standards. Teachers vetted and approved the test items. All of the information I’ve provided here is accessible by the general public and any student, parent, or teacher can find examples of the standards, the tests, and preparatory materials any time they want to. In Florida, at least, the test very much is a reflection of what happens on the ground in Florida classrooms.
The test is not the problem. Measuring achievement of standards is not the problem. Measuring teacher performance based upon a tool that is designed to measure student achievement is using a screwdriver to pound a nail – it’s the wrong tool for the job. Politicizing statewide results is the problem. Paying teachers based on their students’ performance is the problem. If you want to measure teacher performance, then there should be an assessment designed to measure teacher achievement of teacher standards, which is a whole other ballgame that nobody is playing right now. Teacher achievement and student achievement are different skill sets based on different standards and should be measured with different assessment tools.
Another problem with standardized testing is the emphasis that some individual districts place on their school grades or rankings based on collective student results. That is not the fault of the test, which is a valid assessment. That is the fault of the administration in those districts who care more about scores than they do about the actual students in their classrooms. If the districts emphasized tailoring curriculum to the standards and teachers teach the standards (example above), then the test should be a blip on the radar.
Florida recently tried to combat that problem by passing legislation making it illegal for teachers to present test prep materials prior to January of any given school year. This is a state law known as SB 1908 (You have to download a PDF if you want to read SB 1908). The writing assessment is currently administered in February and the math, science, and reading assessments are administered in March; SB 1908 requires that these assessments be pushed back to later in the school year. SB 1908 also called for the addition of a fifth content area in Florida’s assessment toolbox, which means that in the next two years, this state will be assessing American history in addition to reading, writing, mathematics, and science.
Remember that discussion about nobody testing diagramming sentences anymore? Well, guess what? I was reading SB 1908 and just learned that this is precisely what Florida is requiring on its writing assessment:
Now. All this to say, not every state is doing exactly what Florida is doing. All states, however, are required by NCLB to assess student achievement of some form of standards. If you suspect that you may not know exactly what you are talking about, then I suggest googling for your state’s Department of Education and finding out as much as you can about your state’s assessment program. I think it’s likely that you will end up fighting some of your own ignorance. Good on anyone who takes the time to research it out. To Florida Dopers: Follow my links, follow my links, follow my links. FLDOE does not do a good job, IMHO, at its own PR. I am biased, of course, but I think its an excellent assessment program, rigorous and fair. But I think the results should not be misused or misinterpreted to draw any conclusions other than “X number of third-graders at School Y demonstrated that they know how to solve 12 x 3.”
More links for anyone who cares (This is all publicly available information; I’m just helping you get to it, if you are interested.)
Concerned about disabled kids?
Here’s information on Accommodations – this includes developmental disabilities, visually impaired students, hearing impaired students, and students who haven’t quite learned English yet. Florida does not translate its tests into Spanish. If a student is developmentally disabled and cannot perform at grade level, that student is exempt from being required to take the FCAT as long as he or she has a Special Education Plan, and other appropriate assessments are acceptable for reporting to the state. English Language Learners are also given special accommodations, such as a translator, so they have the same shot at success as any other kid.
Exactly who writes this test and how does it go from an idea to a paper test in front of a student?
All the answers to that, explaining the process of teacher committee vetting, psychometric analysis, and more information than you ever wanted to know, ever, is here.
**What’s actually on the Florida test? What are they testing? Is that fair? ** Florida released some tests here. The items within cannot ever be used on the test again because the answers are provided (I think). It is very expensive to develop an item and release it to the public, never to be used again, so most states do not release their tests and Florida agreed – due to public pressure – to release a select few.
**What do teachers use to prep the kids for the upcoming test? ** Florida produces Sample Test Materials each year. I am currently in production for the 2010 assessment sample test materials, which will be released and posted to the site linked to above in January 2010.
What good is all this testing doing? Can Florida prove that student mastery of standards is improving?
Florida produces documents that study student results over many years of testing and analyzes those results. Committees of Florida teachers look at the analysis and provide recommendations to other teachers for areas needing improvement, as well as offering instructional implications (how to teach something better). The Lessons Learned document for the writing assessment will be published mid-year 2010, but reading, mathematics, and science reports are found here.
**How are the short- and extended-response items scored? Don’t kids get credit for creative thinking? **
Why, yes, they do, in fact. These reports show examples of actual student responses to mathematics, reading, science, and writing assessments in Florida. For each possible score point, an example of a student response is analyzed to explain how the scoring rubrics applied to the response.
**Who participates on these teacher committees and what do they do, exactly? **
On this page, select “Participation in State Assessment Committees” (You have to download a PDF). The same list is also provided in more detail, in the FCAT Handbook, here, pp 41–62, or section 4 “Test Development and Construction” (link #9) This is a list of all the committee vetting and psychometric analysis that every aspect of Florida’s test items go through before finally making it to a test.
**How are the test items written? Who writes them? Who decides what is an appropriate item for Florida’s test? **
Here is a link to some of the Test Item Specifications. Florida hires professional item writers (most are former or current teachers) from out of Florida. The reason Florida teachers do not write the items is that of security. You can’t have all the Florida teachers knowing exactly what items will be on the test because some will then drill the correct answers instead of actually teaching kids how to solve the problems, or write papers. These Item Specifications are made available to FL educators, however, who can then write an appropriate item themselves to use as a test prep tool in addition to the other materials provided. The FL DOE, their test contractor, and committees of Florida teachers (who sign their lives away in a security agreement) review the items for relevance to real-world classroom experience. Only reading and mathematics can be found here; we are currently in production for other content areas. The writing assessment does not have test item specifications because the test is a writing prompt and kids write an essay, so there’s no machine-scored bubble test for writing. (Florida did it for a couple years, but it wasn’t a very good way to assess the writing standards, so we dropped it.)
I’ve taught in two test-based education systems (French-style Cameroon and China) for the last three years. I’ve known dozens of American teachers working abroad in a variety of countries.
I don’t know a single American teacher working in these systems who think it is better.
I’ve taught students who spent the last seven years learning English. They literally memorize dictionaries, and can spout off the definitions of thousands of words. They have the rules and exceptions of grammar down pat. They have memorized hundreds of idioms.
And yet, when they come up to me, they can barely form a sentence.
Seven years of study, and they literally cannot talk.
So much of learning is not what you know, but what you can do with it. Students who grow up in test-based education systems know all kinds of stuff. But they seriously lack some skills we consider very important. Most of my university students cannot perform simple “look on wikipedia” style research. They can’t even get to the “swap contact info” stage when assigned group projects.
There is no doubt in my mind that our education system is one of the things that makes America so powerful. We are not taught to memorize- we are taught to find out what we need to know. We are taught how to use our skills in everyday work-like situations (like group presentations- which most countries never do.) And we are taught not just to absorb information, but to process it and draw unique conclusions. And I truly think this is what accounts of American’s flexibility and know-how…which is why we are the outsourcers and not the outsourcees.
As a parent of bright kids, this is really my biggest issue. My kids are prepped for the test (and the tests they take here really appear to be pretty good - lots of logic skills - but no writing at their grade levels) with normal classroom work. But for three weeks before each testing cycle - twice a year - six weeks of a short school year - classroom activity grinds to a halt to prepare for “the test.” Since ours are adaptive computer based tests, we need to make sure all the kids are computer literate enough to be able to click buttons - which actually isn’t a skill all third and forth graders have in a diverse school where a lot of the parents don’t have a computer at home and there is only one in the classroom. We have to make sure they remember the problem solving skills they should have gotten all along. They take practice tests. Science and Social Studies are dropped completely during this time. Because they don’t want too much pressure, and they want well rested kids - the classroom “teaching” day is shortened down to just test prep activities. They spend hours “resting” by watching not terribly educational movies. And after the test, there is - of course - the celebratory party and “day off.”
The other 25ish weeks a year, resources are disproportionally allocated to the kids who are having the most issues passing the tests. Kids who are doing fine are left to self learn, but dozens (literally at my school there are dozens) of specialists work with the at risk kids. From special education teachers to reading and math specialists to occupational and physical therapists to ESL teachers.
Well, the problem with this is that the classroom just isn’t a good place to learn a language at all, at least not to speak the language with facility.
Given the right learning environment–not a one hour a day classroom–I would think it pretty easy to come up with a standardized test for the subject. The problem is the learning environment, not the standardized test.
But my students can’t go to my office hours (where they really would gain useful speaking experience) because they are too busy cramming for tests. All of my individual tutoring, clubs, discussion groups, etc. grind to a halt when test time approaches. Indeed, most of my students see oral English classes to be a drain on their valuable study time because oral English is not easily testable (the only real way is individual interviews, which can never be done on a grand scale.)
It’s not just language, either. My Cameroonian computer students were equally difficult. When I was teaching them how to use Word, they would memorize the use of each and every button. And then when they’d forget, they’d be stuck. I’d have a dozen students a day asking me “Madame, I forgot, which button in Bold?”
Most of us, on the other hand, learn computer apps by using them. We learn HOW to figure out what button do, and how to find the buttons we might need even if we are not quite sure where they are. A person going through the American school system spending an equal amount of time learning would never be asking questions like “Where is Bold?”
