Reply #54: Two years ago, I got to look over the kidlet's CSAT Standardized test

[haele]

for advancement to 10th grade.
5% of the social studies/civics, and reading comprehension questions were vague enough that the question had two possible correct answers out of the four, depending on what "learning level" you read the question at. Someone with a higher level education, or who read a different textbook series or even just read voraciously, could answer the question a different way Only one of the correct answers would be scored correctly, so if she picked the "wrong right answer", she'd get it wrong.
Guess that was meant to be tricky, to see which teachers and schools were teaching the proper curriculum, or which ones were being too, well, independent.
Out of the grammar and reading/writing comprehension questions, there were at least 2% that had no right answer.
A couple science questions looked as if they were being asked on the wrong level. Also, if the kid was taking a different science sequence, he or she could have easily not had a course that some of the questions were being asked about, and just have to rely on guesswork.
The math questions were pretty standard, not much you could mess up on those.
Overall test typos - about 2%.
Throw in a glitchy computer or an incorrect keycode for the test, and you can easily fail an honor student who was just having a bad test day - a 80% pass can easily become a 70% fail with a subjectively developed standardized test that has a full 10% development error built in.
And yes, they allow a certain percentage of test answer errors in these tests - even higher level tests like the ASVAB, SAT, and LSAT are allowed some version errors in their current incarnations. Especially since more and more developers depend on "Spellcheck" rather than pay a subject matter expert to go over the final draft in detail.
As for taking an important test several times and getting progressively worse? Well, we're getting into the area where it's not a matter of mental smarts but emotional smarts. Teenagers are notoriously "sensitive" - even the good ones that struggle with the dreaded esteem issues or with a higher functioning mental illness like some forms of OCD or autism spectrum can succumb to a failure loop reaction when they hit a nasty surprise like failing when they were expecting to at least pass. If you've never watched someone hit an emotional wall where they just can't pass that one multiple choice test that doesn't allow them to explore the edges of the question or give them any feedback, but could easily write seven 10-page essays on each of the subjects, then it's probably very easy to say "they don't know the course material well enough".
Standardized tests don't really measure anything but a rudimentary awareness of the subject under test and the ability to take a multiple choice test.

Haele