Measuring STAAR Readability Remains “Inconclusive”
Follow-Up UT Study on Grades 3-8 STAAR
Again Finds Tests Align with their TEKS
The TEA has just released the second (final) part of a legislatively mandated report on grades 3-to-8 STAAR tests that looked at the 2020 assessments, and reached the same conclusions as Part 1 of the study released in December that examined the 2019 versions of the same assessments. See TEN, Dec. 9, for a summary of Part 1.
The bottom-line conclusions for both reports by researchers from the Meadows Center for Preventing Educational Risk at UT Austin is that:
- TEKS alignment — The overwhelming majority of test questions on the 17 assessments were determined to be aligned to the content standards (TEKS) of the subjects tested.
- Test question readability — The researchers could not determine if the readability levels of the test questions were written above the expected comprehension levels for the students tested.
“Because we did not have confidence in the 2019 item-level results, we concluded that analyzing item readability in a reliable manner was not possible. For the 2020 assessment, we conducted several of the same analyses on a sample of items and found the same pattern emerging,” the final report says, adding:
"Therefore, we again concluded that analyzing item readability in a reliable manner for this report is not possible.”
Not Measured: “Difficulty”
Meanwhile, in both Parts 1 and 2, it was noted that the researchers did not (because they were not asked to) determine the difficulty of the test questions.
“An analysis of item difficulty requires a different methodology than an analysis of readability,” the researchers said in identically worded sentences in both parts.
HB3 Mandated
The study was mandated by 2019’s HB3 in response to three prior STAAR readability studies — two from Texas A&M Commerce (in 2012 and 2018) and another study from the University of Mary Hardin-Baylor (in 2015) — that concluded that some of the tests included questions with readability comprehension levels that were above the grade levels of the students taking the exams.
The release of the final part of the study prompted state Rep. Matt Shaheen, R-Plano, to announce (Feb. 14) that he plans to lead an effort in the Legislature to replace the STAAR tests with “an innovative alternative that maximizes student academic potential.” (Shaheen is running unopposed in his Republican primary race, and will face a Democratic challenger in November.)
Shaheen said his push to replace the STAAR with something else was formed by the pre-HB3 research, flaws that he said had occurred in the Meadows Center research, and conversations he has had with hundreds of teachers.