Posted on: Tuesday, August 3, 2004
Test scores weren't compromised by flaws, firm says
By Derrick DePledge
Advertiser Education Writer
Harcourt Assessment Inc. made two minor scoring adjustments to correct flaws on the standardized tests given to Hawai'i public-school students last spring, a company official said yesterday.
The Texas company, which creates the tests Hawai'i schools use to measure progress under the federal No Child Left Behind law, found that student performance had not been compromised by 45 flaws it found on the math and reading exams.
Most of the flaws involved directions, sample questions or printing mistakes. The company initially believed that only one actual test question contained an error, but subsequently found an error in another question, and decided to give students small bonuses for both questions.
The discovery of the test flaws was disappointing for Harcourt and the state Department of Education, which relies on the tests to track annual school progress under the law. Sixty percent of Hawai'i schools failed to make their goals under the law in the 2002-2003 school year, and administrators are awaiting the results of last school year's scores to see if schools have made any gains.
No Child Left Behind requires schools to make annual progress toward having all students proficient in core subjects by 2014, and schools that consistently do poorly face sanctions.
Schools have just started to receive the test scores, which are available to parents but are not tied to student grades. Statewide results will not be released for several weeks. Since no major corrections were necessary, schools that fail to make their goals apparently will not be able to cite the test flaws as a factor.
John Olson, vice president of psychometric and research services for Harcourt, assured state and school officials yesterday that the results are accurate. The company looked at whether the flaws influenced how students answered questions, using patterns from previous student answers and other statistical models as guides. An outside vendor double-checked Harcourt's findings.
"We didn't see that they had an impact on student performance," Olson said.
Harcourt, which has testing contracts in more than 20 states, has a five-year, $20 million contract with the DOE. The company has apologized for the errors and is in negotiations with the DOE on paying a penalty.
The company also will be more careful about following its quality-control procedures, Olson said, which will include staging mock tests so staff can detect any flaws.
Selvin Chin-Chance, who directs student testing for the DOE, said that while the scoring adjustments were not extensive, the state and Harcourt are taking the experience seriously. "It doesn't mean that it wasn't bad. It was bad," he said.
Reach Derrick DePledge at ddepledge@honoluluadvertiser.com or 525-8084.
