Testing experts say so far Florida's problems with its new statewide exam, the Florida Standards Assessments, are likely not serious enough for the state to consider throwing out this year’s test scores.
Earlier this month a software problem meant students had trouble logging onto the writing exam for several days, and some students who did sign in to the exam were booted out of the system. Later, hackers tried to shut down the exam by swamping test severs with traffic.
Those glitches have led some lawmakers and parents to ask for a return to paper-and-pencil exams. But those same testing experts say Florida shouldn’t abandon computer-based tests at the same time classroom lessons are becoming more high-tech.
“The startup problems that Florida had do not seem to reach a place where you would have to throw out the results,” said Doug McRae, who retired as an executive with curriculum and testing company McGraw-Hill. “I would recommend that you would really need to have upwards of 10 percent of the population affected by problems before you have to seriously consider not using the results.”
It didn’t take long for school leaders to challenge the results of Florida’s new state test when problems cropped up.
“There are more questions than answers,” Miami-Dade Superintendent Alberto Carvalho said at the time.
Parents and lawmakers like Miami Democratic Senator Dwight Bullard started asking a question soon after: If we’re having these problems with online tests, why not just do things the old-fashioned way?
Bullard proposed requiring paper-and-pencil tests until the state and school districts worked out technological problems with the Florida Standards Assessments.
McRae, the retired testing executive, is a psychometrician -- the big word that describes the science of building tests and making sure those exams measure what’s intended. He and other experts say computerized tests have many advantages over paper and pencil.
There’s more flexibility to ask open-ended questions, or have students perform a series of detailed tasks.
The Florida Standards Assessments has audio questions, for instance, or questions that ask students to drag correct answers to a specific place or choose from a menu of options – sometimes with more than one correct answer.
But McRae said the most important reason to use online computerized exams is because school lessons are also becoming more high-tech. Florida has required schools to deliver half of lessons digitally when classes start in the fall of 2015.
When schools modernize lessons, McRae says they need to modernize tests as well.
“If Florida is down the path on getting computerized instruction in place and the kids are familiar with it,”McRae said, “then it would be a disadvantage for those students to do it in the paper and pencil mode."
Schools districts across the state vary in how much instruction is digital. In Miami-Dade schools, some grades have electronic textbooks, and others don’t. Other students get personalized math lessons through the i-Ready computer-based curriculum.
Henry Braun, the director of the Center for Study of Testing, Evaluation and Education Policy at Boston College, agreed. Employers have already made the switch to computerized exams to license and certify employees.
Eventually, computerized testing will improve enough to allow near-constant measurement of a student’s progress.
But right now, computerized testing is still improving. Florida students have taken computerized exams for several years and have had glitches in the past.
Other states have had more significant troubles. Kansas threw out the results of its test last year after hackers disrupted the exam’s trial run.
Studies showed students who took the test during disruptions skipped more questions than those who took the test uninterrupted.
In Florida, students had long waits to sign in to the test. Some were booted from the exam. But more than 90 percent of students scheduled to take the exam completed it.
Braun said Florida needs an outsider to make sure the test results are legitimate.
“I think would be very important to restore confidence, both in the state and the vendor, would be to have an independent group to review what happened,” he said.
He says the state should try to address problems with students or schools individually to make sure the results are fair.
But not everyone thinks Florida students are ready for computerized exams.
“Many of our students are not competent typists,” said Darcey Addo, a Brevard County teacher who is also running for school board. “You can not test what you have not taught… if you want to test whether a kid types well, you need to teach them to type.”
Addo is active in the state’s opt-out movement, which coaches parents on how to withhold their children from state tests. The group argues decisions on student, teacher, and school performance shouldn’t be based on standardized test results.
But Addo says there are more important questions about whether the Florida Standards Assessments are reliable. Education Commissioner Pam Stewart told lawmakers the test had been reviewed and the results were trustworthy.
Addo says she and other activists have asked for those reports – but have yet to receive them.
“We don’t have the data, and we have asked extensively,” Addo said. “But Commissioner Stewart says it’s available, so all we ask for is to see it.”
Lawmakers are raising doubts about whether the new test results will be reliable. They’re considering changing requirements for third-grade reading scores and asking whether the state should issue grades for public schools this year.
The Florida Department of Education said they won’t throw out test results because the cyber attacks didn’t access test data. When repeatedly asked whether the delays caused by software affected results, state education officials refused to answer.
Spokesman Meghan Collins said BUROS Center for Testing will again review Florida’s results. That report should be available in late summer or early fall.
