Formula tweak helps school rankings
The Arizona Republic
Dec. 26, 2004
'Underperforming' sites fell sharply this year vs. 2003
When Arizona ranked 1,203 public schools in
October, the number of "performing" and "highly performing" schools suddenly
swelled, while those ranked "underperforming" plummeted.
In a miraculous turnaround, the number of "underperforming" schools dropped from
136 to 43, including 11 "failing."
The labels released this fall left parents and educators wondering: Did schools
improve that much or were the standards lowered?
The more than 800 schools lumped in the middle, or "performing" rank, have
little in common.
At the top end, there's Anthem Elementary in far north Phoenix, where nearly 9
of 10 third-graders passed the AIMS reading test. At the opposite end, there's
Desert Garden Elementary in Glendale, where just 4 of 10 third-graders passed
the test. Both received a "performing" rank.
Patrick Yennie, Anthem principal, can live with the gap.
"It's not frustrating, it just is," Yennie said. The principal said he did not
receive one parent phone call complaining about the school's label.
"I don't know if there's anything inherently wrong with performing," Yennie
Catherine Stafford is superintendent of the Avondale Elementary School District
in the West Valley and said the complex formula the state uses to rank schools
each October needs to be easier to grasp.
"I want to understand it better so I can
help my teachers," Stafford said. "Every year the formula changes, and we're the
ones that have to stand in front of the (district) board in a public meeting and
Arizona State Superintendent of Public Instruction Tom Horne said there is
always a tension between simplicity and fairness.
As you make adjustments (in the formula) to
make it fair, you make it more complicated," Horne said. "The art of public
policy-making is to find the golden mean between simplicity and fairness."
The search for that golden mean will continue, Horne warned. While the formula
was tweaked this year, it is expected to change dramatically in 2005.
Mark Kemp, father of three students in the Kyrene Elementary district, said he
can't remember how the state ranked his boys' schools. He said parents with time
and know-how help ensure schools are good, while schools in poorer
neighborhoods, where parents are busy just trying to help their families
survive, need extra help. He views the state's system as "political jockeying."
"It's easy for a politician to say we have the AIMS test and now we have
accountability and everything's OK," said Kemp, who called it "papering over"
the real problem of the lack of funds for students who really need the money.
State schools chief Horne credits the improved rankings to hard work by schools
and by the Arizona Department of Education, which sent coaches, known as
"solutions teams," to every struggling school. The teams retrained teachers and
principals and helped write goals that lay out a plan to raise student test
The state agency knew that last spring's test scores were critical to the
rankings: There were 81 schools that had "underperforming" labels for two years.
A third year with the ranking would place those schools on the state's first
list of "failing" schools. That, in turn, would mean costly intervention.
So it was in the state's best interest to keep those 81 schools off the failing
list. It succeeded. In October, just 12 schools were deemed failing.
"It's a tribute to teachers and principals, to the schools and the solution
teams and assist coaches who made the 81 schools their top priority," Horne
Garthanne de Ocampo is the new principal at Phoenix's Emerson Elementary and
credits the state teams for giving her teachers the spark they needed to make
changes. Her school moved from "underperforming" the last two years to
"performing" in 2004.
The principal isn't sure how that happened.
The percentage of third-graders passing AIMS at Emerson dropped in 2004, while
the percentage of fifth-graders increased slightly. The principal said the
state's confusing formula appears to change from year to year.
"It's very hard to understand exactly what needs to be done to continue as a
performing school," de Ocampo said.
State officials said Emerson jumped to "performing" because it improved student
AIMS scores, averaged over 2002, 2003 and 2004, compared with the school's AIMS
scores averaged over 2000 and 2001. It also met the federal standard called
Adequate Yearly Progress this year, which it did not last year.
Roger Freeman is testing director for Paradise Valley Unified School District
and it's his job to know exactly how the formula works. Freeman warned parents
that the formula isolates the state's few exemplary schools and those most in
need of help.
"If people think this system accurately rates schools, their ability to teach
and the students' performance in terms of shopping for the best schools, it
doesn't," Freeman said.
State Department of Education officials admit there's more to Arizona's
miraculous growth in performing schools than good coaching.
First, the state ranked an additional 105 schools. Second, Arizona schools were
helped by tweaks to the formula.
The formula, which works on a point system, used two statewide test scores.
Arizona's Instrument to Measure Standards, a math, reading and writing test, was
taken by third-, fifth-, and eighth-graders and high school students. The
national Stanford 9 test was taken by students from second grade through ninth
Here are the changes that helped create the 2004 miracle:
• Not all test scores count.
The state throws out scores of any student who hasn't attended the school for at
least a year. It throws out scores of disabled students who take special tests
or get help, such as having the test read to them, and it throws out the scores
of elementary students who have been learning English for less than three full
For example, Paradise Valley's Palomino Primary is a "performing" school where a
large percent of children are learning English. In 2004, the state used test
scores from only 29 third-graders out of 169 third-graders enrolled.
In 2003, many schools still relied on students to give personal information,
such as birth dates, gender, and how long they attended the school. Kids used
nicknames, made mistakes, or purposefully put in wrong information that skewed
the results. The state relied on schools to correct the data but many schools
didn't. This year, the state verified more school data through its electronic
student tracking system.
• A school received a "preliminary determination" of its rank a month before
parents and the public learned how their school faired.
During the intervening time, schools could send the state formal written appeals
if they believed their rankings were wrong. The state changed the rank if the
school could prove there was an error in the data, such as missing student test
scores, or if state officials agreed some occurrence beyond the school's control
negatively affected how students scored on tests. That could include bad weather
that stopped school buses or construction that hampered learning, such as
shutting down heating or cooling systems. Of the 1,203 schools ranked on Oct.
15, 82 appealed; 20 were successful.
• A small technical change caused a big jump in rankings.
In 2003, after a school threw out the AIMS scores that didn't count, if the
total number of students tested in 2000 and 2001 was less than 31, all the AIMS
scores for that school's grade and subject, for all years, were not used. For
example, if scores from 29 students counted in a school's third-grade reading in
2000 and 2001, all third-grade reading scores for every year, even 2003, were
not used. So only fifth- and eighth-grade scores were included in the schools'
In 2004, that number was changed to a total of 16 student scores for the
2000-2001 school year. That allowed schools to get credit for scores in far more
grades and subjects. For example, if a combined 29 student scores counted in
third-grade reading in 2000 and 2001, now that same school could use all
third-grade reading scores for every year to help determine the rank.
Jennifer Regalado, Arizona Department of Education's director of accountability,
said the formula is sensitive to small improvements in single grades. Allowing
schools to count far more student scores pushed more schools into a higher
ranking, helped mainly by third-graders who historically rank highest on tests.
• The formula gives schools one extra point if the school met a federal standard
called Adequate Yearly Progress. If schools failed to test 95 percent of
students in certain categories, as required by federal standards, the feds
allowed them to average the last two years to reach the 95 percent mark. The
state also allowed high schools to set a make-up date for students who missed
the first round of AIMS testing. Accountability Director Regalado said that one
extra point "absolutely made a difference" for many schools. This year, 307
schools failed to make Adequate Yearly Progress, down from 444 in 2003. That
means 137 more schools this year got to add a point, a point that helped many
schools that were sitting on the bubble.
Just as educators begin to understand the state formula, they should expect big
changes next year, forced by new tests.
The national Stanford 9 test will be replaced by a test called Terra Nova. The
state's elementary AIMS test will be combined with Terra Nova and become one new
test, now called the Dual Purpose AIMS. It will be given to students in third
grade through eighth grade. The high school AIMS test will remain basically the
same, but with new questions created by Arizona teachers and, along with
proposed new passing scores, it could be easier to pass. Second-graders and
ninth-graders will take only the Terra Nova.