THE TESTING CHARADE
Pretending to Make Schools Better
By Daniel Koretz
288 pp. University of Chicago Press. $25.
Available at Brookline Booksmith
by Mike Offner
January 2018
Test scores are a primary focus of the administration of the Public Schools of Brookline, and the costs are substantial: teachers feel pressured to “teach to the test” rather than to teach in the manner they feel is best for our children; Brookline students feel considerable stress and anxiety about the tests; the testing itself uses large amounts of time that could be otherwise focused on teaching and learning; and children who are part of groups labeled as “underperforming” can experience feelings of helplessness, marginalization, and low self-esteem.
Given these costs, what are the benefits? Does all of this testing accomplish something worthwhile? Does it help Brookline students learn, achieve, or grow?
Harvard professor Daniel Koretz’s latest work, The Testing Charade: Pretending to Make Schools Better makes a compelling case that the culture and policy of testing not only offers few if any benefits, but in fact does considerable damage to public school students and the greater public school communities. Koretz has studied testing for more than 30 years, has published five books and dozens of articles, and has a CV that is 25 pages long. He writes:
Pressure to raise scores on achievement tests dominates American education today. It shapes what is taught and how it is taught. It influences the problems students are given in math class (often questions from earlier tests), the materials they are given to read, the essays and other work they are required to produce, and often the manner in which educators are rewarded, punished, and even fired. In many cases it determines which students are promoted or graduate. This is the result of decades of “education reforms” that progressively expand the amount of externally imposed testing and ratcheted up the pressure to raise test scores.
In Brookline, our children take the Massachusetts Comprehensive Assessment System (MCAS) tests in Grades Three through Eight, and then again in Grade Ten. Third Grade teachers, knowing that their students will take the MCAS for the first time, know that they will inevitably be judged, at least in part, by their students’ MCAS scores.
And although students do not take the MCAS in Second Grade, the Second Grade teachers feel pressure to have their students well trained in test-oriented questions so that these teachers do not “let down” their colleagues who teach Third Grade. And so it goes, all the way down to kindergarten, where teachers cannot escape the pressure to send their students along to each successive grade well prepared for the MCAS, so as not to create undue stress or negati ve attention f or any of their colleagues.
Brookline is hardly unique in suffering from the many negative aspects of the test-driven culture of public education. According to Professor Koretz:
Test-based accountability has led teachers to waste time on all manner of undesirable test preparation -- for example, teaching children tricks to answer multiple-choice questions or ways to game the rules used to score tests. Testing and test preparation have displaced a sizable share of actual instruction, in a school year that is already short by international standards.
One might wonder, “Does what Professor Koretz writes about really apply to Brookline? Aren’t we progressive and sophisticated enough to not fall victim to the test-prep pressure that Professor Koretz describes?”
A starting point to answer those questions is the memo that is sent to Brookline principals and teachers at the start of each year, telling them how classroom time should be allocated. This annual memo from the Brookline Office of the Superintendent starts as follows:
Each year we send out the time allocations document centrally to all teachers. Each year, the time allocation document induces anxiety in many teachers as they find that the time allocations are not realistic and cannot be met within our school day.
Given that this statement is directly from our superintendent’s office, Brookline parents might ask, “If the time allocations memo causes so much anxiety and is not realistic, why hasn’t the superintendent’s office changed it?”
The evidence of the emphasis that Brookline puts on MCAS preparation is apparent in the time allocations memo itself. The MCAS for grades Three through Eight tests two areas, which it defines as “English Language Arts” and “Mathematics.” And sure enough, in the Brookline time allocations memo, teachers in Grades K through Five are told to spend a total of about 60% of instructional time on areas defined, literally, as “English Language Arts” and “Mathematics.” The remainder of time is divided among: Social Studies, Science, Art, Music, Physical Education, and World Language. In other words, our classrooms’ time allocations are approximately as follows:
But what’s wrong with tests?
“What’s wrong with tests?” a Brookline parent might ask. “Don’t we want to know how well our children are learning?”
Professor Koretz offers three categories of pernicious corruption of teaching that test-focused education culture creates: (1) reallocation between subjects; (2) reallocation within a subject; and (3) coaching.
The reallocation between subjects is illustrated above in the pie chart showing Brookline’s Time Allocation Guidelines. The case of Brookline is consistent with what Koretz has observed generally: “To start, what would you expect to happen if you put great pressure on teachers to raise their scores on tests of a few subjects but ignored everything else? This is not rocket science: you would expect them to cut back on things that don’t count and shift resources to the tested subjects.... We’ve known for decades that they often cut back on subjects like social studies, art, and music.”
And many Brookline parents might share Professor Koretz’s concern that “[s]tudents who don’t learn social studies and science, for example, are poorly equipped to be informed citizens and will be less competent in many lines of work.”
Professor Koretz also writes, “Some educators have also curtailed nonacademic but important activities, such as recess. (Anyone who thinks that recess is unimportant hasn’t taught in an elementary school.)”
Indeed, it’s as though Professor Koretz were thinking specifically of Brookline, as at the start of the 2017-2018 academic year, the Brookline administration reduced Second Grade recess at Runkle by 50%, despite the extensive evidence that recess is critical to students’ academic achievement and other outcomes, as explained by Dr. Rebecca Breslow , who is a Brookline parent as well as a Sports Medicine physician at Brigham & Women’s Hospital and Harvard Medical School.
Koretz observes further that even within a given subject, teachers feel pressure to teach only the precise content that tests measure. “Why would I teach irregular polygons?” he quotes a Boston teacher as asking, and then explains, “She didn’t mean that irregular polygons are unimportant; she meant that to the best of her recollection, irregular polygons didn’t appear on the state test for the grades she taught.”
And finally, there is the problem of “coaching.” For example, teachers can feel pressure to use the “process of elimination” (POE) approach to selecting correct answers on standardized tests. “The problem with POE,” Koretz writes, “is that some of the students who find the correct answer by eliminating incorrect ones would be unable to generate the correct answer if they weren’t given alternatives from which to select...”
Then there is the strategy of “plugging in” numbers to an equation in a test to find something that works. For example, teachers have learned to teach students that the dimensions of triangles are usually 3, 4, and, 5, or, alternatively, 5, 12, and 13, because these are the lowest sets of integers that work for the Pythagorean Theorem , a favorite topic of test-makers. So these dimensions are known as “Pythagorean Triples.”
Of course, in the real world, triangles don’t necessarily have these convenient dimensions. But the message, Koretz writes, is “Don’t bother memorizing the complicated theorem; just memorize the triples, which is easier and faster. Most of the students will get the item right, and everyone can go home happy. Well, almost everyone. Just don’t hire one of them to build your roof.”
Equity
One of the claims about education reform and its central use of testing is that it can increase “equity,” or, broadly speaking, the score gaps between disadvantaged groups and advantaged groups. Unfortunately it’s the other way around. Koretz writes that testing actually hurts disadvantaged children:
As if all of this were not depressing enough, there is yet another disturbing part of the story. Inappropriate test preparation, like score inflation, is more severe in some places than in others. Teachers of high-achieving students have less reason to indulge in bad preparation for high-stakes tests because the majority of their students will score adequately without it -- in particular, above the “proficient” cut score that counts for accountability purposes. So one would expect that test preparation would be a more severe problem in schools serving high concentrations of disadvantaged students, and it is. Once again, disadvantaged kids are getting the short end of the stick. Ironically, some aspects of the reforms that were intended to help disadvantaged students appear to have contributed to this demoralizing result.
And Professor Koretz has good company in his view that focusing on test scores is actually in equitable. For example, American University Professor Ibram Kendi, in his article “Why the Academic Achievement Gap is a Racist Idea,” writes:
These days, many people are criticizing the testing movement. Colleges are slowly diminishing the importance of standardized testing in admissions decisions. We are seeing unprecedented numbers of wealthy white parents opting their school children out of these tests.
But few testing critics are bursting its biggest bubble: the existence of the achievement gap itself. To believe in the existence of any sort of racial hierarchy is actually to believe in a racist idea. The achievement gap between the races–with Whites and Asians at the top and Blacks and Latinos at the bottom–is a racial hierarchy. And this popular racial hierarchy has been constructed by our religious faith in standardized testing ....
What if we measured literacy by how knowledgeable individuals are about their own environment: how much individuals knew all those complex equations and verbal and nonverbal vocabularies of their everyday life?
What if we measured intellect by an individual’s desire to know? What if we measured intellect by how open an individual’s mind is to self-critique and new ideas?
What if our educational system focused on opening minds instead of filling minds and testing how full they are? What if we realized the best way to standardize a highly effective educational system is not by standardizing our tests but by standardizing our schools to encourage intellectual openness and difference?
But intellectual difference , and multiple literacies, languages, and vocabularies, are only valued in a multi-cultural society that truly values diversity and difference. The testing movement does not value multiculturalism. The testing movement does not value the antiracist equality of difference. The testing movement values the racist hierarchy of difference, and its bastard 100-year-old child: the academic achievement gap.
And UMass Professor Ricardo Rosa has written in Voices of Urban Education that “[p]erformance assessments must be culturally responsive in order to truly serve the needs of students from all backgrounds.” Professor Rosa explains further:
If we begin, as I do, from the perspective that institutions, including schools, are designed in the image and interests of those who rule, we must be very cautious about re-creating an educational reform environment where people of color and the poor will continue to be marginalized. If performance-based assessment is considered in the same frame as current testing regimes, which is entirely possible, it becomes just another reform fad (I don’t mean to suggest that performance-based assessments is a new one) that re-inscribes the power of systems of categorization and the conferring of rewards to those who are already materially, racially, and culturally privileged. From this perspective, performance-based assessments become another repressive surveillance technique in the lives of children and adolescents.
What now?
Given the failings of standardized testing, Professor Koretz recommends that schools focus on more holistic and fundamental evaluations of classroom practices and student outcomes, and acknowledge explicitly that many critical factors in students’ achievement are completely outside of schools’ control. He recommends, for example, that communities try to evaluate how well they help their children learn how to persist at challenging tasks and how to work well in groups.
Further, he stresses the important of human judgement as opposed to computer-generated data: “...we need to accept the need for human judgment in evaluating schools. This is in some ways unfortunate, given the difficult problems inherent in relying on judgement, but there really is no practical alternative. We will need to find better ways to use judgement, not avoid it.”
Professor Koretz also recommends creating counterbalances to administrators’ incentive to focus on test scores: “If principals have an incentive to eliminate recess to squeeze out a little more time for test prep,” he writes, “someone should have an incentive to put a stop to this.”
In addition, Professor Koretz offers some compelling and fascinating descriptions of the public education systems in Finland, Holland, and Singapore. While it’s not clear how practical these examples are for the United States’ own systems, the discussion of these other nations’ approaches is thought-provoking and enlightening.
___
Anyone who wonders how much the issues in The Testing Charade apply to Brookline need only have open and candid conversations with our teachers and students. The right answer for our children seems to be that beyond meeting state requirements for offering the MCAS for Grades 3-8 and for Grade 10, the Public Schools of Brookline should not make standardized testing or test scores a focal point of policies or priorities in any manner.
In Holliston, MA, Superintendent Brad Jackson, who is also the vice president of the Massachusetts Association of School Superintendents, told teachers that he is “taking their handcuffs off” and encouraging them to teach in the manner they believe is best for their students without regard to the MCAS. It’s a remarkable display of courage, leadership, and commitment to the well being of children. If you believe The Testing Charade, Brookline should follow.
