The 21st Century Principal

"Metric fixation is the seemingly irresistible pressure to measure performance, to publicize it, and to reward it, often in the face of evidence that this just doesn't work very well." Jerry Muller, The Tyranny of Metrics

There is absolutely no doubt that many 21st century education leaders (and politicians) are infected with the metric-fixation disease. The mad illness persists in spite of the fact that no achievement gaps are closing, and no miraculous gains (in their own standardized tests) has occurred.  Perhaps its time find a cure for this persistent disease that is distorting education. The only vaccination against this malady is a sudden jolt of common sense and the realization that not everything worthwhile in this world is measurable. 

Muller, J. (2018). The Tyranny of Metrics, Princeton University Press: Princeton, NJ.

"The effects of teaching may not show up until long after students leave school and in ways the teacher never dreamed of." Elliot Eisner, The Arts and the Creation of Mind

How do you measure the true educational impact of a teacher? If you consult psychometricians, they say it is simple. You pre-test, deliver instruction, and you post-test. On this our "educational sciences" are based. But do these tests really measure the teacher's most important impact on students' lives? Is the most important task of a teacher to demonstrate that they can "improve a student's test scores?" And, equally important, no matter what our state and federal education bureaucracy tells us, "Are these state standardized test results really capturing learning that will be meaningful to those students' future lives, or are these results simply better predictors on how students will score on other standardized tests?"

Elliot Eisner's book, The Arts and the Creation of Mind depicts an image of a teacher that is much more complex and complete than that currently promoted by the "education sciences" to which modern education finds itself enslaved. The teacher is much more farsighted than the teacher who can't wait for the latest standardized test scores at the end of the year. Eisner's teacher is an "environmental designer" who "creates" situations and places where students gain "an appetite to learn."

Eisner's teacher is not a technician who uses "test data" to choose "canned scripts" and the latest adopted "scientifically validated methods and curriculum" whose purpose, is not to inspire wonder and imagination, but whose purpose is to make some education administrator or politician feel like they are effectively improving education. The teacher should not be teaching by following recipes; they should be engaging students in a "mind-altering curriculum" that forever changes them into forever learners.

What's wrong with the current grip that so-called "education sciences" have on schools is that they have created an impoverished, assembly-line form of education that students don't have to participate in; they only need to be subjected to it. Our education system still strives to run "smoothly," in a standardized manner and as efficiently as possible, and to get as many students through the credentialing process. It is short-sighted and its vision can't see beyond the "testing extravaganza at the end of the year.

But as Eisner makes clear in his book, if you really want "educational gold" in the classroom, then a "high-degree teaching artistry is needed. You need classrooms of "improvisation and unpredictability," not classrooms constructed according to rigid scientific principles. The teacher, in this innovative and creative classroom, is not a scientist who constantly studies the latest test data and looks at his repertoire of "research-based, scientifically-validated" classroom scripts for the one to apply because the data indicates it is called for. The teacher is what Eisner calls "a midwife to the child's creative nature."

As I look back at my years in elementary school, I see one teacher who I would really say was the midwife to my own creative nature. She didn't make noise about my performances on tests. She genuinely questioned and encouraged me when I showed curiosity in the solar system, astronomy, biology, tadpoles, frogs, and trees. She listened attentively when I read stories I had written aloud in class and encouraged me to write more. She encouraged me to read anything and everything I could get my hands on in our school library, even helping me get permission from the librarian to wander into and check out books from the "junior high section" instead of the elementary section where all six-graders were constrained. I read more books that year than perhaps in any other time of my life because of her. In a word, she designed an environment that helped me grow my curiosity and a massive appetite to learn that is still alive today.

My greatest concern with the Standards-Standardized-Testing-Research-Based-Accountability educational milieu we've created in our schools is the damage it is doing to students far into their futures. Does all this focus and obsession with test scores really matter in the lives of our students? The true impact we have on student lives is an impact that hasn't happened yet, and its an impact that can't be measured by standardized tests. My sixth grade teacher had no idea that the classroom environment she created would mean that I would become a teacher myself. She had no idea that I would become a principal. She also had no idea that my passion for reading, writing, and appetite to learn would stay with me the rest of my life.

Perhaps if we really want to focus on "student outcomes" we need to set our sights beyond test data and create places of imagination, creativity, and innovation where curiosity is treasured, learning is not just measured, but valued. Not everything worthwhile can be reduced to pre-tests and post-tests, and the real impact of our work with students will be measured by the lives they live far beyond the classroom.

It seems new Secretary of Education, John King seeks to continue the same reliance on test scores and superficial ratings systems to determine how effective schools are doing that his predecessor Arne Duncan pushed. From his recent remarks, King wants to force states into using a "A-F Rating system" (or something similar) to rate the effectiveness of schools, which happens to be the same nonsensical idea that the North Carolina Legislature and North Carolina Governor Pat McCrory has imposed on public schools in North Carolina. (See "Education Secretary Takes Heat for Pushing Single Rating of Schools"). He wants states to come up with a "single summative rating for schools or districts" that captures the success or failure of schools in the most simplest way possible.

In 2011, I made clear my concerns with "Test Pep Rallies" with my blog post "Test Pep Rallies: Good Practice or Waste of Time?" I still feel that such practices are more harmful than useful. As in my earlier post, I still believe that:

• Test Pep Rallies potentially harm students and learning. It's one thing to encourage a student to do his or her best; it's another to place emphasis on performance levels, where self-worth might be wrongly tied to test results. In my thinking, Test Pep Rallies have too much potential for making state test results too important, especially if held for the purpose of promoting test performance. Encourage students to always do their best, not just when testing season comes along.
• Test Pep Rallies reinforce the "Culture of Test Prep" in schools rather than worthwhile learning. Very little worthwhile learning takes place in schools where test prep is the goal for everything the school does.Its one thing to use data in decision-making; its quite another to use test results to determine everything that happens. Test Pep Rallies are about Test Prep, not about celebrating accomplishment. They're shortsighted practices for the short term that has not lasting impact on anything.
• Test Pep Rallies are a waste of time. Why do we even want to elevate a standardized test to such a high level? Students could be celebrating real learning and accomplishments instead of focusing on a test no one will pay attention to five or ten years in the future.

Of course we all acknowledge state standardized tests can't possibly measure everything that matters. Our state even forbids the use of state test scores as the sole basis of making high-stakes decisions. Yet, as the season of testing falls upon us, the obsession with bubble sheets and number two pencils begins anew. But we need to be reminded that tests are only a small piece of information that matters about our students and our teachers. Before we start scheduling the "Test Prep Pep Rallies" and giving our students those motivational speeches that elevate these tests higher than they should be, we need to be truthful with our students: in the grand scheme of public education, these test scores do not measure what's most important, and the results certainly do not come close to capturing what it means to be an excellent teacher, especially when excellent teachers impact lives more than test scores.

Recently, a Florida teacher wrote this letter to her students just before the onslaught of state tests. I think perhaps it reminds us that testing should always be put in proper perspective, not elevated to some major life goal or achievement. No one is going brag 10 years down the road that they made a "Level 5" on their English End of Course Test or their Biology End of Course Test. As this teacher points out, there are many more things worthwhile to brag about.


"My Dearest Students,

This week you will take your Florida State Assessments (FSA) for reading and math. I know how hard you worked, but there is something important that you must know. The FSA does not assess all of what makes each of you special and unique. The people who create these tests and score them do not know each of you the way I do, and certainly not the way your families do.

They do not know that some of you speak two languages, or that you love to sing or paint a picture. They don't know that your friends can count on you to be there for them, that your laughter can brighten the dreariest day, or that your face turns red when you feel shy. They have not heard you tell differences between a King Cobra and a rattler. They do not know that you participate in sports, wonder about the future, or that sometimes you take care of your little brother or sister after school. They do not know that despite dealing with bad circumstances, you still come to school with a smile. They do not know that you can tell a great story or that really love spending time (baking, hunting, mudding, fishing, shopping...) with special family members and friends. They do not know that you can be very trustworthy, kind or thoughtful, and that you try every day to be your very best.

The scores you will get from this test will tell you something, but they will not tell you everything. There are many ways of being smart. You are smart! You are enough! You are the light that brightens my day! So while you are preparing for this test and while you are in the midst of it all, remember that there is no way to "test" all the amazing and awesome things that make you, YOU!

Thank you for taking the time to read this. Please keep all kids in the state of Florida in your thoughts tomorrow. Thank you."

Here's my own three reminders to educators as we find ourselves on the eve of another "Season of Standardized Testing."

1. As we move into the "Season of Testing" let's remember that we don't teach test-takers; we teach real human beings with interests, hopes, dreams and passions that can't be reduced to multiple-choice questions.

2. As long as students and teachers give us their best, we acknowledge and celebrate that. Celebrate accomplishments such as poems written and published; hours of world-changing community service served; and songs written and sung. Celebrate what the bubble sheets ignore.

3. Keep testing in its place as one piece of data. Don't elevate it needlessly. Don't hold school-wide pep rallies that elevate these things superficially. Leave in their place.

Perhaps there's a truth here about accountability and testing that we have ignored: the use of standardized testing, value-added models, and growth models in teacher evaluations is all about subjecting the classroom to techniques of surveillance. To put it bluntly, they are spy tactics. Their purpose is to peer into the classroom to see if teachers are teaching the "prescribed" curriculum, and to see if teachers are adhering to the "rules of best practice" as they teach. These surveillance techniques are based on a fundamental mistrust of teachers' professional judgment regarding how they should be teaching and how their students perform.

The current efforts to perfect the evaluation of teachers aren't really just about improving teacher effectiveness: they are about sharpening the gaze into the classroom. They are about making the classroom more visible to those higher up the administrative chain. They are about finding the means to "objectively" determine whether teachers are teaching in the manner prescribed by "best practices" and whether they are teaching only the content that can be subjected to testing.

What testing and accountability experts have discovered though, is that tests are imperfect. The image they project of the teachers' performance is at best blurred and opaque. Even with new-fangled "value-added models," seeing teachers' effectiveness is foggy and unreliable. Now, they are seeking other ways to increase classroom surveillance. They are seeking other "spies" which might provide them with a clearer gaze of what's happening in the classroom. What are those new techniques for gazing into classrooms? They are called "student surveys."

Interestingly, student surveys turn students into 25 or 30 pairs of eyes that can report back to administration regarding whether classrooms are conducted in the manner dictated by the laws of "best practices." The student survey becomes another instrument with which the administration and government can sharpen its gaze into the classroom to make sure teachers' conduct adheres to "best practices." Underlying the use of student surveys is the assumption as well that if teachers know that 25 or 30 pairs of eyes are watching that might potentially report deviance back to the administration, those teachers will engage in the expected teaching behaviors. Thus, the control of the classroom becomes more complete.

We need to perhaps realize that the push to ever better classroom data is maybe more about control and transformation of the teaching and the teaching profession into a non-profession where teachers are simply "technicians of learning" whose professional judgment means nothing. The next generation of teachers will not need to exercise professional judgment: they will only need to conduct their classes in the prescribed manner by the "sciences of teaching." The question then becomes, "Are we really producing the kinds of students that our world needs?" Students newly manufactured and stamped with the US Department of Education's approval as being "Globally competent."




- Posted using BlogPress from my iPad

Today, the North Carolina Department of Public Instruction released the school report cards mandated by North Carolina’s legislature. (See those here.) These report cards graded each school with a letter grade A-F. Once again, our state has taken a step backward into absurdity with this action. Grading all the things our schools do with a single letter grade reduces, once again, what matters in North Carolina schools the most, to test scores. Once again, our state has elevated state testing to even higher stakes. Schools will now work in earnest prepping students for tests and getting those numbers up.

But elevating test scores is not only what this exercise in madness does; it also clearly demonstrates what’s wrong with education, and society, in North Carolina. In its article entitled “NC Public School Letter Grades Released, Reflecting Student Family Incomes,” The News and Observer sums up the real truth we learn from these report cards.

We don’t really learn which schools are failing and which are succeeding because the data used for this is narrowly focused on test score data and a few other indicators. What we learn of real importance is stated so aptly in this article:

“Among the schools where 80 percent or more of the students qualify for free and reduced lunch, 81 percent received a D or F. Only one of those schools got an A. At the other end of the spectrum, more than 90 percent of schools where fewer than 20 percent of the students qualify for free and reduced lunch received an A or B. Only one of those schools received an F."

In other words, our legislature and North Carolina Department of Public Instruction hasn’t come up with a test at all on how our schools or do; they’ve developed the perfect test for poverty. In fact, it really shows that North Carolina’s barrage of tests are great for indentifying students who live in poverty! Probably much better than actually measuring student achievement.

Sadly though, I suspect the motivation behind this A-F grading system isn’t really about improving public education at all. After all, our North Carolina State Legislature proved during its last session it is no friend to public education, why would we expect different.? No, this grading system is simply another attempt by our political leaders to drum up or even manufacture false charges of failure so that they can continue to push their pet project of school vouchers and their blind obedience to free markets.

Public schools have suffered under Secretary Arne Duncan's Race to the Top and No Child Left Behind law waivers. Testing, not learning has become the focus. Schools have cut arts programs and non-tested subjects. Enormous amounts of time are spent during the school year getting students ready for the tests. And, since the Obama administration took office, there are many states like North Carolina that administer a record number of state tests, and the use those results as a part of teacher evaluations. It has been this President's education policy that has done more to elevate test scores to even higher levels than under No Child Left Behind. 

Now, Arne Duncan is once again trying to elevate test scores even higher: he wants to use test scores to evaluate the effectiveness of teacher programs too.

Under Arne Duncan's latest effort to hold somebody else accountable for education except himself and politicians, Duncan now wants to create a new, massive bureaucratic procedure to judge the "effectiveness" of teacher preparations programs around the country. This behemoth proposal would bizarrely twist test scores once more in the name of accountability. As I read through this proposed procedure, I simply grow more and more angry at a President and Secretary of Education who simply have no clue as to what their "test-them-if-they-breathe" education agenda has done to schools, students, teachers, classrooms, and the future of the education profession. If you read the fine print of this massive document, you can quickly read between the lines regarding what Arne Duncan is actually proposing.

• Using test scores, most likely value-added measures, to determine the effectiveness of teacher preparation programs that receive federal funding.
• The development of a massive pile of red tape and bureaucratic procedures to make sure teacher preparation programs comply to the dictates of the US Department of Education.
• An enormous overreach of federal power and powergrab by the US Department of Education.

What happens when all that matters are test scores? Just ask former El Paso Superintendent Lorenzo Garcia who was sentenced by a federal judge to three and half years for a test-cheating scandal(See “Former EPISD Superintendent Lorenzo Garcia Gets 42 Months, Offers No Apologies for Scandal.”) Don’t get me wrong, I don’t shift the blame for one minute away from from Garcia. As a leader, compromising your morals to cook the books is never excusable, even if there’s pressure from elsewhere to do so. Yet, Garcia’s actions are understandable in a corporate American culture where “cooking the books” in order to deceive investors is acceptable. After all, is that not what the financial meltdown was about? Was it not about Wall Streeters who hid toxic loans from investors while making exorbitant salaries? “Cooking the books” has become an American management strategy, so should anyone be really surprised when school leaders like Garcia, or Atlanta Public Schools Superintendent Beverly Hall does just that to make their school systems appear to perform better than they really are?

In his book, Who’s Araid of the Big Bad Dragon: Why China Has the Best (and Worst)Education System in the World, Yong Zhao blames the accountability system based on high stakes testing. True. This obession with test scores in the United States is unhealthy. Our politicians and state education leaders have convinced many school leaders that obtaining higher test scores is the ultimate goal and product of school systems. No wonder there are cheaters who cook the books of school accountability to make it look like their districts are performing better than they really are. Accountability in the United States is evolving into the same game that Wall Street bankers play; cook the books to make it look like things are better than they are.

But at the end of the day who are these who cook the books really fooling? Are our students really learning more than they ever had? Are we actually producing the best graduates we have ever produced? Is our graduation rate really any higher than it has ever been? In some ways, I am afraid accountability in education has become a game that educators play. In a culture where numbers ultimately matter more than kids, education has adopted the exact same thinking that Wall Street adopted; whatever you can do to make your bottom line appear better than actually is becomes acceptable. Instead of focusing genuinely on the kids, accountability is game of data manipulation.

Garcia and Hall did a great disservice to education by cheating. Their actions are inexcusable. Yet, there are other educators still playing the game of accountability, shifting data points around, in order to make it “appear that their school or district” is on top.

Wall Street's “book-cooking” tactics have no place in education. What we do as educators determines the course of young lives, and when our focus shifts from that to numbers alone, we too are as guilty as those who wrecked the economy, and who “cook the books of accountability” for appearances sake.

“A VAM (Value-Added Model) score may provide teachers and administrators with information on their students’ performance and identify areas where improvement is needed, but it does not provide information on how to improve the teaching.” American Statistical Association

Today, I spent a little time looking over the American Statistical Association’s "ASA Statement on Using Value-Added Models for Educational Assessment.” That statement serves as a reminder to school leaders regarding what these models can and cannot do. Here, in North Carolina and in other states, as school leaders begin looking at  No Child Left Behind Waiver-imposed value added rankings on teachers, they would do well to remind themselves of the cautions describe by ASA last April. Here’s some really poignant reminders from that statement:

• “Estimates from VAMs should always be accompanied by measures of precision and a discussion of the assumptions and possible limitations of the model. These limitations are particularly relevant if VAMs are used for high-stakes purposes.”
• “VAMs are generally based on standardized test scores, and do not directly measure potential teacher contributions toward other student outcomes.”
• “VAMs typically measure correlation, not causation: Effects—positive or negative—attributed to a teacher may actually be caused by other factors that are not captured in the model.”
• “Under some conditions, VAM scores and rankings can change substantially when a different model or test is used, and a thorough analysis should be undertaken to evaluate the sensitivity of estimates to different models.”
• “Most VAM studies find that teachers account for about 1% to 14% of the variability in test scores, and that the majority of opportunities for quality improvement are found in the system-level conditions.
• “Ranking teachers by their VAM scores can have unintended consequences that reduce quality.”
• “The measure of student achievement is typically a score on a standardized test, and VAMs are only as good as the data fed into them.”
• “Most VAMs predict only performance on the test and not necessarily long-range learning outcomes.”
• “The VAM scores themselves have large standard errors, even when calculated using several years of data.”

In this season of VAM-viewing, it is vital that informed school leaders remind themselves of the limitations of this data. You can’t take the word of companies promoting these models as “objective” and “fool-proof” measures of teacher quality. After all, they have those multimillion dollar contracts or will lose them if one casts doubt about VAM use. Still, a 21st century school leader needs to have a more balanced view of VAM and its limitations.

Value-added ratings should never be used to inform school leaders about teacher quality. There are just too many problems. In the spirit of reviewing VAM data with teachers, here’s my top ten reminders or cautions about using value-added data in judging teacher quality:

1.  Remember the limitations of the data. Though many states and companies providing VAM data fail to provide extensive explanations and discussion about the limitations of their particular value-added model, be sure those limitations are there. It is common to hide these limitations in statistical lingo and jargon, but as a school leader, you would do well to read the fine print, research for yourself, and understand value-added modeling for yourself. Once you understand the limitations of VAMs you will reluctantly make high stakes decisions based on such data.

2. Remember that VAMs are based on imperfect standardized test scores. No tests directly measure teacher  contributions to student learning. In fact, in many states, tests used in VAMS were never intended to be used in a manner to judge teacher quality. For example, the ACT is commonly used in VAMS to determine teacher quality, but it was not designed for that purpose. As you review your VAM data, keep in mind the imperfect testing system your state has. That should give you pause in thinking that the VAM data really tells you flawlessly anything about a teacher’s quality.

3. Because VAMs measure correlation not causation, remind yourself as you look at a teacher’s VAM data that he or she alone did not cause those scores or that data. There are many, many other things that could have had a hand in those scores. No matter what promises statistics companies or policymakers make, remember that VAMs are as imperfect as the tests, the teacher, the students, and the system. VAM data should not be used to make causal inferences about the quality of teaching.

4. Remember that different VAM models produce different rankings. Even choosing one model over another reflects subjective judgment. For example, some state’s choose VAMs that do not control for other variables such as student demographical background because they feel to do so makes an excuse for lower performance for low-socioeconomic students. That is a subjective value judgment on which VAM to use. Because of this subjective judgment, they aren’t perfectly objective. All VAM models aren't equal.

5. Remind yourself that most VAM studies find that teachers account for about 1 to 14 % of variability in test scores. This means that teachers may not have as much control over test scores as many of those using VAMs to determine teacher quality assume. In a perfect manufacturing system where teachers are responsible for churning out test scores, VAMs make sense. Our schools are far from perfect, and there are many, many things out there impacting scores. Teaching is not a manufacturing process nor will it ever be.

6. Remind yourself that should you use VAMs in a high stakes manner, you may actually decrease the quality of student learning and harm the climate of your school. Turning your school into a place where only test scores matter, where teaching to the test is everybody’s business is a real possibility should you place too much emphasis on VAM data. Schools who obsess about test scores aren't fun places for anybody, teachers or students. Balance views of VAM data as well as test data is important.

7. Remember that all VAM models are only as good as the data fed into them. In practical terms, remember the imperfect nature of all standardized tests as you discuss VAM data. Even though states don’t always acknowledge the limitations of their tests, that doesn’t mean you can’t. Keep the imperfect nature of tests and VAMs in mind always. Perhaps then, you want use data unfairly.

8. Remember that VAMs only predict performance on a single test. They do not tell you thing about the long-range impact of that teacher on student performance.

9. Finally, VAMs can have large standard errors. Without getting entangled in statistical lingo, just let it suffice to say that VAMs themselves are imperfect. Keep that in mind when reviewing the data with teachers.

The improper use of VAM data by school leaders can downright harm education. It can turn schools into places where in-depth learning matters less than test content. It can turn teaching into a scripted process of just covering the content. It can turn schools from places of high engagement, to places where no one really wants to be. School leaders can prevent that by keeping VAM data in proper perspective, as the "ASA Statement on Using Value-Added Models for Educational Assessment" does.

"You can prep kids for a standardized test, get a bump in scores, yet not be providing a very good education." Mike Rose, "The Mismeasure of Teaching and Learning: How Contemporary School Reform Fails the Test"

It should not be a surprise at all to politicians, policymakers, and educators that the cry and backlash against testing and accountability is growing. During my 25 year career, I've seen the number of state tests administered in public high schools in North Carolina grow from 1 to well-over 2 dozen. Testing and test scores are the talk, and the focus is almost always on "how can we get those test scores up?" About the only ones, with the exception of a few teachers, who are enthusiastic about all this testing are school administrators, who for the first time have a "cattle prod" as Taubman calls it in his book Teaching by Numbers, to shock those teachers who get out of line and who aren't "producing." Blind acceptance of test scores as "the only data of importance" is common, because such data is seen as an "objective" measure, another myth perpetuated by testing and accountability supporters. But is that true?

Peter Taubman's book Teaching by Numbers: Deconstructing the Discourse of Standards and Accountability in Education points out some flaws with this blind acceptance of testing as "the data" on which to base all educational decisions. He writes,

"Fundamentally, tests provide little more than data, but just as one must question the confessions extracted under torture, one has to wonder just how reliable that data is, when it is wrung out of students shocked by the constant administration of tests." 

In other words, no one questions, least of all school administrators, this "data" we're looking at as measures of teacher, school effectiveness and student learning. Tests are data, but how good is that data when students have been subjected to test-after-test-after-test as we do in high schools in North Carolina? North Carolina education leaders truly believe in the maxim, "If it breathes; test it." Data collected under the oppressive, tortuous testing system in our state isn't foolproof, and our jobs as administrators, educators and teachers is to remember that when we start looking at numbers.

There's no doubt when our state education leaders, administrators use the phrase "accountability" they mean primarily multiple choice tests designed to keep teachers, administrators and whole schools in line. As Taubman writes again,

"All too evident, accountability translates into teachers' responsibility for their students' learning as measured by performance on tests." 

I would add that when our state leaders and most administrators use the phrase student achievement they are only speaking about test scores. The testing math in North Carolina is captured by this equation:

Student Achievement=Test Scores

Reducing learning to a test score is great if you are accountant, but for those of us who know teaching, we know that genuine learning is rarely, if ever, only contained between the letters of a multiple choice question. Real, worth-while learning is not always subject to being captured on a a standardized test.

Administrators love tests though, and with the same enthusiasm that politicians do. Why is this? I think Taubman once again hits the bullseye. He writes,

"One reason administrators are sympathetic to testing, the data it generates, and various practices connected to testing and data aggregation is that these provide control from a distance, a fundamental component of what is called audit culture." 

Testing allows principals to become accountants, district leaders to become accountant managers, and superintendents and state level leaders become CEOs. Through test scores, all levels of administrators finally have a tool to control what happens in classrooms. They can dictate how teachers can act, and even in some school systems, teachers are given scripts to follow to make sure they cover what is to be tested. Increase the number of tests administered and you control more and more of what happens in schools. All that talk about allowing teacher decision-making, but holding teachers accountable for those decisions is just empty rhetoric. Tests are measures of control and compliance, and they are gradually strangling public education. Testing finally gives administrators what they think is an "objective" tool for getting rid of teachers and for making sure everyone is compliant.

Test data also gives administrators at every level "bragging points." It gives them something to boast about to the public, to business, to industry, and to politicians. Never mind that testing almost always reduces teaching and learning to only what can tested. Taubman gets it right once again when he writes,

"Tests constitute one way the educational reforms show the educational system. Extracting data from students, teachers and schools, they force our noses into the bottom line. Keeping us under constant surveillance, they make us vulnerable to centers of control beyond our reach, and, providing the illusion of objective accountability and meritocracy, they reduce education to right answers and information." 

Testing is about keeping teachers and students noses to the bottom line. It is about using the "illusion of objective accountability" to make sure no one gets out of line.

There is no question that this accountability and testing culture is negatively affecting teaching as a profession as well. According to Taubman,

"High stakes tests erode the autonomy of teachers, for if tests determine the curriculum, and if tests tell us what is important to know as a teacher, and if these tests are fabricated by centers of control beyond the reach of teachers, then the teachers' passions, commitments, and wisdom count less and less." 

As mentioned earlier, accountability and testing is in some case reducing the act of teaching to little more than a "scripted lesson." Instructional delivery is simply following the state or district lesson plan. Teacher autonomy due to the massive testing load is at an all time low. Teaching is no longer a profession; it is a factory job, whose goal is to churn out test scores. If a teacher fails to "make production," they are branded "Ineffective" or "Not Making Expected Growth" as its called in North Carolina.

What is more amazing is that state educational leaders just don't get it. Enrollments in education programs in colleges and universities for training teachers is at an all time low, and it isn't just about salaries. Teaching is just not very attractive when your job is test-score production. Talk to any students about becoming a teacher, and they laugh in your face. Even worse is when you find yourself as an educator no longer encouraging young people to become teachers because being a public educator anywhere, much less North Carolina, has been robbed its ability to be satisfying career because too much emphasis is placed on accountability and testing.

Where does all this end? I wish we knew. North Carolina, as do other states, continues to ramp up its testing by adding new tests, and the state stubbornly hangs on to its massive testing regimen. Will it be when there's no one entering teacher education programs in our state? Will it be when there is no one with more than 10 years experience left teaching in the classroom? Or, will it be when parents, students, and teachers finally push back and say they've had enough? Testing and accountability is more oppressive than ever in North Carolina and elsewhere, and it is sinking public education and the teaching profession along with it.

“How reward power is exercised affects outcome. Compliance is most likely if the reward is something valued by the target person. Thus, it is essential to determine what rewards are valued, and a leader should not assume that it be the same for everyone.” Gary Yukl, Leadership in Organizations

As our political leaders and state level policy makers continue to try to find ways to “improve our K-12” systems of education, one persistent idea that just won’t go away is the idea of merit pay and punishment by accountability. They still remain faithful to the idea that somehow teachers will raise test scores if they are offered a big enough carrot or if their livelihoods are somehow placed in jeopardy enough to bring about a level of fear strong enough to give them the test scores they desire. After over a decade of “test-reward-and-punish” policies under No Child Left Behind and Race to the Top, you would think they would finally give up. Instead, money is still being poured into even more standards development and testing, in the hope that somehow education reform magic will happen. What these educational policymakers and politicians just don’t understand is performance pay and punishments are dead in the water before they are even implemented.

One of the reasons for the uselessness of merit pay is captured succinctly by Gary Yukl in his book, Leadership in Organizations. Rewards will only bring about compliance if those rewards are something valued by the "target person.” Don’t get me wrong, teachers and educators want to be paid fairly and be able to live comfortably, but educators know going into the the job that what they are doing is an endeavor much greater that a paycheck. Most are just not built to pursue the big carrots for their own sake. That is one thing that politicians and policymakers don’t get. Perhaps they are motivated by greed, but many of us are not.

Another problem with the carrot and stick approach to education reform is that many educators just don’t believe that test scores are a worthy goal to pursue. Most teachers who have been in the classroom see the tests for what they really are: a single measure focused on a small portion of learning given at a single point in time. That means the test can give s snapshot of only a sliver of learning, but it can’t be the ultimate goal of learning because so much of learning falls outside testing. Our current public education system is asking educators to believe that test scores are an important goal of learning, and many aren’t buying it, and never will.

As Yulk points out, “Even when the conditions are favorable for using rewards, they are more likely to result in compliance rather than commitment.” Rewards only get people to do what is required; they do not engage people’s hearts and minds totally in the goal of education. Under rewards, people aren’t committed to their jobs, the kids, or to the profession. Our current system of accountability and testing along with its reward and punish for test score performance will never work because at its heart, because teaching requires more than compliance; it requires dedication and commitment and no amount of money can purchase that.

One of the most insidious by-products of the age of testing and accountability is the suggestion that educators should "focus on bubble students" in order to raise a school's test scores. For those who might not know what the term "bubble student" is, in education lingo, the bubble student is the student who has the greatest chance of demonstrating growth or an increase in test scores. Many a scheme has been devised to determine who these students are, and talk to any educational or curricular material salesmen, and you are more than likely going to hear this phrase: "Our materials will help you identify those students who have the greatest chance of demonstrating growth, and we give you the materials to focus on them."

Is there not anyone else who feels a bit of disgust at this sleazy sales pitch and idea? Basically, the suggestion is this: you can identify those kids who have the greatest chance of demonstrating higher test scores and focus on them. This also implies that "less focus" will be on other students for whom gains will be harder and more resource-intensive. Whatever happened to teaching "all students?

We have our testing and accountability culture to thank for this perversion. Because test scores become the ultimate indicator of quality, any strategy is on the table, including ignoring some students in order to help those who show the greatest promise of demonstrating growth. If I were a parent of a lower-ability student or a gifted student, who are usually likely victims of "bubble-student" strategies, I would hire a lawyer immediately. There's a pretty good chance that behind the use of such talk is the idea that the school is going to purposefully focus on "money students", that is, students who have the greatest chance of producing test scores, and neglect those at the very bottom and the very top who aren't going to demonstrate the greatest test score gains.

The practice of focusing on "bubble students" or "money students" as its also called is unethical and perverse. No one would suggest to a physician that he only treat those who have the greatest chance of healing. I certainly don't want a mechanic who only takes the easiest cases of repair, and writes the others off as too resource intensive. Any suggestion of this strategy for raising test scores has zero place in schools.

The practice of focusing on bubble students is a direct consequence of this fetishization and idolization of tests present in education today. Make test scores the ultimate goal, and you get perverse educational practices like focusing on the bubble students and ignoring other students because they are less likely to "bring the gains desired." By the way, any sales person who uses that in pitching his products, has immediately lost a sale.

"We would like to dethrone measurement from its godly position, to reveal the false god it has been. We want instead to offer measurement a new job—that of helpful servant. We want to use measurement to give us the kind and quality of feedback that supports and welcomes people to step forward with their desire to contribute, to learn, and to achieve." Margaret Wheatley, Finding Our Way: Leadership for an Uncertain Time

Want to know what's wrong with testing and accountability today? It's more about a "gotcha game" than really trying to help teachers improve their craft. Over and over ad nauseam, those pushing these tests talk about using test data to improve teaching and thereby student learning, but that's not what is happening at all.

In American education, despite what many testing and accountability advocates say, testing is driving our education system. Decision after decision is based on what will "produce the best test scores." What's wrong with that? Nothing at all, if those tests truly and accurately capture worthwhile learning, but sadly, our quest for the "Holy Grail" of tests has not been productive. All the tests and bubble sheets we subject students to are incapable of capturing real learning. I don't have the same faith in testing that many educators have. There will never be a test, nor a set of standards that saves education.

I suggest that we do as Wheatley suggests in her book Finding Our Way. Let's "dethrone measurement," in this case testing and reveal that it is a "false god." We've had well over 10 years of "test worship" and absolutely nothing to show for it. No Child Left Behind began elevating testing to deity levels, and Race to the Top has only elevated testing even higher, to the point that we're now deciding the fate of teacher assistants, teacher careers, student promotions, even the status of whole schools based on single test scores. We have made "tests" our crystal balls through which we can identify a bad teacher or bad school. We have test scores to tell us how much impact a kindergarten teacher might have on future earnings. Really? Do we really believe in the power of tests and the power of data that much?

We do need to dethrone testing a bit and make it a servant of good education rather than the dictator it has become. I'm afraid that won't happen until this fundamental faith in the infallibility of test scores ends. Let's hope our education system isn't destroyed first.

Veteran Educator and education writer Marion Brady had some thought provoking words in this recent post on Valarie Strauss' Washington Post Answer Sheet blog. (See "What do standardized tests really measure?") In that post, Brady provides some gems that should provoke more discussion on the damage to public education that is occurring by those who insist, as Brady puts it, "Test scores are infallible indicators of quality."

Yesterday, in my post about how the use of value-added measures in teacher evaluations in Tennessee has perverted both education practice and teacher evaluation, I called these individuals who insist on the "infallibility of test scores" as fundamentalists. The dictionary definition of "fundamentalist" is:

"Fundamentalist: strict adherence to any set of basic ideas or principles"

In the case of the testing fundamentalists, there is a strict adherence in the belief that tests scores are infallible indicators of quality of both teaching and learning. Why they might argue that they don't believe test scores are "infallible," they still use them as if they were infallible.  While I called them "fundamentalists" as a jest in part, but there is some truth to that statement. Too easily educators, politicians, businesses, and the general public have come to view "test scores as indicators of quality" and take the attitude "that numbers don't lie" and that they are "objective" and that belief is driving much of the strict adherence to current testing and accountability reform.

Get into an honest discussion with a true believer in test-scores-as-indicators-of-quality, and they sometimes acknowledge the problems with tests, the testing process. But, and this always happen, they resort to the argument, "Well, that's the best we've got." It's easy to see what's wrong with that argument. Basing the future of a child and a teacher on test scores and defining "teaching quality" as only test score results ignores the real complexity of learning.

When test scores are worshipped (or used in a fundamentalist manner) as the "true and infallible" indicator of teaching and learning quality, both are reduced to simplistic, rote activities. As Brady points out, "Teaching---trying to shape minds---is hard complicated work." But herein is the problem. Those who worship at the altar of bubble sheets, Pearson, and College Board, don't see learning as "trying to shape minds." They see learning as a simple imparting of knowledge from teacher to student. Brady points that out when he says that Bill Gates sees "learning as a product of teaching." By reducing teaching to a process of product delivery in the form of test scores, then all this blather about testing, accountability, and value-added measures makes sense. But if anyone argues against these beliefs that are labeled as "status-quo supporters" as if they were some kind of heretic to question this doctrine.

Test scores are only test scores. They might sometimes tell us something about teaching and learning, and sometimes they tell us more about a student's socioeconomic status, or the kinds of support the child is getting at home. Test scores are and always will be subject to error, and they aren't as "objective" as the true believers believe. We can't use test scores "as if they were infallible indicators of learning."

What happens when schools and school districts let test scores rule the day? Check out letter of resignation by a 14 year veteran teacher from Tennessee who was found "ineffective" due to her TVAAS ratings.

"What Defines an 'Ineffective' Teacher?"

What immediately becomes clear to me is the following:

• The problem with the accountability in education movement, like in this case, is that too many educators, politicians and state level testing bureaucrats think "any old test will do" when it comes to obtaining data for teacher ratings. Little time is taken to check to see if the tests really test what is being taught, and whether or not using the tests to evaluate teacher effectiveness is even valid. In this accountability madness, there is always an assumption that test data does not lie and that it's objectivity is a given. Both are wrong. Test data is just numbers, but the inferences, like teacher effectiveness, we make from those numbers can be wrong. Any old test will not do when high stakes of any kind are attached.
• Our education system has become blinded by its own test data. In other words, our education decision-makers have blinders on because it is somehow seen sacred that all decisions should be tied to data, and good data are test scores. You can't rely on teacher judgment because it is tainted with subjectivity, so those making these bizarre decisions about accountability through test scores and value-added measures immediately discard everything else. In this case, and I fear in many others around the country, the teaching profession and our education system is being destroyed by "testing fundamentalists" who have become blind to reason and to the possibility that their teacher evaluation systems are hurting real people and even that they might be wrong.
• The "testing fundamentalists" are beginning to see the fruits of their blind, ideological belief in sacredness of test score data. When test scores matter above all else, educational decision-making gets just "plain stupid." It is just plain bizarre that a 14 year veteran teacher was coached by an instructional coach WHO WAS ONCE HER STUDENT TEACHER. The person she mentored four years earlier is now mentoring her. When educators blindly follow the data trail, they end up in bizarre situations like this one.

"In the workplace, there is no getting around the fact that "the basic purpose of merit pay is manipulative." Alfie Kohn, Punished by Rewards: The Trouble with Gold Stars, Incentive Plans, A's, Praise, and Other Bribes

Merit pay is one of those ideas in education that just won't die. When budgets get tight, policymakers and politicians both look at the money being spent on educator salaries and the idea of paying educators based on performance starts looking attractive. "If only we identify the best teachers and pay them more, all will be well," they think. Then, the task of trying to identify and operationalize what a "best teacher" looks like begins, and it immediately falls apart. There's never been any agreement on what characterizes a "good teacher," and there probably never will. The current reform and accountability craze would have us believe "test scores a good teacher makes," but those of us who've been in classrooms for sometime know that tests don't always tell us what a good student is much less a good teacher. The pursuit of trying to find a specific, clear definition of good teaching and a good teacher is impossible, because teaching and learning, for that matter, are way too complex to reduce to a simple operationalized definition.

As Kohn points out, the problem with merit pay is that it is manipulative. It is simply an attempt to control educators and elicit a behavior, and in most cases, the desired behavior is the production of higher test scores. The problem is, many of us educators know "getting higher test scores" is a superficial goal. Getting a high score on a North Carolina Final Exam or End of Grade Test means very little in the lives of our students. We can't say to our students, for example, "If you get a high grade on this reading End of Grade Test, you'll be successful in life." If we do say that, we're trivializing education. So the idea of manipulating teachers to get them to raise test scores by merit pay is doomed to fail for those of us who see education's purpose as more universal and global. Educating good bubble-sheet bubblers is quite different than educating solid citizens who can take their place in the world and perhaps change things for the better.

In the end, merit pay will always fail in education, because the enterprise in which we engage is much too complex to be subject to its manipulative effects. Merit pay has been tried and it failed every time. Unfortunately, those who still have the faith in manipulation by reward just can't seem to let go of an anachronistic view of human motivation.

The Obama Administration announced its latest plan today that would tie federal funding of teacher preparation programs to test scores. Duncan and the Obama Administration are seeking to place the federal government in the middle of teacher preparation. I suspect this measure will only make it more difficult to convince young people to become teachers and educators, and it will likely have some perverse affects on teacher preparation in this country just as the use of testing for high stakes decisions has had on schools and classrooms.

On closer examination, this proposal contradicts research, logic, and common sense.

• There is no evidence that using high stakes tests as a means to improve teacher preparation programs will work. There is plenty of evidence that using tests (as has been done under NCLB and Race to the Top) to make high stakes decisions has had quite a few unintended consequences in schools and classrooms. These consequences have included things like the over-emphasis on test-prep, teaching to the test, narrowing the curriculum to test content, cutting of arts programs, use of programmed instruction, and cheating, among many others.  One can only imagine what those effects will be on teacher preparation programs when tests are used to determine continued federal funding. For example, teacher candidates will now perhaps be well-versed in things like test-prep and teaching to the test, as opposed to engaging students in authentic instruction. Perhaps teaching candidates can also learn how to narrow the curriculum so that it focuses only on what is tested . The effects of using high stakes testing to make high stakes decisions is well-documented in the research, and adding testing stakes to teacher preparation will most likely affect those programs in adverse ways just as it has in schools and classrooms.
• Surprisingly, the Obama administration's idea of tying test scores to teacher preparation programs comes even after the American Statistical Association recently stated that value-added formulas should be used with caution, because teachers only account for less than 15 percent  (or even less in some studies) of the variability in test scores. In other words, most of what happens in the classroom is beyond a teacher's control. Perhaps the Obama administration is still looking to find "Superteachers" as John Kuhn has called them who are capable of performing miracles. Unfortunately, I am afraid there are so few, if any, especially in an environment, which his policies have created, that is so hostile to teachers and the education profession in general. The important tenet of accountability is that you hold individuals accountable for that which is in their direct control. This policy violates that tenet in many ways.
• Many state tests aren't of high enough quality to be even considered for using in this high stakes manner, and the use of commercial tests like ACT and SAT makes little sense because they haven't been designed for this purpose. Add the fact that, once again, there are entirely too many teachers not subject to test scores. That's the same issue with his Race to the Top and NCLB waiver policy. State tests were mostly designed to tell what students know, not tell how well teachers are teaching. Once again, Arne Duncan and the Obama administration are pushing for using tests in still another way for which they weren't designed.
• This new policy of using test scores to evaluate teacher prep programs should also have some interesting implications in practice. I could easily see this policy exacerbating the problem of getting good teachers in high needs schools. In addition, I could also see this policy affecting which school's graduates choose to teach in as well. It's common sense. Are you going to select teaching in a school where test scores are abysmally low, and your job is to somehow miraculously to raise them? As a teacher prep program, are you going to encourage your graduates to teach in schools with historically low test scores? Perverse education policy often brings about perverse practice.

 As this Politico article points out, Duncan is expecting a great deal of opposition to this latest plan, (See Barack Obama Cracks Down on Poor Teacher Training.) How could he expect things to be different? Educators and parents grow tired of all the testing and emphasis on testing. What will be more interesting to see is whether he and  the rest of the Obama Administration will be willing to listen to educators on this matter. Sadly, the history of this administration and Arne Duncan probably provides that answer even as I ask the question.

In 2011, I explored the topic of "Test Pep Rallies" in a blog post entitled "Test Pep Rallies: Good Practice or Waste of Time". With the advent of Race to the Top and the Obama administration's education policy forcing states to elevate standardized tests to even higher stakes, I wondered if the practice had become even more prevalent. I again did a quick search for research on the topic, and as far as I can tell, no studies of whether the practice even affects student achievement exist. Still, though,  the Test Pep Rallies continue and in some cases have become part of celebrated school culture and tradition.

In New York, a "Rock the Test" rally has gotten bigger over the years and continues strongly. PS 55 recently held its "Rock the Test" rally on March 31. (See "Pep Rally Readies PS 55 Students to 'Rock the Test' as They Begin State Exams.") However, the message seems to be a bit different. The principal of PS 55, Sharon Fishman states "The message to students and parents should be that this is just a test; no matter how they do, it's not the end of the world." Still, the school talks about "psyching students up the test" while at the same time talks about calming students' nerves and helping them deal with stress.

These rallies continue in other states too. In Nebraska, Elkhorn Elementary also had a Test Pep Rally on March 28 to "Fire Up Kids for the Test." This school goes so far as to compare "getting ready for the test" to "getting ready for the big game" and even brought former Nebraska football players to remind students that "the test is their game." A local TV station WOWT joined in as well to get students pumped for the state tests.

Still another school in Indiana tried to get students pumped for the tests on April 10. On that date, Mary Beck Elementary in Indiana held its test pep rally "to get students feeling positive about tests." This rally included a Test-Cheer, games, and performances by teachers and students on how to perform well on tests. This rally was all about "getting students excited about the tests." (See "Students Participate in Pep Rally Before ISTEP Testing Starts.")

All over the country, schools are still holding “Test Pep Rally Events” obviously with the hope that these events will somehow have a positive effect on student achievement scores, even though there is still absolutely no evidence that such practices work. I just can't help but wonder if there are other educators like me who see these practices as harmful and downright unethical. Since my 2011 post about Test Pep Rallies, my big question now is "Why have we allowed our culture of education come to a point where tests even deserve this kind of emphasis?" This practice of using "Test Pep Rallies" has to be one of the most bizarre rituals to come out of the testing and accountability culture yet!

In my original 2011 post I raised a series of questions about Test Pep Rally use that I think is even more pertinent in today's even higher-stakes testing atmosphere.

• How does holding a Pep Rally over-emphasize the test's importance? Does not this practice buy in to the idea that "only the test matters?" It would seem that Test Pep Rallies only reinforce a school cultural idea that one's value is determined by a test. Is that a message we want to send to kids?
• How does holding “Test Pep Rallies” foster a culture where “teaching to test” is expected and the norm? It would be interesting to see if these schools holding these rallies are dominated by school cultures where the teaching that occurs focuses mostly on the test, and that the de-facto curriculum is actually the test content. But even if they aren't, what is the hidden message about tests we are sending kids with these kinds of events?
• Do these “Test Pep Rallies” work as intended? Do they even affect test scores? Even if one buys into the idea of elevating of test scores to this level, does that mean having these rallies raise test scores? There's no evidence of this at all.
• How has the use of Value-added teacher evaluations affected the frequency of these test pep rallies? One would suspect as our education system places even higher-stakes on testing, the occurrence of these test pep rallies will increase, taking even more valuable time away from learning.
• Do these Test Pep Rallies foster a culture that trivializes learning and makes standardized tests the focus of all learning? This question of course depends on what your definition of learning is. If it's a test score, then the answer is no. If the answer is yes, then perhaps you see learning as more than a test score. There is something about this practice that seems demean learning and what we're about as educators.

I still have no doubts that administrators and teachers who hold “Test Pep Rallies” really mean well, and that this practice is one of many practices where schools are trying to adapt to politics that place testing on an undeserved pedestal. Yet, it is even more important today that school leaders avoid falling into the “do-whatever’s-necessary-to raise test-scores” trap. In the end, do we really want to give imperfect tests that kind of weight in our students’ lives? The answer to that is not a question of effectiveness. For me it was a question of ethics 3 years ago, and it is even more so today.

One of the leading educator organizations in this country is calling for an end to the over-reliance on standardized testing in accountability practices.They are calling for what is called "multimetric accountability" by which, according to their press-release includes:

• not using standardized test scores as the "sole measure of student achievement, educator effectiveness, or school quality."
• having an accountability system that "promotes continuous support and improvement and that also is: "public and transparent, includes a range of subjects beyond English and mathematics, and that incorporates non-academic factors such as measures of school climate, safety, and parental engagement."