The 21st Century Principal

"Metric fixation is the seemingly irresistible pressure to measure performance, to publicize it, and to reward it, often in the face of evidence that this just doesn't work very well." Jerry Muller, The Tyranny of Metrics

There is absolutely no doubt that many 21st century education leaders (and politicians) are infected with the metric-fixation disease. The mad illness persists in spite of the fact that no achievement gaps are closing, and no miraculous gains (in their own standardized tests) has occurred.  Perhaps its time find a cure for this persistent disease that is distorting education. The only vaccination against this malady is a sudden jolt of common sense and the realization that not everything worthwhile in this world is measurable. 

Muller, J. (2018). The Tyranny of Metrics, Princeton University Press: Princeton, NJ.

Somehow I am not surprised at all that the Council of Chief State School Officers still believe that we can “test our way to school equity.” They still believe if we “just get the tests right” and hold those teachers’ noses to the grindstones, then “Poof” all our students will receive an equal education. They met and pledged their allegiance to the “Accountability Doctrine” recently in a Council of Chief State School Officers policy forum in Charlotte, North Carolina. (See Ed Weeks Article “State Chiefs Pledge to Continue Focus on Accountability.”) What they don’t seem to get that we had at least 15 years of “Accountability and Testing” and schools are no more equitable, in fact, there’s evidence that they are more unequal than ever.

“Our members want to be held accountable,” said Chris Minnich, the executive director of the CCSSO. That’s nice for him to say, coming from a man who has never spent a single day as a teacher in a public school classroom. If you check out this glowing biography from CCSSO, you can see he knows his policy though. Too bad he has no clue about the nuts and bolts of classroom teaching.

That’s the real problem here. It’s not that the accountability is always a bad thing; its that we have people pushing these damaging policy initiatives who are policy wonks, but know nothing about what No Child Left Behind and current Accountability fetishes have done to schools and ultimately to students. We are not going to make our schools equitable by the having the right tests and accountability systems. We are going to address equity by advocating for a society that does not favor those who have over those who do not have.

To continue my critique of the “accountability and testing regime,” I have been thinking about what the ultimate goals of those whose faith and belief in the promise of standardized testing, statistical technologies, and classroom surveillance are. They have captured the discourse in education and conveniently made unacceptable anything critical anyone else has to say about testing and its high stakes deployment. An educator who questions it is not taken seriously and is deemed out of bounds. Testing and accountability seeks, in a nutshell, to make teaching and learning “measurable, calculable, in order to be controllable."

What does it mean to make teaching and learning “measurable?” It means reducing teaching and learning to “indicators” or “standards” that can simply be determined to be not present or present. It means making teaching and learning into something that can be captured using the available technologies at our disposal, such as teacher observations and standardized testing. Teaching, then, is made measurable by teacher evaluations, and, more recently, using statistical measures such as value-added models, which both result in what is hopefully “objective” and widely accepted as being “true” measures of acceptable teaching and learning, because they happen to be numerical.

As an administrator, I have heard many of my colleagues make the statement, “If it isn’t measurable; it didn’t happen.” That statement captures beautifully the complete faith in testing and measurement that currently exists in education. But it is also a statement of ignorance. Even the best psychometricians will say that “NOT EVERYTHING IN TEACHING AND LEARNING THAT IS WORTHWHILE IS MEASURABLE.” But this faith in “educational measurement” is at the heart of current educational reform, and it is still believed by many educators, politicians, and policymakers to hold the “silver bullet” that will finally make all public education effective. “We just don’t measure enough and measure effectively” is the belief that keeps driving round after round of testing-and-accountability-based reforms” in education. Tests are cheap in comparison to really dealing with the equity issues of healthcare and poverty. With tests and statistical tools, the belief that one can erase these social justice problems, but sadly that is not the case.

For those of us in the schools, those of us in tune with the teachers and students there, we see the results of this: an education system that continues to be distorted and twisted, that ultimately meets the needs of a few, mainly those who can use these “measurable results” to determine their own effectiveness and the effectiveness of their own ideas. An education where test results are still valued over individuals, and any old methodology that results in higher test scores is acceptable. Testing takes precedence over everything else schools do: just look at a state’s testing regulations if you want to see this. In other words, no matter the rhetoric coming from testing and accountability addicts, testing is driving everything in schools, and that’s they way they want it. That keeps them in power and needed.

Making teaching and learning “calculable” is very much akin to making it “measurable.” Making what we do in schools “calculable” is seeking to reduce what we are supposed to be doing to numbers. Somehow, our current system views “numbers” as somehow more objective, therefore superior to other things like judgment or intuition. This desire to make everything “calculable” leads to bizarre decision-making, where outcomes are ridiculously reduced to numerical values, even if those values distort the process and result. Standardized tests do this very well. They can’t measure an “effective essay” for example. Determining whether an essay , or musical composition, or painting is “effective” is by nature a “judgment.” And, whether it is effective in all instances and in all ways is relative. It might be effective at one thing or in one instance, but not another. Rarely are major literary pieces simply “effective for all time” or “in all ways.” The same applies to music, art, and so many other human endeavors. So, in the name of “objectivity,” current testing manics send essays, compositions, and even paintings to “outside” observers to evaluate all in the quest for “objectivity.” But such actions might create a facade of objectivity based on faith, but it completely results in an unfair evaluation of student work. For, who knows better whether a student has progressed than that teacher who has been in the trenches with that student, day after day and seen their incremental growth first-hand. So, the pursuit of making teaching and learning “calculable” is to simply turn it into numerical values or make it have the facade of “objectivity” because the belief is that “numbers don’t lie.” Testing and accountability becomes more about distrust of teachers and their judgments, than really trying to provide an effective education for students. "We can't trust teachers' judgments about students, so must use tests and other outside evaluators," is the rationale.

It is this desire to make teaching and learning both measurable and calculable that leads me to the final goal of accountability and testing as I see it: to make teaching and learning controllable. Policymakers, education reformers, and even politicians all believe they hold the “ultimate vision” of what effective teaching and learning is. They believe, armed with their many contradictory studies on the subject, that they hold the answers. Answers in hand, they seek to control teaching and learning in order to mold it into their image of effectiveness. Through tactics of measurement and calculability based in standardized testing and measurement, they use high stakes decision-making to weed out the “deviant” practices that don’t meet “best practices standards.” The problem lies though with the truth that both teaching and learning is so complex that to reduce it to universal rules of effectiveness ends up distorting it and neutralizing it to simply a “technical knowledge” that anyone can understand, including administrators and policymakers and education reformers who have never spent a day engaging in teaching in classrooms and making decisions about student learning. Teachers, as a result, find themselves engaging in a strangely distorted form of teaching that must jump through the hoops of “best practices” in order to get the “results” desired by this twisted system of education. Teaching the test and test prep are two examples of this distortion. They have become assembly-line workers who “add” knowledge to students as they roll down the assembly-line, and testing with this value-added component is the “quality control mechanism” that drives teachers in the entire system to produce even more “globally competent graduates" that can produce ‘number one test scores’ on international tests such as PISA. Under the testing and accountability regime, teachers are reduced to technicians whose judgement does not count and means nothing. Test results and other “quantitative” measures are hierarchically superior.

In the end, if you wanted to design an education system that turns education into a factory-like system that produces standard results, you couldn’t have done better with that created by our current accountability and testing regime. If you wanted to create a system that transforms and de-professionalizes teaching as a profession, you can’t do much better. In the end, our public education system might ultimately match up to the vision of those who adhere avidly to accountability and testing practices, but I can’t help but wonder whether those teachers in this system find the same level of satisfaction and dedication to students when test results are valued so highly. I also have to wonder what kinds of students such a system of this really produces. Perhaps, that’s what’s desired by accountability and testing advocates: they want students who don’t question; who don’t criticize; who don’t engage in learning deemed irrelevant such as the arts, and learning seen as deviant. They want both students and teachers who “just do their jobs” and not engage in dreams of how things might be different or better.

I feel a bit obligated to explain why there have been so few posts to The 21st Century Principal blog this year. I am continuing my doctoral work through Appalachian State University, so I’ve spent countless hours reading about the French philosopher Michel Foucault and value-added model research. Now I am sure someone might want to ask what could these two subjects possibly have in common?

Well, I am working on a poststructural analysis of current accountability practices. What I hope to be able to do expose even more of the bizarreness behind our continued fetish with using standardized tests to measure everything in education. Somehow, we in the United States just can’t let go of this “If-it-breathes-let’s-test-it approach to education. The faith that if we somehow are able to find “just the right standards” and the “right tests to measure them,” our students will excel in school in life remains strong, and the United States will be number one in international tests, and all our students will find companies just dying to give them high paying jobs because of their superb test performance. I hope you notice the sarcasm.

I just don’t have the time to write blog posts like I was, but I am still reading and writing and learning. And, I am still just as critical of our accountability and testing fetish as ever.

What happens when all that matters are test scores? Just ask former El Paso Superintendent Lorenzo Garcia who was sentenced by a federal judge to three and half years for a test-cheating scandal(See “Former EPISD Superintendent Lorenzo Garcia Gets 42 Months, Offers No Apologies for Scandal.”) Don’t get me wrong, I don’t shift the blame for one minute away from from Garcia. As a leader, compromising your morals to cook the books is never excusable, even if there’s pressure from elsewhere to do so. Yet, Garcia’s actions are understandable in a corporate American culture where “cooking the books” in order to deceive investors is acceptable. After all, is that not what the financial meltdown was about? Was it not about Wall Streeters who hid toxic loans from investors while making exorbitant salaries? “Cooking the books” has become an American management strategy, so should anyone be really surprised when school leaders like Garcia, or Atlanta Public Schools Superintendent Beverly Hall does just that to make their school systems appear to perform better than they really are?

In his book, Who’s Araid of the Big Bad Dragon: Why China Has the Best (and Worst)Education System in the World, Yong Zhao blames the accountability system based on high stakes testing. True. This obession with test scores in the United States is unhealthy. Our politicians and state education leaders have convinced many school leaders that obtaining higher test scores is the ultimate goal and product of school systems. No wonder there are cheaters who cook the books of school accountability to make it look like their districts are performing better than they really are. Accountability in the United States is evolving into the same game that Wall Street bankers play; cook the books to make it look like things are better than they are.

But at the end of the day who are these who cook the books really fooling? Are our students really learning more than they ever had? Are we actually producing the best graduates we have ever produced? Is our graduation rate really any higher than it has ever been? In some ways, I am afraid accountability in education has become a game that educators play. In a culture where numbers ultimately matter more than kids, education has adopted the exact same thinking that Wall Street adopted; whatever you can do to make your bottom line appear better than actually is becomes acceptable. Instead of focusing genuinely on the kids, accountability is game of data manipulation.

Garcia and Hall did a great disservice to education by cheating. Their actions are inexcusable. Yet, there are other educators still playing the game of accountability, shifting data points around, in order to make it “appear that their school or district” is on top.

Wall Street's “book-cooking” tactics have no place in education. What we do as educators determines the course of young lives, and when our focus shifts from that to numbers alone, we too are as guilty as those who wrecked the economy, and who “cook the books of accountability” for appearances sake.

What happens when schools and school districts let test scores rule the day? Check out letter of resignation by a 14 year veteran teacher from Tennessee who was found "ineffective" due to her TVAAS ratings.

"What Defines an 'Ineffective' Teacher?"

What immediately becomes clear to me is the following:

• The problem with the accountability in education movement, like in this case, is that too many educators, politicians and state level testing bureaucrats think "any old test will do" when it comes to obtaining data for teacher ratings. Little time is taken to check to see if the tests really test what is being taught, and whether or not using the tests to evaluate teacher effectiveness is even valid. In this accountability madness, there is always an assumption that test data does not lie and that it's objectivity is a given. Both are wrong. Test data is just numbers, but the inferences, like teacher effectiveness, we make from those numbers can be wrong. Any old test will not do when high stakes of any kind are attached.
• Our education system has become blinded by its own test data. In other words, our education decision-makers have blinders on because it is somehow seen sacred that all decisions should be tied to data, and good data are test scores. You can't rely on teacher judgment because it is tainted with subjectivity, so those making these bizarre decisions about accountability through test scores and value-added measures immediately discard everything else. In this case, and I fear in many others around the country, the teaching profession and our education system is being destroyed by "testing fundamentalists" who have become blind to reason and to the possibility that their teacher evaluation systems are hurting real people and even that they might be wrong.
• The "testing fundamentalists" are beginning to see the fruits of their blind, ideological belief in sacredness of test score data. When test scores matter above all else, educational decision-making gets just "plain stupid." It is just plain bizarre that a 14 year veteran teacher was coached by an instructional coach WHO WAS ONCE HER STUDENT TEACHER. The person she mentored four years earlier is now mentoring her. When educators blindly follow the data trail, they end up in bizarre situations like this one.

Recently, I received an anonymous comment on my blog post entitled “Merit Pay in Education: An Exercise Manipulation and Futility.” I have purposefully chosen to not publish anonymous comments on this blog because I believe firmly that if you have something to say, then you should be willing to divulge your identity. Part of your message is who you are, and hiding your identity is actually hiding part of your message. At any rate, the commenter who calls him or herself “Engaged Parent” seemed to practically dare me to publish his or her comment. I won’t publish it as a comment, but I will share it here with my own commentary because there are some fundamental misconceptions apparent in that comment. Here’s “Engaged Parent’s” comment, then I will respond.

Even though you're going to NOT publish this comment I think it's worth sending it to you anyway. A teacher you are yes? I'm going to assume that.

Over and over again I see the same comments on how we can measure the success of education. It's great that we keep hearing how a merit program doesn't work but what's the alternative? It would be nice (for once) to hear a teacher tell us how a "bad" teacher can get filtered out of the system. I'm sure you will agree that not all teachers are good, there are always some bad apples in the bushel, no way around it. 
I agree, test scores are not always a good judge of how a teacher is doing but it does or it at least should give key indicators to a teacher that maybe what they're doing isn't the best for that particular mix of kids and that another approach might work better. But do they do that? I don't know.
Why are teachers the only ones who don't have to be put under the scrutiny of evaluation? The rest of the world has to go through it.

What I really want to say is I think that a happy medium could be met if careful thought was put in to it.
For instance:
10 % what did your students think of you?
15% what did your students parents think of you?
25% test scores
10% peer scores
15% self score
25% principal scoresIs that not a fair assessment? If not then come up with SOMETHING because personally I'm sick of hearing how teachers don't want to be assessed.

First of all, let me say that yes, I am a teacher, and I don’t think I’ve hidden that fact anywhere on this blog. You can find that information on the blog sidebar. I am currently a principal. As a teacher though, I am more concerned with your logic and misconceptions in your comment than anything else. First of all, you seem to suggest that merit pay should be implemented because we have no alternatives. Following this argument means that the rationale for implementing merit pay lies, not in whether it will work or not, but because we don’t have any other alternatives. That in itself is faulty reasoning. I certainly hope my physician doesn’t implement a treatment or my mechanic doesn’t simply initiate a car repair because he or she says, “What’s the alternative?” without considering the evidence of symptoms and lab test results that identify the problem.  You are hearing criticisms of performance pay because it has been researched and has failed to bring about the improvements sought, which is increased student learning. It has been tried in public education, even in North Carolina with no appreciable effect on the quality of education.

You next assertion seems to be thinly veiled when you state “It would be nice (for once) to hear a teacher tell us how a “bad” teacher can get filtered out of the system.” It appears you have a belief that “bad” teachers never get dismissed. I would agree with you that not all teachers are good, which I gather you really mean effective at helping students learn among the many other things teachers do. But contrary to your belief, they do “get filtered” out of the system, at least in my experience. So your belief that teachers who don’t do their jobs very good somehow are not subject to dismissal is another misconception. Teachers can be dismissed fairly but with “due process,” which means that I as an administrator must thoroughly document my rationale for doing so. Many administrators see “due process” or tenure as many call it, as an obstacle, but due process rights were put in place because historically, our education system was notorious for political firings and reprimands. Teaching historically has been quite political with school board members or even administrators firing teachers for noxious reasons. Teachers have been fired for being pregnant or even so a school board member can then hire a son or nephew. At any rate, it is a misconception that teachers who aren’t doing their jobs can’t be fired. It simply takes leadership and willingness to first try to help that teacher improve, then take the steps necessary to counsel them into another profession.

Still another misconception from you comment is that teachers aren’t under the “scrutiny of evaluations.” I have been an educator in North Carolina for 25 years, and I have always been subject to evaluations, so the idea that teachers aren’t somehow evaluated just isn’t true. Teachers in my state have been evaluated for years, and some are dismissed as a result of those evaluations. When you hear all the criticism of current evaluations systems, those do not come from the desire to avoid evaluations; it comes from the desire to have those evaluations be fair. You yourself acknowledge that test scores alone are not always a “good judge” of teacher, but there are much deeper issues with using test scores as a part of evaluation, such as the fact that more 70 percent of courses in a high school do not have these state tests. Teachers that I know aren’t trying to avoid being evaluated as you suggest; they simply want those evaluations to be fair and just, as any employee in any line of work would want.

Now, let’s take a look at the evaluation system you suggest. You provide several interesting sources of evidence for teacher evaluations. First of all, you would base 10% of the evaluation on “what students thought of the teacher.”  I hope you aren’t suggesting that students rate the teacher as a person. I suspect you are really suggesting that students are somehow surveyed on what kind of job they think their teachers are doing. This is reasonable in some ways, and I’ll agree with you. But there obviously has to be careful attention paid to the survey instrument so that its questions get at the heart of instructional prowess and not opinions about the teacher personally. The question then becomes, what do students know about a teacher’s instructional ability? The answer to that question could be used as a basis for survey questions. Our school has used student surveys for over 5 years, and one quite common problem is that the responses can sometimes be about anything other than the teaching. The data is useful, however, because it can help us make changes where there are genuine complaints and issues.

The second source of evidence for teacher evaluation you suggest is “what the parents think of their teachers.” Again, that seems reasonable, if what you are suggesting is a parent survey that focuses on a teacher’s ability to teach. For obvious reasons, the survey would need to move beyond asking the parent what they thought of the teacher as an instructor. But there are some issues that would need to be ironed out. For example, one issue that would need to be dealt with would be the fact that few parents witness the actual teaching teachers do in the classroom. They get most of their information about what happens in the classroom secondhand. I can tell you as an administrator that quite often  that secondhand information isn’t entirely accurate. A student often goes home and tells a parent one story about something that’s happened in their classroom or to them or what a teacher has done. That parent next calls the principal, clearly angry, until they hear what really happened. Using parent opinions about teacher practice would be especially difficult since they do not witness teaching in action; they have to rely on hearsay, which as we know is unacceptable in court rooms. Parent surveys could focus, however, on the parts of teaching that they do directly witness, such as teacher to home communication. At any rate, if parent surveys were to be used in teacher evaluations, they would need to be much more than simply asking a parent what they thought of his or her teacher.

The third source of evidence you suggest for teacher evaluations are test scores. That also seems reasonable until you try to implement that practice which we’ve done in North Carolina.  The issues are many. Some of the tests are inferior and are of questionable quality. Then there’s the fact that not all subjects are tested, which leads to questions like: Do you evaluate only those in tested areas, or do you develop and administer tests in every single subject area, which is quite costly in terms of time and money? In addition, some tests were designed in a way that they make poor choices for use in evaluating teachers. Tests like ACT and SAT were created for an entirely different purpose rather than measuring teacher quality. Additionally, tests designed for assessing student achievement have never been proved valid for assessing teacher quality. Other issues with test scores used as part of teacher evaluations? How do you separate the effects of previous years teachers on this year’s teacher scores? The list of issues with using tests as evidence of teacher effectiveness is lengthy, that’s why teachers and myself question the practice. A test is a test is a test is how much of the world sees it, but most of us who’ve been in education know tests aren’t as simple as we wish they were.

The fourth source of evidence for teacher evaluations that you suggest are “peer scores” which I suspect you actually mean peer ratings. Peer ratings seem to make sense too, after all, who would know better than a teacher’s peers whether or not they are effectively teaching. But the problem with this one is similar to the parent one: there are few teachers who actually know how effectively a teacher is teaching, unless of course you set up a system by which they peer observe. That would work, but it offers logistical problems such as finding time and means for each teacher to observe each other in action. Currently, we use peer observations in evaluations as a part of our process so your suggestion isn’t far from current reality. There is one additional issue with peer ratings too; teachers are often very reluctant to honestly rate their peers because quite often, they have to work with these individuals, and they are often their friends. You have to be pretty naive about human nature to see peer ratings working very well.

The fifth source of evidence for teacher evaluations you suggest are “self scores” which I would suspect you mean self ratings. Believe it or not, we already ask our teachers to do “self-assessments.” While these aren’t directly connecting to a teacher’s final rating, they are used as a part of the evaluation process early in the year. Teachers use them to honestly look at how they stand with our teacher evaluation, and many use it as a basis for their professional development plan. Of course it’s like any self-assessment; it’s only as useful as the amount of honestly employed in its completion. The main problem with self-ratings is if high stakes are tied to it in any way, how noble do you think someone would be to give themselves a poor rating if it could impact their job status?

Your final source of evidence for teacher evaluations you suggest are “principal scores” which I take to mean principal ratings. In North Carolina, this is practice. Principals complete a summative evaluation of ratings on teachers at the conclusion of every year. This is in turn used to suggest professional development for the next year. Principal ratings are certainly not without issues as well. Most of us have worked for bosses who were tyrants and completely incapable of making fair and partial judgments. Because of this danger alone, principal ratings have issues too.

I would say “Engaged Parent” you do get one principle right in your evaluation suggestion; we do need multiple sources of evidence for teacher evaluations. The issue is simply deciding which ones will effectively improve instruction for our students. We do need multiple sources of evidence to evaluate teachers, but if we choose the incorrect evidence, then our teachers don’t teach well and what we set out to do which was improve student learning never happens. We can’t afford to get teacher evaluation wrong because in the end our children will suffer.

So, “Engaged Parent” we do have evaluations of teachers. We also use much of the evidence you suggest above, and there’s talk of adding other sources as well. Teachers as a rule in North Carolina do not complain about being evaluated. Like any employee they do complain about evaluated unfairly, and that is a great deal of the criticism you’re hearing. I would suggest that since you have a great interest in teacher evaluations, you might want to read a book called Evaluating America’s Teachers Mission Possible? by W. James Popham. Popham goes to great length to argue what a good, fair and valid teacher evaluation would look like. He even goes through about every single source of evidence you suggest and points out the problems with them and suggests how those problems might be alleviated. It’s a good read for anyone, educator or non-educator, looking into the evaluation of teachers.

Today,  there was this post on the Public School Forum of North Carolina website defending the Common Core. That post, entitled "Common Core Standards Will Help NC Meet Its Challenges," was written by Caroline McCullen, the director of education initiatives at SAS which already has a multi-million dollar contract with North Carolina to provide value-added teacher data analysis software for teacher and principal evaluations.

Let me say up front in the interest of disclosure. I am a skeptic about the Common Core, not because I do not believe in the need to have higher standards. Also, who knows, the scheme might ultimately bring about some positive change in education. But, as a skeptic and admitted cynic about the reasons the Common Core were developed, my cynicism says that perhaps these standards were developed as a commercial opportunity for corporations like Pearson and other test companies to make more money off the public school systems. After all, as Diane Ravitch points out, those tasked for developing these standards "contained few educators, but a significant number of representatives of the testing industry." (See "Everything Your Need to Know Know about the Common Core.") Now, Ravitch has made it clear that she doesn't support the Common Core either. However, I will set aside my own skepticism and cynicism and look carefully at the arguments used to defend the Common Core the post.

• The Common Core is "more rigorous than North Carolina's previous standards."
• The Common Core is "benchmarked against college and career readiness measures.."
• The Common Core is "endorsed by governors of both parties across the country."
• The Common Core will "assure North Carolina citizens and business community that our students and future employees are challenged in the classroom and prepared to meet to compete in the global economy."
• Because North Carolina NAEP scores declined in 2009, the state's standards "were lower" there the Common Core standards are needed.
• The Common Core is needed because we need more rigorous tests so that we can compare students across the state and across the country.

“We need to be careful that the tests we use in a properly designed teacher-appraisal system do, in fact, contribute to a valid (that is accurate) inference about a teacher’s quality.” W. James Popham, Evaluating America’s Teachers: Mission Possible?

North Carolina took the plunge this year and started using test scores as part of teacher and principal evaluations. The state has even invented a "new" kind of test, called a "Measure of Student Learning" in order to make sure there is plenty of test data to go around. What is particularly telling is how "carefully" the state crafted the term "Measures of Student Learning." It's as if somehow, not calling it a test, makes it not a test. State level educational logic never ceases to amaze me. Of course, the state then started calling these "Measures of Student Learning" something else. They started calling them "Common Exams." Notice again, the careful use of the word "exam" rather than "test." It's almost as if you don't call it a test, it isn't a test, but apparently state level policymakers haven't heard the old saw about a rose still being a rose even if it has another name.

Besides North Carolina's struggle with what to call their newly implemented tests, there's still the question of what the unintended consequences of having thousands of teachers "teaching to the test" is going to do for students in our state. Ultimately, being able brag that your students "Have the best scores in the world" is most likely what politicians and state level education officials are after. That's why they see salvation through test scores as the means to the "Educational Promised Land." Ultimately, there's a flawed logic driving this whole accountability and testing movement: it's the whole idea that learning can be entirely reduced to bubble sheet answer sheets and taken in a single sitting. And, that teachers can't be trusted to tell when a student has demonstrated that they have learned or not.

In my years as an educator, I have been amazed how trusting and accepting educators in North Carolina are when it comes to the latest policy flowing down from on high. It's as if they accept that those at the state level know more than they do, or somehow have access to magical information they do not have. So, when they implement something like the use of test scores in evaluations, many educators accept that the powers that be at the state level know what they are doing, so they trust them. Given the history of reform ideas and educational policy that travels down from on high, this "trust" is highly misplaced. I like to think that state level education officials mean well, but what often has happened during my career, these ideas when implemented locally have sometimes been a disaster and have been sometimes downright bad for kids. Instead of being so trusting, I submit that all educators in the schools and districts need to become skeptics and ask tough questions of our state-level, and federal level policymakers. We should never accept the "trust me, this will work" answer.

It is in this spirit of skepticism, I turn to Popham's book, Evaluating America's Teachers: Mission Possible? and our state's venture into making high stakes testing even more high stakes. In spite of what our state-level policymakers say,  I am not fully satisfied that North Carolina's tests are adequate measures of educator effectiveness, and  a healthy skepticism is still in order. This whole push to add test scores to teacher and principal evaluations has been a rush from the start. Depending on when you asked questions, how the tests were to be implemented has changed multiple times throughout the last two years. Never mind the fact that not a single teacher in North Carolina even saw the test before they were implemented. In their rush to have "test data" it's as if our state level policymakers think "any old data will do." They have failed to take the time to establish whether any of these tests really tell us anything about teaching quality.

In light of our state's push into "higher stakes testing", I think Popham reminds us of some important key issues and ideas about tests and teacher evaluations that state politicians and policymakers seem to forget.

• “Tests are not valid or invalid. Instead, it is a test-based inference whose validity is at issue.” In other words, it isn't the test that’s valid or invalid, it is the inferences drawn from those tests that have these qualities. It boils down to whether you can actually make an inference based on the test or not. The question is whether North Carolina's tests, which have been implemented haphazardly and a thrown-together-manner, actually tell us anything at all about the quality of teaching in our classrooms. Can I honestly say Teacher A is a good teacher because she added "this much" value to her students' Measures of Student Learning? Seems to me that it puts a great deal of faith in a single test.
• “Tests allow us to make inferences about a test taker. This inference, depending on the appropriateness of the test as a support for the inference being made, may be valid or invalid.” As Popham points out, the inference we make about the learner may be valid or invalid depending on the “appropriateness of the test” in its role to support the inference being made. As we know, the word validity is the extent to which that inference, or conclusion, is well-founded or corresponds to the real world. This boils down to whether the inference we draw about a student is valid or not. For example, should we infer, based on a student’s test scores that he is not proficient in the subject, we must be satisfied that the test we are using is the “appropriate measure,” and we must also make sure the conclusion we draw considers all real world facts. Ignoring a student’s socio-economic status, or even whether he experienced  a death in the family, can make our inference about the student’s proficiency invalid. Then there's the whole issue about making an inference about a teacher or principal's effectiveness using this same test. Has North Carolina sufficiently established the appropriateness of their Measures of Student Learning, End of Grade Tests, End of Course Tests, as instruments that allow for making inferences about teacher and principal quality? I'm not sure they have. Another question, do these Measures of Student learning allow us to make valid inferences about teacher quality? I'm not convinced they do.

As North Carolina moves forward with a teacher and principal appraisal instrument that uses test scores to determine effectiveness, all educators need to educate themselves and scrupulously ask questions of policymakers at all levels.

As Popham suggests, “If heavy importance is being given to students’ performances on state tests for which there is no evidence supporting such an evaluative usage, then teachers (I would add principals too) might wish to engage in further study of this issue so that, armed with pertinent arguments, they can attempt to persuade educational decision makers that more appropriate evidence should be sought.” In other words, all educators, administrators, and teachers need to study how North Carolina or any state is using test scores to determine educator effectiveness.

Administrators owe it to their teachers, and themselves, to understand that some of these tests were never designed to determine educator effectiveness, so that data needs to be viewed with skepticism. And, I would add that the manner in which these Measures of Student Learning were developed and are administered may not allow them to draw valid inferences about teacher quality. Test scores in North Carolina currently are only 1/6th of the teacher evaluation, and effective administrators are going to keep this in mind and not let the allure of numbers numb them to the other 5 standards.

The one lesson politicians should have learned from No Child Left Behind, is that when all of your energies and resources are turned to just improving test scores, failure is the result.  As education historian Diane Ravitch states eloquently in her book, The Death and the Life of the Great American School System, ”Our schools will not improve if we rely exclusively on tests as the means of deciding the fate of students, teachers, principals, and schools.” Sadly, I’m not sure our current political leaders have learned the lesson yet that schools will not improve by solely focusing on using test scores and standards to improve them.

Our national education policy is still dominated by a “test-centric” approach to reform that ignores so many other factors that impact education such as poverty, inadequate health care, and lack of gainful employment. According to education scholar Linda Darling-Hammond, “The United States has the highest poverty rate for children among industrialized nations,” (The Flat World and Education, Linda Darling-Hammond, 2010). We want to “Race to the Top” but we’re looking for short cuts to get there. We want standards and “better tests” but we don’t want to engage in the hard, difficult work of addressing poverty, lack of health care, lack of good, affordable housing, and lack of opportunity for jobs with living wages. As long as national education policy is driven by a blind belief in test results and national standards, 10 years from now, we will be either staring at the same dismal  conditions both educationally and economically if we’re lucky, or we will be much worse with a society with an even wider gap between those that have and those that have not.

What then is the answer? Just how bad are things in different parts of the country? This morning I stumbled upon a 5-year initiative by the American Federation of Teachers and partners like Cisco, Blue Cross Blue Shield, College Board, among many others, that focuses on the educational improvement of an entire community ravaged by unemployment, lost opportunity and lost promise. McDowell County West Virginia has not fared well at all since 1980 and that community is the focus of this initiative.


While it is easy to become entangled in the debate about the role of teachers unions in education when debating education policy, I think it is admirable that the AFT and its partners are putting into practice what they’ve been trying to make politicians understand all along; education reform must do more than focus on test scores and standards. It has to also address the dreadful conditions some of our fellow US citizens find themselves living in.

With this post, I am not taking sides in the debate about unions per se. I do believe, after 20+ years experience, and seeing countless students struggling to live in forgotten communities without the basics most of us take for granted, that the answer to our problems as a country lies, not in investing in more and different tests, or in national standards, but in focusing on the crushing problems facing our poorest students.

After watching the video below about “Reconnecting McDowell” I was reminded of an incident that happened in one of the schools where I once worked. I walked by a table during lunch one day, and a young 11 year old girl sat there with her head down. She had enormous tears in her eyes. I walked up, leaned down and asked her to step out the lunchroom for a minute. Once out of the hearing of others, I asked, “What’s wrong?” Through her tears, she blurted, “I don’t have any lunch money. My parents didn’t have any to give me.” She proceeded to tell me that when she went through the lunch line, the cafeteria took her plate away and refused to serve her lunch because she owed so much money. I took her back through the lunch line and told her to get anything she wanted, and that it would be taken care of. You can debate all you want about why a child does not have money to eat. You can accuse her parents of not taking care of her, but the reality for her is she was not going to be able to eat that day, and a focus on raising her test scores was not going to change that reality.

As I understand it, Reconnecting McDowell is an effort to try to improve the education of a community, and not do it by just focusing on test scores. It is an effort to focus on poverty, healthcare, housing, and  a broken community. I have been to McDowell County West Virginia and have seen firsthand all that the video describes. That is why this effort caught my attention. Poverty is real, and those of us who have worked in schools where it exists know its faces.


Link to Reconnecting McDowell Web Site.