The 21st Century Principal

"You can prep kids for a standardized test, get a bump in scores, yet not be providing a very good education." Mike Rose, "The Mismeasure of Teaching and Learning: How Contemporary School Reform Fails the Test"

It should not be a surprise at all to politicians, policymakers, and educators that the cry and backlash against testing and accountability is growing. During my 25 year career, I've seen the number of state tests administered in public high schools in North Carolina grow from 1 to well-over 2 dozen. Testing and test scores are the talk, and the focus is almost always on "how can we get those test scores up?" About the only ones, with the exception of a few teachers, who are enthusiastic about all this testing are school administrators, who for the first time have a "cattle prod" as Taubman calls it in his book Teaching by Numbers, to shock those teachers who get out of line and who aren't "producing." Blind acceptance of test scores as "the only data of importance" is common, because such data is seen as an "objective" measure, another myth perpetuated by testing and accountability supporters. But is that true?

Peter Taubman's book Teaching by Numbers: Deconstructing the Discourse of Standards and Accountability in Education points out some flaws with this blind acceptance of testing as "the data" on which to base all educational decisions. He writes,

"Fundamentally, tests provide little more than data, but just as one must question the confessions extracted under torture, one has to wonder just how reliable that data is, when it is wrung out of students shocked by the constant administration of tests." 

In other words, no one questions, least of all school administrators, this "data" we're looking at as measures of teacher, school effectiveness and student learning. Tests are data, but how good is that data when students have been subjected to test-after-test-after-test as we do in high schools in North Carolina? North Carolina education leaders truly believe in the maxim, "If it breathes; test it." Data collected under the oppressive, tortuous testing system in our state isn't foolproof, and our jobs as administrators, educators and teachers is to remember that when we start looking at numbers.

There's no doubt when our state education leaders, administrators use the phrase "accountability" they mean primarily multiple choice tests designed to keep teachers, administrators and whole schools in line. As Taubman writes again,

"All too evident, accountability translates into teachers' responsibility for their students' learning as measured by performance on tests." 

I would add that when our state leaders and most administrators use the phrase student achievement they are only speaking about test scores. The testing math in North Carolina is captured by this equation:

Student Achievement=Test Scores

Reducing learning to a test score is great if you are accountant, but for those of us who know teaching, we know that genuine learning is rarely, if ever, only contained between the letters of a multiple choice question. Real, worth-while learning is not always subject to being captured on a a standardized test.

Administrators love tests though, and with the same enthusiasm that politicians do. Why is this? I think Taubman once again hits the bullseye. He writes,

"One reason administrators are sympathetic to testing, the data it generates, and various practices connected to testing and data aggregation is that these provide control from a distance, a fundamental component of what is called audit culture." 

Testing allows principals to become accountants, district leaders to become accountant managers, and superintendents and state level leaders become CEOs. Through test scores, all levels of administrators finally have a tool to control what happens in classrooms. They can dictate how teachers can act, and even in some school systems, teachers are given scripts to follow to make sure they cover what is to be tested. Increase the number of tests administered and you control more and more of what happens in schools. All that talk about allowing teacher decision-making, but holding teachers accountable for those decisions is just empty rhetoric. Tests are measures of control and compliance, and they are gradually strangling public education. Testing finally gives administrators what they think is an "objective" tool for getting rid of teachers and for making sure everyone is compliant.

Test data also gives administrators at every level "bragging points." It gives them something to boast about to the public, to business, to industry, and to politicians. Never mind that testing almost always reduces teaching and learning to only what can tested. Taubman gets it right once again when he writes,

"Tests constitute one way the educational reforms show the educational system. Extracting data from students, teachers and schools, they force our noses into the bottom line. Keeping us under constant surveillance, they make us vulnerable to centers of control beyond our reach, and, providing the illusion of objective accountability and meritocracy, they reduce education to right answers and information." 

Testing is about keeping teachers and students noses to the bottom line. It is about using the "illusion of objective accountability" to make sure no one gets out of line.

There is no question that this accountability and testing culture is negatively affecting teaching as a profession as well. According to Taubman,

"High stakes tests erode the autonomy of teachers, for if tests determine the curriculum, and if tests tell us what is important to know as a teacher, and if these tests are fabricated by centers of control beyond the reach of teachers, then the teachers' passions, commitments, and wisdom count less and less." 

As mentioned earlier, accountability and testing is in some case reducing the act of teaching to little more than a "scripted lesson." Instructional delivery is simply following the state or district lesson plan. Teacher autonomy due to the massive testing load is at an all time low. Teaching is no longer a profession; it is a factory job, whose goal is to churn out test scores. If a teacher fails to "make production," they are branded "Ineffective" or "Not Making Expected Growth" as its called in North Carolina.

What is more amazing is that state educational leaders just don't get it. Enrollments in education programs in colleges and universities for training teachers is at an all time low, and it isn't just about salaries. Teaching is just not very attractive when your job is test-score production. Talk to any students about becoming a teacher, and they laugh in your face. Even worse is when you find yourself as an educator no longer encouraging young people to become teachers because being a public educator anywhere, much less North Carolina, has been robbed its ability to be satisfying career because too much emphasis is placed on accountability and testing.

Where does all this end? I wish we knew. North Carolina, as do other states, continues to ramp up its testing by adding new tests, and the state stubbornly hangs on to its massive testing regimen. Will it be when there's no one entering teacher education programs in our state? Will it be when there is no one with more than 10 years experience left teaching in the classroom? Or, will it be when parents, students, and teachers finally push back and say they've had enough? Testing and accountability is more oppressive than ever in North Carolina and elsewhere, and it is sinking public education and the teaching profession along with it.

“We need to be careful that the tests we use in a properly designed teacher-appraisal system do, in fact, contribute to a valid (that is accurate) inference about a teacher’s quality.” W. James Popham, Evaluating America’s Teachers: Mission Possible?

North Carolina took the plunge this year and started using test scores as part of teacher and principal evaluations. The state has even invented a "new" kind of test, called a "Measure of Student Learning" in order to make sure there is plenty of test data to go around. What is particularly telling is how "carefully" the state crafted the term "Measures of Student Learning." It's as if somehow, not calling it a test, makes it not a test. State level educational logic never ceases to amaze me. Of course, the state then started calling these "Measures of Student Learning" something else. They started calling them "Common Exams." Notice again, the careful use of the word "exam" rather than "test." It's almost as if you don't call it a test, it isn't a test, but apparently state level policymakers haven't heard the old saw about a rose still being a rose even if it has another name.

Besides North Carolina's struggle with what to call their newly implemented tests, there's still the question of what the unintended consequences of having thousands of teachers "teaching to the test" is going to do for students in our state. Ultimately, being able brag that your students "Have the best scores in the world" is most likely what politicians and state level education officials are after. That's why they see salvation through test scores as the means to the "Educational Promised Land." Ultimately, there's a flawed logic driving this whole accountability and testing movement: it's the whole idea that learning can be entirely reduced to bubble sheet answer sheets and taken in a single sitting. And, that teachers can't be trusted to tell when a student has demonstrated that they have learned or not.

In my years as an educator, I have been amazed how trusting and accepting educators in North Carolina are when it comes to the latest policy flowing down from on high. It's as if they accept that those at the state level know more than they do, or somehow have access to magical information they do not have. So, when they implement something like the use of test scores in evaluations, many educators accept that the powers that be at the state level know what they are doing, so they trust them. Given the history of reform ideas and educational policy that travels down from on high, this "trust" is highly misplaced. I like to think that state level education officials mean well, but what often has happened during my career, these ideas when implemented locally have sometimes been a disaster and have been sometimes downright bad for kids. Instead of being so trusting, I submit that all educators in the schools and districts need to become skeptics and ask tough questions of our state-level, and federal level policymakers. We should never accept the "trust me, this will work" answer.

It is in this spirit of skepticism, I turn to Popham's book, Evaluating America's Teachers: Mission Possible? and our state's venture into making high stakes testing even more high stakes. In spite of what our state-level policymakers say,  I am not fully satisfied that North Carolina's tests are adequate measures of educator effectiveness, and  a healthy skepticism is still in order. This whole push to add test scores to teacher and principal evaluations has been a rush from the start. Depending on when you asked questions, how the tests were to be implemented has changed multiple times throughout the last two years. Never mind the fact that not a single teacher in North Carolina even saw the test before they were implemented. In their rush to have "test data" it's as if our state level policymakers think "any old data will do." They have failed to take the time to establish whether any of these tests really tell us anything about teaching quality.

In light of our state's push into "higher stakes testing", I think Popham reminds us of some important key issues and ideas about tests and teacher evaluations that state politicians and policymakers seem to forget.

• “Tests are not valid or invalid. Instead, it is a test-based inference whose validity is at issue.” In other words, it isn't the test that’s valid or invalid, it is the inferences drawn from those tests that have these qualities. It boils down to whether you can actually make an inference based on the test or not. The question is whether North Carolina's tests, which have been implemented haphazardly and a thrown-together-manner, actually tell us anything at all about the quality of teaching in our classrooms. Can I honestly say Teacher A is a good teacher because she added "this much" value to her students' Measures of Student Learning? Seems to me that it puts a great deal of faith in a single test.
• “Tests allow us to make inferences about a test taker. This inference, depending on the appropriateness of the test as a support for the inference being made, may be valid or invalid.” As Popham points out, the inference we make about the learner may be valid or invalid depending on the “appropriateness of the test” in its role to support the inference being made. As we know, the word validity is the extent to which that inference, or conclusion, is well-founded or corresponds to the real world. This boils down to whether the inference we draw about a student is valid or not. For example, should we infer, based on a student’s test scores that he is not proficient in the subject, we must be satisfied that the test we are using is the “appropriate measure,” and we must also make sure the conclusion we draw considers all real world facts. Ignoring a student’s socio-economic status, or even whether he experienced  a death in the family, can make our inference about the student’s proficiency invalid. Then there's the whole issue about making an inference about a teacher or principal's effectiveness using this same test. Has North Carolina sufficiently established the appropriateness of their Measures of Student Learning, End of Grade Tests, End of Course Tests, as instruments that allow for making inferences about teacher and principal quality? I'm not sure they have. Another question, do these Measures of Student learning allow us to make valid inferences about teacher quality? I'm not convinced they do.

As North Carolina moves forward with a teacher and principal appraisal instrument that uses test scores to determine effectiveness, all educators need to educate themselves and scrupulously ask questions of policymakers at all levels.

As Popham suggests, “If heavy importance is being given to students’ performances on state tests for which there is no evidence supporting such an evaluative usage, then teachers (I would add principals too) might wish to engage in further study of this issue so that, armed with pertinent arguments, they can attempt to persuade educational decision makers that more appropriate evidence should be sought.” In other words, all educators, administrators, and teachers need to study how North Carolina or any state is using test scores to determine educator effectiveness.

Administrators owe it to their teachers, and themselves, to understand that some of these tests were never designed to determine educator effectiveness, so that data needs to be viewed with skepticism. And, I would add that the manner in which these Measures of Student Learning were developed and are administered may not allow them to draw valid inferences about teacher quality. Test scores in North Carolina currently are only 1/6th of the teacher evaluation, and effective administrators are going to keep this in mind and not let the allure of numbers numb them to the other 5 standards.

"Who would want to teach in a system that measures your worth as an educator by how much your students can regurgitate on a two-hour multiple-choice test and that has reduced much of the curriculum to tedious test-prep exercises?" writes Tony Wagner in his latest book entitled Creating Innovators: The Making of Young People Who Will Change the World.

Who, indeed, would want to teach in the kind of education system Wagner describes in this question? Perhaps a better question would be, "Who would want to learn in such a system?" Yet, with all the increased emphasis across the country on test scores as part of both teacher and principal evaluations, the kind of education system Wagner predicts is already coming to fruition, and my state is on the fast track to such a system.

North Carolina schools have had a "Learning Culture" characterized as "getting students ready for THE TEST" since it began rolling out state tests in the 1990s. Now, North Carolina has moved from using those same tests to determine student proficiency to determine teacher and principal proficiency. And, for subjects that do not currently have a standardized test, they are creating a TEST, to not measure student learning, but to measure educator proficiency. The end result  of these measures will obviously be that the state that once proclaimed proudly "First in Flight" on its license plates, can soon declare "First in Test-Prep." 

Sadly, though, one can but wonder if all this emphasis on test scores is going to totally destroy or keep us from developing very kind of "Learning Culture" that we should be fostering for 21st century learners. That culture should emphasize, collaboration, multidisciplinary learning, thoughtful risk taking, trial and error, creating, and intrinsic motivational learning. Test-Prep learning cultures are an anathema to each of these.

In my experience, schools and districts with "Test-Prep Learning Cultures" are characterized by some of the following:

• Student learning is reduced to what can be fit within the confines of A, B, C, and D on a bubble sheet. There is no time for independent exploration and learning. Students spend their days taking endless quizzes and tests in multiple choice format. Projects? Forget it! They take away valuable time better spent getting students to bubble-in right answers.
• Teaching is reduced to "only the essentials found on THE TEST." Nothing else matters. No room for student curiosity. Teachers spend inordinate amounts of time analyzing tests and test items and build learning around what they find.
• Teaching is about "covering the curriculum" not about whether students actually get it or find it relevant. Teachers end up repeating to students many times, "You need to learn this because it will probably "be on the test" not because it will help you be a better 21st century citizen or even help you get a job one day.
• Signs and posters on the walls remind everyone "Days to Test Day" as if on that day, the most important event of our students' lives is going happen. What a let-down, to have the most important event of the year be one, big "Bubblesheet Fest" at the end of the year. Also, one can only imagine the pressure these kinds of things add to kids on test day. These posters and signs are a clear indicator of what Test-Prep Learning Culture Schools value the most.
• Students and teachers participate in "Test-Prep Pep Rallies" or other similar events to fire them up to take THE TEST. In "Test-Prep Learning Culture Schools" principals and teachers will go to great lengths to motivate students to get engaged in THE TEST. These kinds of events also communicate the message to students who do not score well on THE TEST that they are somehow unworthy. 
• Students are judged in every way by their TEST SCORES. The are classified as smart and proficient based on their last End of Course Test or End of Grade Test. Students who are creative and talented in the areas of art and music and not test-takers are at worst de-valued. At the least, they have no way to engage these interests.
• Subjects are separated into silos, each with its own test. There's no time for multidisciplinary learning. There is only time to teach the content that is on THE TEST. The superficial boundaries between knowledge areas are reinforced in a Test-Prep Learning Culture.
• Getting the right answer is more important than anything else. There is no room for experimentation, and the only thing you learn from a wrong answer is that "you were wrong, period." Questions that do not have only one right answer are irrelevant or ignored. Failure is not a learning experience; it is to be avoided at all costs.

A veteran teacher once told me in the early days of my career as an educator, "Be careful what you wish for. When federal and state agenices do away with one policy, they almost always come up with something much worse." In my naivete, I obviously did not believe that. I still had an unwavering faith in the system, and that those who make the rules always mean well and often know more about those things than I do. Now, 20 some years after that conversation, I have to admit, my old friend had many things right. When our policymakers and politicians do reform, revise, revamp, or scuttle an education policy, the result is always something much worse. Our current example of this? The Obama administration's transformation of No Child Left Behind to Race to the Top and the whole series of waivers states can apply for to escape the sanctions of NCLB, which most everyone agrees is a bad law, but a paralyzed federal government can't agree on how to fix. Take North Carolina as an example.

In a meeting this past Monday I attended, representatives from the North Carolina Department of Public Instruction provided educators with a presentation describing how our state is adding a sixth standard to our teacher evaluation and an eighth standard to our principal evaluation directly tying those evaluations to test scores. What I discovered at that meeting was that the standards proposed are worded innocuously and can hardly be questioned. For example the teacher standard reads:

"Teachers contribute to the academic success of students. The work of the teacher results in acceptable, measurable progress for students based on established performance expectations using appropriate data to demonstrate growth."

I would think just about all teachers hope that what they're doing is contributing to the academic success of their students. The big difference in opinions among educators, however, is perhaps what this "academic success" is and whether growth measured by a test score is accurate. The principal standard is also written in this hard-argue-against language:

"ACADEMIC ACHIEVEMENT LEADERSHIP: School executives will contribute to the academic success of students. The work of the school executive will result in acceptable, measurable progress for students based on established performance expectations using appropriate data to demonstrate growth."

Both of these standards make sense on the surface. There's no educator alive who would argue that teachers and principals are not responsible for the achievement of their students. In my years as an educator, there's not a day that passes where concern about whether our students are learning what we're asking them to do isn't on my mind. The problem is not with these standards, but it is with the North Carolina Department of Public Instruction's interpretation of what "student achievement" is. Under their plan, student achievement = growth on standardized test scores. While that is a nice, neat simplification of what achievement is, it ignores all learning that can't be tested with standardized testing. By interpreting student achievement as test score growth, the state is simply making testing in North Carolina even higher-stakes than before. With this test-emphasis, we will be well on our way to becoming test-prep factories that don't churn out educated students, but excellent test-takers. North Carolina is going to subject students to tests early and often, not to measure student progress, but to measure how well the teacher is doing. This betrays an underlying, but mistaken belief that tests can be used to tell you how teachers and principals are doing. North Carolina has defined "effective teaching" and "effective school administration" as simply growth demonstrated (by whatever model they can create) by test score performance.

Ultimately, the meeting I attended was billed as an opportunity for educators like myself to provide "feedback" to North Carolina Department of Public Instruction personnel on this proposed teacher evaluation change. However, in practice, it seemed more of a here's-what-we're-going-to-do-to-you session, but we want to give the "appearance that we're listening to educators." (That's a tactic perfected by Arne Duncan.) During the session, when educators expressed concerns about what the state is planning to do, they were often cut off by DPI personnel who interrupted to defend the state's plans rather than sincerely allowing educators to voice their concerns and listening. They felt the need to stifle honest opinion by cutting off people while they were speaking. Instead of being an opportunity to express concerns, it was an opportunity for the state of North Carolina to try to summarily dismiss those concerns.

To this educator, there are two equally frightening things about this whole test-centered approach to teacher and principal evaluations that I and other educators at this meeting tried to bring up, but the North Carolina Department of Public Instruction representatives quickly let us know that we were wrong.

• First of all, North Carolina is planning to evaluate teachers and administrators  by using test scores this spring even though they have not decided what growth model they are going to use, nor do they know exactly how this evaluation is going to take place. What's worse, they will not know at least until February of this year when the State Board has an opportunity to hear the first reading. In typical bureaucratic fashion, they are rolling out a program and implementing it even before it is finalized and clearly defined. Historically, I can recall two other examples of North Carolina Department of Public Instruction initiatives that were rolled out before they were defined well. These were when the state during the 1990's rolled out the ABC Accountability Model, and in the early 2000s, when the state rolled out the NC WISE student data system used throughout the state. Both of these were rolled out without any deep thought on its practical application in the districts. They were both rolled out bugs and all, and classroom teachers and educators had to suffer while the state got its act together. Now, it appears our State Department of Public Instruction is doing it yet again, except this time there are much higher stakes tied to it. North Carolina will be evaluating educators' careers based on something that isn't even clearly defined, and won't be until three-fourths of the year is completed.
• Secondly, North Carolina is also planning to create tests for every single subject taught, administer these tests, and use the results, not to see how students are doing, but to see if teachers and principals are able to raise test scores. Recently, Charlotte-Mecklinburg proposed this "test-everything-that-moves" approach, and it blew up in the administration's face so bad, they had to take money from Bill Gates and the Broad foundation to hire public relations personnel to try to sell it. If something smells so bad that you have to repackage it to sell, then perhaps there's something fundamentally wrong with it. In spite of the fact that educators at this meeting expressed concern over how much more time we will spend testing under this proposal, and how problematic it will be to implement, the Department of Public Instruction personnel at this meeting dismissed these concerns outright repeatedly, and told us how wrong we were.