The 21st Century Principal

It seems new Secretary of Education, John King seeks to continue the same reliance on test scores and superficial ratings systems to determine how effective schools are doing that his predecessor Arne Duncan pushed. From his recent remarks, King wants to force states into using a "A-F Rating system" (or something similar) to rate the effectiveness of schools, which happens to be the same nonsensical idea that the North Carolina Legislature and North Carolina Governor Pat McCrory has imposed on public schools in North Carolina. (See "Education Secretary Takes Heat for Pushing Single Rating of Schools"). He wants states to come up with a "single summative rating for schools or districts" that captures the success or failure of schools in the most simplest way possible.

Of course we all acknowledge state standardized tests can't possibly measure everything that matters. Our state even forbids the use of state test scores as the sole basis of making high-stakes decisions. Yet, as the season of testing falls upon us, the obsession with bubble sheets and number two pencils begins anew. But we need to be reminded that tests are only a small piece of information that matters about our students and our teachers. Before we start scheduling the "Test Prep Pep Rallies" and giving our students those motivational speeches that elevate these tests higher than they should be, we need to be truthful with our students: in the grand scheme of public education, these test scores do not measure what's most important, and the results certainly do not come close to capturing what it means to be an excellent teacher, especially when excellent teachers impact lives more than test scores.

Recently, a Florida teacher wrote this letter to her students just before the onslaught of state tests. I think perhaps it reminds us that testing should always be put in proper perspective, not elevated to some major life goal or achievement. No one is going brag 10 years down the road that they made a "Level 5" on their English End of Course Test or their Biology End of Course Test. As this teacher points out, there are many more things worthwhile to brag about.


"My Dearest Students,

This week you will take your Florida State Assessments (FSA) for reading and math. I know how hard you worked, but there is something important that you must know. The FSA does not assess all of what makes each of you special and unique. The people who create these tests and score them do not know each of you the way I do, and certainly not the way your families do.

They do not know that some of you speak two languages, or that you love to sing or paint a picture. They don't know that your friends can count on you to be there for them, that your laughter can brighten the dreariest day, or that your face turns red when you feel shy. They have not heard you tell differences between a King Cobra and a rattler. They do not know that you participate in sports, wonder about the future, or that sometimes you take care of your little brother or sister after school. They do not know that despite dealing with bad circumstances, you still come to school with a smile. They do not know that you can tell a great story or that really love spending time (baking, hunting, mudding, fishing, shopping...) with special family members and friends. They do not know that you can be very trustworthy, kind or thoughtful, and that you try every day to be your very best.

The scores you will get from this test will tell you something, but they will not tell you everything. There are many ways of being smart. You are smart! You are enough! You are the light that brightens my day! So while you are preparing for this test and while you are in the midst of it all, remember that there is no way to "test" all the amazing and awesome things that make you, YOU!

Thank you for taking the time to read this. Please keep all kids in the state of Florida in your thoughts tomorrow. Thank you."

Here's my own three reminders to educators as we find ourselves on the eve of another "Season of Standardized Testing."

1. As we move into the "Season of Testing" let's remember that we don't teach test-takers; we teach real human beings with interests, hopes, dreams and passions that can't be reduced to multiple-choice questions.

2. As long as students and teachers give us their best, we acknowledge and celebrate that. Celebrate accomplishments such as poems written and published; hours of world-changing community service served; and songs written and sung. Celebrate what the bubble sheets ignore.

3. Keep testing in its place as one piece of data. Don't elevate it needlessly. Don't hold school-wide pep rallies that elevate these things superficially. Leave in their place.

I’ve been disconnected for a bit due to the demands of being a school administrator and as a student working on my on education, so that explains why I missed the passing of one of the most passionate educators I’ve met online, Joe Bower. For those of you who didn’t know Joe Bower, he is the author of the blog For the Love of Learning, and was an advocate for transforming schools into places where real learning happens instead of the focus on grades and testing that so currently preoccupies educators, politicians, and policymakers. He and I connected on a number of occasions through Twitter on issues of authentically assessing what students know rather than “bubble-sheeting” them to boredom.

Joe Bower and I never physically met, but through the power of social media and the Internet, we met and connected on several occasions, and it was clear that I met a true advocate for changing education in ways that make it better for children. His ideas concerning doing away with grading and testing and focusing on providing students with opportunities to show exactly what they could do fits so well with my own thinking about teaching and learning. His voice in this area will be missed. I encourage all educators to visit his blog Love of Learning and experience his ideas for making education better in the 21st century. Joe Bower the advocate for children and for authentic education will be missed. He truly loved learning and shared that with all of us.

Perhaps there's a truth here about accountability and testing that we have ignored: the use of standardized testing, value-added models, and growth models in teacher evaluations is all about subjecting the classroom to techniques of surveillance. To put it bluntly, they are spy tactics. Their purpose is to peer into the classroom to see if teachers are teaching the "prescribed" curriculum, and to see if teachers are adhering to the "rules of best practice" as they teach. These surveillance techniques are based on a fundamental mistrust of teachers' professional judgment regarding how they should be teaching and how their students perform.

The current efforts to perfect the evaluation of teachers aren't really just about improving teacher effectiveness: they are about sharpening the gaze into the classroom. They are about making the classroom more visible to those higher up the administrative chain. They are about finding the means to "objectively" determine whether teachers are teaching in the manner prescribed by "best practices" and whether they are teaching only the content that can be subjected to testing.

What testing and accountability experts have discovered though, is that tests are imperfect. The image they project of the teachers' performance is at best blurred and opaque. Even with new-fangled "value-added models," seeing teachers' effectiveness is foggy and unreliable. Now, they are seeking other ways to increase classroom surveillance. They are seeking other "spies" which might provide them with a clearer gaze of what's happening in the classroom. What are those new techniques for gazing into classrooms? They are called "student surveys."

Interestingly, student surveys turn students into 25 or 30 pairs of eyes that can report back to administration regarding whether classrooms are conducted in the manner dictated by the laws of "best practices." The student survey becomes another instrument with which the administration and government can sharpen its gaze into the classroom to make sure teachers' conduct adheres to "best practices." Underlying the use of student surveys is the assumption as well that if teachers know that 25 or 30 pairs of eyes are watching that might potentially report deviance back to the administration, those teachers will engage in the expected teaching behaviors. Thus, the control of the classroom becomes more complete.

We need to perhaps realize that the push to ever better classroom data is maybe more about control and transformation of the teaching and the teaching profession into a non-profession where teachers are simply "technicians of learning" whose professional judgment means nothing. The next generation of teachers will not need to exercise professional judgment: they will only need to conduct their classes in the prescribed manner by the "sciences of teaching." The question then becomes, "Are we really producing the kinds of students that our world needs?" Students newly manufactured and stamped with the US Department of Education's approval as being "Globally competent."




- Posted using BlogPress from my iPad

The American Education Research Association (AERA) has released a new statement about the use of value-added models (VAMs) in educator evaluations and to evaluate educator preparation programs. It is no secret that as an experienced educator of twenty-six years, I do not find VAMs very useful or fair indicators of teacher effectiveness. The results are often only available three or four months into the school year, so even if they were in a useful form that could inform specific classroom instruction, they arrive much too late, at least for first semester students to be of use.

But the AERA clearly points out that using VAMs as indicators of teacher effectiveness are still too deeply flawed to be used in that manner. (See the AERA statement for yourself here.) Many states, including North Carolina, have charged full speed ahead after being blackmailed by the Obama administration into adopting VAMs. This has occurred in spite of concerns over the limitations and flaws with their use.

There is certainly no disagreement from me that there is always room for improvement in teacher effectiveness, but I also think our false faith in the objectivity of value-added models sees these statistical models as some kind of “savior of public education” for which they are not, nor will ever be. Their limitations are too great to be useful for anything except as a small piece of data schools can consider about how their students are doing.

Here’s limitations outlined by the AERA statement:

• Current state tests are too limited to measure teacher effectiveness, and most were not designed for that purpose anyway. They cover only a limited amount of content teachers teach and they are too imprecise to be used in determining teacher quality. They also only measure grade-level standards so they fail to measure the growth of students above or below those standards.
  
• VAM estimates have not been shown to effectively isolate estimates of teacher effectiveness from other school fators or outside of school factors. To expect VAMs to do this entirely is unrealistic and foolhardy.

As usual, the adoption of VAMs illustrates one very bad flaw education leaders and education policy makers have: they adopt what they see as “common sense” measures without conducting critical and empirical explorations about whether the policies will work as they intend them. The history of public education is littered with these actions, and you would think a wise education leader would learn that what just seems “common sense” or “conventional wisdom” is perhaps nothing of the sort.

Today, the North Carolina Department of Public Instruction released the school report cards mandated by North Carolina’s legislature. (See those here.) These report cards graded each school with a letter grade A-F. Once again, our state has taken a step backward into absurdity with this action. Grading all the things our schools do with a single letter grade reduces, once again, what matters in North Carolina schools the most, to test scores. Once again, our state has elevated state testing to even higher stakes. Schools will now work in earnest prepping students for tests and getting those numbers up.

But elevating test scores is not only what this exercise in madness does; it also clearly demonstrates what’s wrong with education, and society, in North Carolina. In its article entitled “NC Public School Letter Grades Released, Reflecting Student Family Incomes,” The News and Observer sums up the real truth we learn from these report cards.

We don’t really learn which schools are failing and which are succeeding because the data used for this is narrowly focused on test score data and a few other indicators. What we learn of real importance is stated so aptly in this article:

“Among the schools where 80 percent or more of the students qualify for free and reduced lunch, 81 percent received a D or F. Only one of those schools got an A. At the other end of the spectrum, more than 90 percent of schools where fewer than 20 percent of the students qualify for free and reduced lunch received an A or B. Only one of those schools received an F."

In other words, our legislature and North Carolina Department of Public Instruction hasn’t come up with a test at all on how our schools or do; they’ve developed the perfect test for poverty. In fact, it really shows that North Carolina’s barrage of tests are great for indentifying students who live in poverty! Probably much better than actually measuring student achievement.

Sadly though, I suspect the motivation behind this A-F grading system isn’t really about improving public education at all. After all, our North Carolina State Legislature proved during its last session it is no friend to public education, why would we expect different.? No, this grading system is simply another attempt by our political leaders to drum up or even manufacture false charges of failure so that they can continue to push their pet project of school vouchers and their blind obedience to free markets.

“Every act of measurement loses more information than it gains, closing the box irretrievable and forever on other potentials.” Margaret Wheatley, Leadership and the New Science

The problem with accountability and testing lies within a single assumption: “that which is the most important content to be learned can be reduced to a single test or be captured in a test question.” If life were a dance between a, b, c, or d, then standardized tests could capture the essence of learning, and we could be satisfied that a correct or incorrect answer on multiple-choice questions actually tell us whether substantial and important learning has taken place. Sadly though, nothing worth while or lasting can be reduced to that level of simplicity.

As Wheatley points out, when observations, in this case tests, are created, choices are made as to what is to be tested and what is to be ignored. That ‘subjective choice’ reflects all manner of value judgments and decisions regarding importance. Hence, the very ‘subjective nature’ of tests like those being administered is questionable. The observation choices made by those who write the very questions on tests reflect their own subjective choices regarding importance. That’s why no standardized tests are ultimately entirely objective. As Wheatley points out, “Every observation is preceded by a choice about what to observe.” The person who makes those choices are exercising their subjective opinion regarding that is worthwhile to learning and what is most important.

To claim that state standardized tests or any standardized tests are “subjective” masks this fact: these tests reflect the subjective judgment of those whose wrote and designed them. It is simply their opinion regarding what is valuable enough to be tested. Next time someone throws the term “objective measures” or “objective testing” at you, remember this. The quest for ultimate objectivity in testing is a fool’s errand.

“The research-based truth is that the teacher effect (i.e., 10-20% of the variance in test scores) is not strong enough to supersede the powers of the student-level and out-of-school influences and effects (i.e., 80-90% of the variance in test scores) from one year to the next.” Audrey Amrein-Beardsley, Rethinking Value-Added Models in Education

The current push by the Obama administration, Arne Duncan, and many state departments of education to solely tie student achievement to teachers, demonstrates that they are ‘true believers’ when it comes to the power of the teacher. That sounds magnanimous. It has the smell of being a strong supporter of teaching. In practice, it continues to erode the quality of our schools. Why? 

In defense of the massive increase in testing and the use of those tests in high stakes decision-making, these individuals point to one assumption: “The teacher is the single most powerful influence on student achievement.” All we need do is focus on the teacher. Ignore that teachers don’t receive adequate funding for their classrooms. Ignore the fact that the school has not been able to buy textbooks in over six years. Ignore the fact that the heating systems, the plumbing systems, and ceilings are all falling into disrepair and crumbling. In a word, under Race to the Top and Duncan’s No Child Left Behind waivers, schools have been given the go ahead and excuse to spend all the funding they can on testing and ignore funding for resources for the classroom and for professional development. In addition, schools have also been given permission to ignore the effects of poverty and other outside-the-school influences on achievement as well. This agenda of relently laying the burden of all learning on the back of the teacher has allowed states to continue to pour increasing amounts of money into testing and accountability systems with the belief that because the teacher is the most important influence of student learning, nothing else matters.

You will see variations of that statement, but this belief about teachers is the heart of what I will call the “Powerful Teacher Doctrine,” a statement of faith so many policymakers, politicians, and even educators have so recently declared allegiance to. It is used as an excuse to ignore and wall-off a whole range of societal problems, such as child poverty and lack of healthcare, because they do not matter as much as the teacher. It is also used the dimiss the need to adequately fund schools, because tied to this belief is another myth, “Just throwing more money at our education problems won’t fix them.” No one is advocating “just throwing money” at problems. What we do advocate is providing money for the resources needed to give our students a quality education. The cheap, mythological fix that all we need do is shift the entire burden of student achievement to the shoulders of teachers is simply throwing less money and creating many new problems. After all, why would anyone want to be a teacher in a system that demands results without providing the means to bring about those results?

So what is wrong with “The Powerful Teacher Doctrine?” We, as Americans, love our heroes, and we have created quite a few, and are quite fond of creating them when none exist; just look at our movie industry. Now, in the interest of school privatization, in the interest of dismantling the teaching profession, and in the interest of accountability promotion, it has been declared that achievement is to be balanced entirely on the backs of teachers. In doing so, we have created conditions in our education system so that teachers may demonstrate their “heroic efforts” against all odds to increase student achievement. We want our teachers succeed against powerful odds, so we create a system, both educational and socially, that forces heroic action. These conditions are created by our politicians, policymakers, state education leaders, who continue to use the “Powerful Teacher Doctrine” to starve schools, classrooms, and teachers of funding, because all that matters is getting a good, heroic teacher in those schools in classrooms and student achievement will increase. Our education leaders and politicians keep education funding stagnant and then expect our schools and teachers to overcome the mess they’ve created.

It is no doubt that education funding remains stagnant. Just look at North Carolina as an example. I haven’t seen a new textbook in five years in our school, and that’s in any subject area. Our legislature this year provided a meager $25 million on the table for textbooks statewide. By the time that’s distributed to the schools, schools like mine only receive around $2,500. Now, how many textbooks can you purchase with that? Just look at the price of a popular science textbook. Take Glencoe’s current student biology text priced at $87.48. If you do the math, purchasing a class set of only 30 is $2624.40. A class set means that not every student gets a textbook either. This is a pure example of how our state politicians and education leaders are using the “Powerful Teacher Doctrine” to keep education funding low in the area of textbooks. After all, the extension of this belief is, "The omnipotent teacher is resourceful and can find ways to teach students without resources.” All we need are good teachers in the classrooms and the resource problem will take care of itself.

This “Powerful Teacher Doctrine” drives other areas of funding as well. North Carolina spends a miniscule amount on professional development too. Teachers are often forced to pay out of their own pockets to attend national conferences in their teaching specialities because there is no funding for such activities. Once again, this belief that good teachers will find ways and resources to improve their instruction is the underlying belief. Many good teachers do find ways to learn how to improve their craft. But when one looks to the medical field or business, these organization invest in resources for professional learning. Not so in public education. The expectation is that teachers come out of college, fully-developed professionally and willing to pay for their own training if it is needed.

My whole point here is not to deny the impact a teacher can have on students in their classrooms. They do impact student learning. The problem I have with the “Powerful Teacher Doctrine” is that our politicians, policymakers, and some educators use it as an excuse to not adequately fund and provide resources needed for education. School leaders often buy into this argument so much, that they forget to advocate for increased resources. They make a conscious decision that “advocating for increased resources for my school or district” is outside my sphere of influence, so they give up. It is this very thinking that has put our schools in the starved conditions in which they operate.

What happens when all that matters are test scores? Just ask former El Paso Superintendent Lorenzo Garcia who was sentenced by a federal judge to three and half years for a test-cheating scandal(See “Former EPISD Superintendent Lorenzo Garcia Gets 42 Months, Offers No Apologies for Scandal.”) Don’t get me wrong, I don’t shift the blame for one minute away from from Garcia. As a leader, compromising your morals to cook the books is never excusable, even if there’s pressure from elsewhere to do so. Yet, Garcia’s actions are understandable in a corporate American culture where “cooking the books” in order to deceive investors is acceptable. After all, is that not what the financial meltdown was about? Was it not about Wall Streeters who hid toxic loans from investors while making exorbitant salaries? “Cooking the books” has become an American management strategy, so should anyone be really surprised when school leaders like Garcia, or Atlanta Public Schools Superintendent Beverly Hall does just that to make their school systems appear to perform better than they really are?

In his book, Who’s Araid of the Big Bad Dragon: Why China Has the Best (and Worst)Education System in the World, Yong Zhao blames the accountability system based on high stakes testing. True. This obession with test scores in the United States is unhealthy. Our politicians and state education leaders have convinced many school leaders that obtaining higher test scores is the ultimate goal and product of school systems. No wonder there are cheaters who cook the books of school accountability to make it look like their districts are performing better than they really are. Accountability in the United States is evolving into the same game that Wall Street bankers play; cook the books to make it look like things are better than they are.

But at the end of the day who are these who cook the books really fooling? Are our students really learning more than they ever had? Are we actually producing the best graduates we have ever produced? Is our graduation rate really any higher than it has ever been? In some ways, I am afraid accountability in education has become a game that educators play. In a culture where numbers ultimately matter more than kids, education has adopted the exact same thinking that Wall Street adopted; whatever you can do to make your bottom line appear better than actually is becomes acceptable. Instead of focusing genuinely on the kids, accountability is game of data manipulation.

Garcia and Hall did a great disservice to education by cheating. Their actions are inexcusable. Yet, there are other educators still playing the game of accountability, shifting data points around, in order to make it “appear that their school or district” is on top.

Wall Street's “book-cooking” tactics have no place in education. What we do as educators determines the course of young lives, and when our focus shifts from that to numbers alone, we too are as guilty as those who wrecked the economy, and who “cook the books of accountability” for appearances sake.

"You can prep kids for a standardized test, get a bump in scores, yet not be providing a very good education." Mike Rose, "The Mismeasure of Teaching and Learning: How Contemporary School Reform Fails the Test"

It should not be a surprise at all to politicians, policymakers, and educators that the cry and backlash against testing and accountability is growing. During my 25 year career, I've seen the number of state tests administered in public high schools in North Carolina grow from 1 to well-over 2 dozen. Testing and test scores are the talk, and the focus is almost always on "how can we get those test scores up?" About the only ones, with the exception of a few teachers, who are enthusiastic about all this testing are school administrators, who for the first time have a "cattle prod" as Taubman calls it in his book Teaching by Numbers, to shock those teachers who get out of line and who aren't "producing." Blind acceptance of test scores as "the only data of importance" is common, because such data is seen as an "objective" measure, another myth perpetuated by testing and accountability supporters. But is that true?

Peter Taubman's book Teaching by Numbers: Deconstructing the Discourse of Standards and Accountability in Education points out some flaws with this blind acceptance of testing as "the data" on which to base all educational decisions. He writes,

"Fundamentally, tests provide little more than data, but just as one must question the confessions extracted under torture, one has to wonder just how reliable that data is, when it is wrung out of students shocked by the constant administration of tests." 

In other words, no one questions, least of all school administrators, this "data" we're looking at as measures of teacher, school effectiveness and student learning. Tests are data, but how good is that data when students have been subjected to test-after-test-after-test as we do in high schools in North Carolina? North Carolina education leaders truly believe in the maxim, "If it breathes; test it." Data collected under the oppressive, tortuous testing system in our state isn't foolproof, and our jobs as administrators, educators and teachers is to remember that when we start looking at numbers.

There's no doubt when our state education leaders, administrators use the phrase "accountability" they mean primarily multiple choice tests designed to keep teachers, administrators and whole schools in line. As Taubman writes again,

"All too evident, accountability translates into teachers' responsibility for their students' learning as measured by performance on tests." 

I would add that when our state leaders and most administrators use the phrase student achievement they are only speaking about test scores. The testing math in North Carolina is captured by this equation:

Student Achievement=Test Scores

Reducing learning to a test score is great if you are accountant, but for those of us who know teaching, we know that genuine learning is rarely, if ever, only contained between the letters of a multiple choice question. Real, worth-while learning is not always subject to being captured on a a standardized test.

Administrators love tests though, and with the same enthusiasm that politicians do. Why is this? I think Taubman once again hits the bullseye. He writes,

"One reason administrators are sympathetic to testing, the data it generates, and various practices connected to testing and data aggregation is that these provide control from a distance, a fundamental component of what is called audit culture." 

Testing allows principals to become accountants, district leaders to become accountant managers, and superintendents and state level leaders become CEOs. Through test scores, all levels of administrators finally have a tool to control what happens in classrooms. They can dictate how teachers can act, and even in some school systems, teachers are given scripts to follow to make sure they cover what is to be tested. Increase the number of tests administered and you control more and more of what happens in schools. All that talk about allowing teacher decision-making, but holding teachers accountable for those decisions is just empty rhetoric. Tests are measures of control and compliance, and they are gradually strangling public education. Testing finally gives administrators what they think is an "objective" tool for getting rid of teachers and for making sure everyone is compliant.

Test data also gives administrators at every level "bragging points." It gives them something to boast about to the public, to business, to industry, and to politicians. Never mind that testing almost always reduces teaching and learning to only what can tested. Taubman gets it right once again when he writes,

"Tests constitute one way the educational reforms show the educational system. Extracting data from students, teachers and schools, they force our noses into the bottom line. Keeping us under constant surveillance, they make us vulnerable to centers of control beyond our reach, and, providing the illusion of objective accountability and meritocracy, they reduce education to right answers and information." 

Testing is about keeping teachers and students noses to the bottom line. It is about using the "illusion of objective accountability" to make sure no one gets out of line.

There is no question that this accountability and testing culture is negatively affecting teaching as a profession as well. According to Taubman,

"High stakes tests erode the autonomy of teachers, for if tests determine the curriculum, and if tests tell us what is important to know as a teacher, and if these tests are fabricated by centers of control beyond the reach of teachers, then the teachers' passions, commitments, and wisdom count less and less." 

As mentioned earlier, accountability and testing is in some case reducing the act of teaching to little more than a "scripted lesson." Instructional delivery is simply following the state or district lesson plan. Teacher autonomy due to the massive testing load is at an all time low. Teaching is no longer a profession; it is a factory job, whose goal is to churn out test scores. If a teacher fails to "make production," they are branded "Ineffective" or "Not Making Expected Growth" as its called in North Carolina.

What is more amazing is that state educational leaders just don't get it. Enrollments in education programs in colleges and universities for training teachers is at an all time low, and it isn't just about salaries. Teaching is just not very attractive when your job is test-score production. Talk to any students about becoming a teacher, and they laugh in your face. Even worse is when you find yourself as an educator no longer encouraging young people to become teachers because being a public educator anywhere, much less North Carolina, has been robbed its ability to be satisfying career because too much emphasis is placed on accountability and testing.

Where does all this end? I wish we knew. North Carolina, as do other states, continues to ramp up its testing by adding new tests, and the state stubbornly hangs on to its massive testing regimen. Will it be when there's no one entering teacher education programs in our state? Will it be when there is no one with more than 10 years experience left teaching in the classroom? Or, will it be when parents, students, and teachers finally push back and say they've had enough? Testing and accountability is more oppressive than ever in North Carolina and elsewhere, and it is sinking public education and the teaching profession along with it.

One of the leading educator organizations in this country is calling for an end to the over-reliance on standardized testing in accountability practices.They are calling for what is called "multimetric accountability" by which, according to their press-release includes:

• not using standardized test scores as the "sole measure of student achievement, educator effectiveness, or school quality."
• having an accountability system that "promotes continuous support and improvement and that also is: "public and transparent, includes a range of subjects beyond English and mathematics, and that incorporates non-academic factors such as measures of school climate, safety, and parental engagement."

The debate over Common Core continues. This video captures a high school senior Ethan Young speaking eloquently about some of the concerns about establishing a national curriculum and our current education system's obsession with testing, accountability and standards. Perhaps our policymakers forget that standardization doesn't equal equity though they might think so. Maybe it is equally true that many who are so powerfully pushing these new standards are the very ones who stand to benefit the most financially from their implementation. Take a look at Ethan Young's take on this. It renews our faith that young people are passionate and do care about their education.

“We need to be careful that the tests we use in a properly designed teacher-appraisal system do, in fact, contribute to a valid (that is accurate) inference about a teacher’s quality.” W. James Popham, Evaluating America’s Teachers: Mission Possible?

North Carolina took the plunge this year and started using test scores as part of teacher and principal evaluations. The state has even invented a "new" kind of test, called a "Measure of Student Learning" in order to make sure there is plenty of test data to go around. What is particularly telling is how "carefully" the state crafted the term "Measures of Student Learning." It's as if somehow, not calling it a test, makes it not a test. State level educational logic never ceases to amaze me. Of course, the state then started calling these "Measures of Student Learning" something else. They started calling them "Common Exams." Notice again, the careful use of the word "exam" rather than "test." It's almost as if you don't call it a test, it isn't a test, but apparently state level policymakers haven't heard the old saw about a rose still being a rose even if it has another name.

Besides North Carolina's struggle with what to call their newly implemented tests, there's still the question of what the unintended consequences of having thousands of teachers "teaching to the test" is going to do for students in our state. Ultimately, being able brag that your students "Have the best scores in the world" is most likely what politicians and state level education officials are after. That's why they see salvation through test scores as the means to the "Educational Promised Land." Ultimately, there's a flawed logic driving this whole accountability and testing movement: it's the whole idea that learning can be entirely reduced to bubble sheet answer sheets and taken in a single sitting. And, that teachers can't be trusted to tell when a student has demonstrated that they have learned or not.

In my years as an educator, I have been amazed how trusting and accepting educators in North Carolina are when it comes to the latest policy flowing down from on high. It's as if they accept that those at the state level know more than they do, or somehow have access to magical information they do not have. So, when they implement something like the use of test scores in evaluations, many educators accept that the powers that be at the state level know what they are doing, so they trust them. Given the history of reform ideas and educational policy that travels down from on high, this "trust" is highly misplaced. I like to think that state level education officials mean well, but what often has happened during my career, these ideas when implemented locally have sometimes been a disaster and have been sometimes downright bad for kids. Instead of being so trusting, I submit that all educators in the schools and districts need to become skeptics and ask tough questions of our state-level, and federal level policymakers. We should never accept the "trust me, this will work" answer.

It is in this spirit of skepticism, I turn to Popham's book, Evaluating America's Teachers: Mission Possible? and our state's venture into making high stakes testing even more high stakes. In spite of what our state-level policymakers say,  I am not fully satisfied that North Carolina's tests are adequate measures of educator effectiveness, and  a healthy skepticism is still in order. This whole push to add test scores to teacher and principal evaluations has been a rush from the start. Depending on when you asked questions, how the tests were to be implemented has changed multiple times throughout the last two years. Never mind the fact that not a single teacher in North Carolina even saw the test before they were implemented. In their rush to have "test data" it's as if our state level policymakers think "any old data will do." They have failed to take the time to establish whether any of these tests really tell us anything about teaching quality.

In light of our state's push into "higher stakes testing", I think Popham reminds us of some important key issues and ideas about tests and teacher evaluations that state politicians and policymakers seem to forget.

• “Tests are not valid or invalid. Instead, it is a test-based inference whose validity is at issue.” In other words, it isn't the test that’s valid or invalid, it is the inferences drawn from those tests that have these qualities. It boils down to whether you can actually make an inference based on the test or not. The question is whether North Carolina's tests, which have been implemented haphazardly and a thrown-together-manner, actually tell us anything at all about the quality of teaching in our classrooms. Can I honestly say Teacher A is a good teacher because she added "this much" value to her students' Measures of Student Learning? Seems to me that it puts a great deal of faith in a single test.
• “Tests allow us to make inferences about a test taker. This inference, depending on the appropriateness of the test as a support for the inference being made, may be valid or invalid.” As Popham points out, the inference we make about the learner may be valid or invalid depending on the “appropriateness of the test” in its role to support the inference being made. As we know, the word validity is the extent to which that inference, or conclusion, is well-founded or corresponds to the real world. This boils down to whether the inference we draw about a student is valid or not. For example, should we infer, based on a student’s test scores that he is not proficient in the subject, we must be satisfied that the test we are using is the “appropriate measure,” and we must also make sure the conclusion we draw considers all real world facts. Ignoring a student’s socio-economic status, or even whether he experienced  a death in the family, can make our inference about the student’s proficiency invalid. Then there's the whole issue about making an inference about a teacher or principal's effectiveness using this same test. Has North Carolina sufficiently established the appropriateness of their Measures of Student Learning, End of Grade Tests, End of Course Tests, as instruments that allow for making inferences about teacher and principal quality? I'm not sure they have. Another question, do these Measures of Student learning allow us to make valid inferences about teacher quality? I'm not convinced they do.

As North Carolina moves forward with a teacher and principal appraisal instrument that uses test scores to determine effectiveness, all educators need to educate themselves and scrupulously ask questions of policymakers at all levels.

As Popham suggests, “If heavy importance is being given to students’ performances on state tests for which there is no evidence supporting such an evaluative usage, then teachers (I would add principals too) might wish to engage in further study of this issue so that, armed with pertinent arguments, they can attempt to persuade educational decision makers that more appropriate evidence should be sought.” In other words, all educators, administrators, and teachers need to study how North Carolina or any state is using test scores to determine educator effectiveness.

Administrators owe it to their teachers, and themselves, to understand that some of these tests were never designed to determine educator effectiveness, so that data needs to be viewed with skepticism. And, I would add that the manner in which these Measures of Student Learning were developed and are administered may not allow them to draw valid inferences about teacher quality. Test scores in North Carolina currently are only 1/6th of the teacher evaluation, and effective administrators are going to keep this in mind and not let the allure of numbers numb them to the other 5 standards.

“Without these common exams, we have no objective way to measure the value teachers give their students, and this is an important part of North Carolina’s teacher evaluation model.” Rebecca Garland, Chief Academic Officer, NC Department of Public Instruction

In a recent post to the Charlotte Observer entitled “Newly Required Tests Aren’t as Numerous as You Think”), North Carolina Public Schools Section Chief, Rebecca Garland, defended that state’s massive increase in the number of high-stakes testing. The main gist of her argument is students were going to be taking teacher-made exams anyway, which would be true if these Common Exams or MSLs (missiles as we like to call them) were on the same level as teacher made exams, but they are not.

North Carolina is elevating the importance of these tests to “high-stakes level” because teacher performance will be judged based on them. In the end, North Carolina has massively increased the number of “high stakes tests” it is administering under its “Common Exam Project.” So Garland's remarks are misleading at best.

Garland defends these tests as well by using the common “buzzwords” used by politicians and other policymakers. Judging from her statements in the above post, in one fail swoop she declares these tests:

• Objective:
• Fair
• Accurate
• Reasonable

Are these Common Exams “objective?” Too often, policymakers and politicians view “objective” as meaning “multiple-choice” or otherwise limited to answers that can’t be verified or constructed. I submit that these common exams are not objective, nor could they ever be. Because these tests were developed by teachers far-removed from the classrooms in which they will be administered, they are not objective. In developing these tests, teachers and state-level test developers make SUBJECTIVE judgments about what would be included on these tests and what would be discarded. This process, by default, is a subjective value judgment on what exactly is worthy of being tested and what is not. The tests might be “objective” in the sense that there may be no room for variability in student answers and do not require the judgment of teachers in determining right and wrong, but the way the tests were developed is a subjective process.

Are these Common Exams fair? Throughout North Carolina’s development of these Common Exams, state level policymakers have used this word repeatedly, as if by declaring these exams "fair" makes them so. There are so many question marks regarding the “fairness” of these tests and how the state has chosen to use them that whether they are “fair” has never been established. For example, take the whole idea of using “value-added” measures. North Carolina has chosen to use this way of evaluating teachers in spite of the fact that statisticians and test designers have cautioned against using such models. In addition, Garland and the North Carolina Department of Public Instruction make the argument that they are only using it for a fraction of the evaluation. But if using value-added measures have flaws at all, should they be used in decisions that determine the individual livelihood and futures of educators? Another question of fairness has to do with the tests themselves. Normal final exams are teacher-designed for a reason. That teacher knows the content that was covered and the students to which that content was taught. That means that teacher is able to design tests that measure what they taught and to tailor those tests to the needs of their students. Using the common exams, teachers are forced to give tests developed by others far removed from their classrooms, making them hardly true measures of what was taught.

Are these Common Exams accurate measures of how good a job teachers are doing? We've been trying to judge school performance based on test scores for years. After No Child Left Behind flopped, it was a natural progression for North Carolina to move to the next flop---using test scores to judge educator effectiveness. North Carolina has not done any studies correlating the use of test scores to determine whether teaching improves. But then again, they define "effective teaching" as having "high test scores." Never mind whether those tests have any validity or quality.

In the end, in spite of what Garland and NC DPI assert, North Carolina’s “Common Exams” is a massive increase in the number of “high stakes” exams our students will be subjected to and to simply say they are like the final exams students usually take is misleading. These tests which have been hurriedly assembled, kept in secret, and developed to appease the United States Department of Education are not the salvation of our North Carolina education system. There is a real problem when our own state department of education turns into a propaganda machine, incessantly trying to sell a testing program that no one wants, rather than assisting and serving the educators and students in this state. In the end, our students are the ones that suffer.

“Developing better tests of student learning in the 21st century is as futile as attempting to find a faster horse and buggy would have been in the 20th century.” Douglas Reeves, “A Framework for Assessing 21st Century Skills”

After last night’s #ncadmin chat about “21st Century Teaching and Learning” I could not stop thinking about questions regarding 21st century skills and assessment.  One question continues to haunt me: "How can we devise an assessment for 21st century skills?" That question made me recall an essay I once read by Douglas Reeves entitled, “A Framework for Assessing 21st Century Skills.”

According to Reeves, It is “not possible to reconcile the demands of 21st century skills with the realities of the traditional testing environment.” The very conditions demanded by our current testing regimen is antithetical to the 21st century skills we want our students to have. As Reeves points out, our assessment practices lag behind “because they are bound by three destructive conditions: standardized conditions, secrecy of content, and individual results.”  These three testing conditions are destructive to our efforts to teach and assess 21st century skills because they were designed in a different era of education. To assess our students' 21st century skills we need an entirely different set of tools. I realize the new testing consortia are exploring "new generation assessments" but I fear they will not let go of those 20th century obsessions with student score comparisons and standardization that make current testing inadequate for assessing 21st century skills.

According to Reeves, there are three qualities of assessment that form a framework for any 21st century assessments we need.

• We need assessment conditions that are variable rather than standardized. The whole idea behind the 20th century idea of standardization is having the ability to “compare students.” Policymakers and politicians, in their demands for accountability, consider the only way to have that accountability is by being able to compare students’ scores. So, in order to make those comparisons, students are herded into the same kind of room environment, given the same kind of pencils, the same scrap paper, the same bubble sheet, and the same amount of time to complete the test. Standardizing the test conditions take precedence over everything, even the needs of the kids. As Reeves points out, in these kinds of conditions “students are rewarded for memorization and following established rules,” not for being creative and being innovative. These standardized conditions worked well in 20th century assessments designed to sort and classify students, but 21st century assessments need to allow for the variability inherent in the messiness of creativity, critical thinking, and problem-solving. Students need to be able to demonstrate that they can solve problems, not in the “manufactured, controlled” environment of a standardized testing room. They need to be able to demonstrate they can solve problems not limited by the conditions standardized testing impose. Being able to draw a diagram, collaborate by speaking to experts, watching videos, read books, and access web databases are not possible in standardized testing conditions, but in the real world, those are the tools people use to solve problems. If we are going to assess students’ 21st century skills, we are going to have to give up the obsessions for comparison of student scores and standardized testing conditions to provide assessment conditions akin to the real-world environments that people use to solve problems.
• We need assessment of students as teams rather than as individuals. Since collaboration is a cornerstone of 21st century skills, we need to stop testing students in isolated silos, and assess their skills the way real world people solve problems, through collaboration. If our students are asked to engage in creativity, problem-solving, and entrepreneurial thinking in the manner that real-world people do, our assessments may need to move from the individual to the collaborative. Instead of passing out bubble sheets to each student, cutting them off from the real world, and demanding they choose the "correct" answers to problems, 21st century assessments need to have students work in teams to analyze and devise solutions to multi-layered problems that do not fit in the confines of answers A,B,C, or D.  Once again, doing this means giving up the obsessions with comparing student scores and standardization.
• We need assessments whose content is public rather than secret. I can’t speak for other states, but my own state of North Carolina protects test content almost obsessive-compulsively as Milton Waddams protects his stapler in the movie Office Space (See Photo Below). Teachers and students are kept totally in the dark about what is going to be on test, leaving them to scrounge around and make all kinds of wild speculations about that content. Teachers are forced to play a game of Concentration as they try to guess what the state is going to ask next. North Carolina testing experts claim their test is derived from the state’s curriculum, but that curriculum is so broad, it makes this kind of testing a game, trying to decide which part of the curriculum will be on the test. As Reeves points out, 21st century assessments require that the kinds of learning we want students to do, be public. Students must be able to study those assessment ideas, and they may even devise their own assessments, or demonstrations of learning. But once again, being open about test content means giving up this obsession with comparing student scores and standardization.

As Douglas Reeves argues, our insistence on comparing students test scores and standardization are serious obstacles to developing 21st century skills assessments. The old standardization model demands too many conditions that are antithetical to 21st century learning. School leaders need to have the courage to ask the tough questions of those who advocate for the “testing status quo.” As Reeves points out, “Educational leaders cannot talk about the need for collaboration, problem-solving, critical thinking, and creativity and at the same time leave teachers and school administrators fenced in by obsolete assessment mechanisms, policies, and assumptions.”

Too many politicians, policymakers, school leaders, and educators are protecting the “testing status quo” and refuse to relinquish those very conditions that keep us from fully transforming our schools. We have fenced ourselves into having 20th century schools by our own 20th century assessments and obsession for standardization and comparison of student test scores. Letting go of the “testing-status-quo” is an enormous step toward a 21st century education system.

"Who would want to teach in a system that measures your worth as an educator by how much your students can regurgitate on a two-hour multiple-choice test and that has reduced much of the curriculum to tedious test-prep exercises?" writes Tony Wagner in his latest book entitled Creating Innovators: The Making of Young People Who Will Change the World.

Who, indeed, would want to teach in the kind of education system Wagner describes in this question? Perhaps a better question would be, "Who would want to learn in such a system?" Yet, with all the increased emphasis across the country on test scores as part of both teacher and principal evaluations, the kind of education system Wagner predicts is already coming to fruition, and my state is on the fast track to such a system.

North Carolina schools have had a "Learning Culture" characterized as "getting students ready for THE TEST" since it began rolling out state tests in the 1990s. Now, North Carolina has moved from using those same tests to determine student proficiency to determine teacher and principal proficiency. And, for subjects that do not currently have a standardized test, they are creating a TEST, to not measure student learning, but to measure educator proficiency. The end result  of these measures will obviously be that the state that once proclaimed proudly "First in Flight" on its license plates, can soon declare "First in Test-Prep." 

Sadly, though, one can but wonder if all this emphasis on test scores is going to totally destroy or keep us from developing very kind of "Learning Culture" that we should be fostering for 21st century learners. That culture should emphasize, collaboration, multidisciplinary learning, thoughtful risk taking, trial and error, creating, and intrinsic motivational learning. Test-Prep learning cultures are an anathema to each of these.

In my experience, schools and districts with "Test-Prep Learning Cultures" are characterized by some of the following:

• Student learning is reduced to what can be fit within the confines of A, B, C, and D on a bubble sheet. There is no time for independent exploration and learning. Students spend their days taking endless quizzes and tests in multiple choice format. Projects? Forget it! They take away valuable time better spent getting students to bubble-in right answers.
• Teaching is reduced to "only the essentials found on THE TEST." Nothing else matters. No room for student curiosity. Teachers spend inordinate amounts of time analyzing tests and test items and build learning around what they find.
• Teaching is about "covering the curriculum" not about whether students actually get it or find it relevant. Teachers end up repeating to students many times, "You need to learn this because it will probably "be on the test" not because it will help you be a better 21st century citizen or even help you get a job one day.
• Signs and posters on the walls remind everyone "Days to Test Day" as if on that day, the most important event of our students' lives is going happen. What a let-down, to have the most important event of the year be one, big "Bubblesheet Fest" at the end of the year. Also, one can only imagine the pressure these kinds of things add to kids on test day. These posters and signs are a clear indicator of what Test-Prep Learning Culture Schools value the most.
• Students and teachers participate in "Test-Prep Pep Rallies" or other similar events to fire them up to take THE TEST. In "Test-Prep Learning Culture Schools" principals and teachers will go to great lengths to motivate students to get engaged in THE TEST. These kinds of events also communicate the message to students who do not score well on THE TEST that they are somehow unworthy. 
• Students are judged in every way by their TEST SCORES. The are classified as smart and proficient based on their last End of Course Test or End of Grade Test. Students who are creative and talented in the areas of art and music and not test-takers are at worst de-valued. At the least, they have no way to engage these interests.
• Subjects are separated into silos, each with its own test. There's no time for multidisciplinary learning. There is only time to teach the content that is on THE TEST. The superficial boundaries between knowledge areas are reinforced in a Test-Prep Learning Culture.
• Getting the right answer is more important than anything else. There is no room for experimentation, and the only thing you learn from a wrong answer is that "you were wrong, period." Questions that do not have only one right answer are irrelevant or ignored. Failure is not a learning experience; it is to be avoided at all costs.