The Anatomy of Education Deform

It starts with a study like this one as reported by the New York Times. A bunch of Ivy League economists get together to study the impact of teachers on students.  “Better” teachers were those whose students had improving standardized exam scores. They then track 2.5 million students over the course of several years. Their findings show students who had at least one “good” teacher between the 4th and 8th grades go on to make $4, 600 more than those who only had one “average” teacher. Furthermore, “students with top teachers are less likely to become pregnant as teenagers, more likely to enroll in college, and more likely to earn more money as adults.” The implication is that we need to evaluate teachers by their students’ exam scores and fire the ones that are “bad”. According to one of the economists who conducted the study, John Friedman, “the message is to fire people (teachers) sooner rather than later.”

This is the embryonic stage of an education deformation. By reading its DNA, we discover everything education deform is and what it is capable of doing.

Scientists deal in the world of inanimate facts. They can use laboratories to create controlled environments, allowing them to eliminate variables to discover true cause and effect relationships. While real scientists in all fields have their hot button debates, those debates take place within a wider tradition of consensus on fundamental ideas. Economists and Educationists, on the other hand, are social scientists who generally have no such consensus. They are divided into ideological camps.Economists can be Keynesians or Neoliberals, Salt Water or Fresh Water. Education “experts” are whole language or phonics, constructivists or traditionalists. Their research has no lab and instead has to take place in the wild, so to speak. That means they cannot control for variables, which means they have no way of knowing if that tax cut was really the thing that boosted the economy or that teacher was the thing that boosted test scores. They observe human interactions as literally millions of factors act upon those interactions and then choose one of those factors as the sole “cause”. Despite the pretense of using data, their conclusions are generally shaped by the ideology they wish to support. After all, in the grand scheme of things, all ideologies in the social sciences have been “proven” at some point with data.

Let us assume for now that their contention is true, that you can assess a “good” teacher by their students’ test scores and that bad teachers adversely impact the futures of their students. Why, then, is “the message …to fire people sooner rather than later”? There is nothing in their research that proves firing bad teachers sooner rather than later is a benefit. First, with whom do you replace those bad teachers? First year teachers would be unknown quantities since they cannot be judged by student exam scores. Would it be beneficial to use them over bad teachers? Why fire anyone at all? The message of their research could just as easily be to mentor or support bad teachers so they can become good teachers. Or maybe the message is we need to do another study on what makes the “good” teachers so good and teach that to all the bad ones. There are literally hundreds of conclusions that can be drawn from this research. Out of all of those conclusions, it is curious that Friedman would choose to spout this one. Logically speaking, it does not necessarily follow from his research.

And the research does not necessarily follow any logic of its own. According the manuscript (which you can read here), they looked at 2.5 million students in the same state. They looked at their enrollment histories, previous test scores and household wealth. If their exam scores went down at the end of a school year with a given teacher, they can tell that the teacher had a negative effect size. In the words of the study, “the jump in the teacher’s impact at the end of the grade taught by that teacher suggests that the observed impact on test scores is most likely due to a causal effect of the teacher rather than systematic differences in student characteristics, as such characteristics would have to be uncorrelated with past test scores and only affect the current test score.”


I teach U.S. History which has a Regents Exam by the end of the school year. The previous year the students take Global History with other teachers, which also culminates in a Regents Exam. Due to scoring rubrics and content, the Global History exam is way tougher than the U.S. History exam. Students who take U.S. History with me generally get higher grades on that exam than they did on the Global exam. Does this mean I am a better teacher than all the teachers of Global? According to this study, the answer is yes. They assume that every test is an equally objective barometer of student achievement. There is no way for the study to control for the varying difficulties of each exam, whether it is the difficulty of the rubric or vocabulary or content. The Global exam also requires kids to remember concepts over a two-year period (Global History is a two-year course), while U.S. History is only one year. Not to mention, students take Global History in their 9th and 10th grades. Everyone knows that 9th and 10th graders are way different than 11th graders. 9th and 10th graders are less mature, less focused and generally in greater danger of dropping out or falling behind. 11th graders are over the hump of their high school years, many of them focusing on getting into college or starting life in the real world. They are young adults, more mature and, yes, generally smarter than kids in previous grades. The economists who did the study, however, believe humans do not change over time. They have fixed “characteristics” as they say, so any change in exam scores must be the result of the teacher. From the top to bottom, the study is wrought with issues like these that fail to take in every single factor that contributes to any given exam score. This makes the study less scientific and more like guesswork.

For these reasons, the conclusion the report makes about the long-term impacts of teachers on students’ live is even more problematic. They say students with good teachers in the younger years go on to have lower rates of teenage pregnancy and higher rates of college admissions and adult earnings. This type of thinking holds teachers responsible for not only the education, but the lifestyle choices of their students, choices made well after the student is out of that teacher’s classroom. Despite the fact that there is no way to account for all the potential causal factors in teenage pregnancy, college admissions and earnings, the economists conclude that teachers have a sizeable impact on these things. There is no accounting for cultural reasons why teenagers might have kids or go to college. The economists assume that just because the students in the study come from the same socioeconomic background, they have controlled for the cultural factor in life decisions. What about a poor kid from a family that sets college as an expectation from a young age? How about a poor kid born to a teenage mother or who lives in a neighborhood where teenage motherhood is common? How can a teacher have an “effect size” over personal decisions made years later? What is the role of the teachers the student has at the time kids make these decisions, teachers who were not part of the study?

Despite all of the problems with studies like this, it will surely become a weapon for the education deformers. They will cite the findings of these three Ivy League economists who conducted the largest and longest study of effect size to date. The reports conclude that we should fire “bad” teachers. The defomers will use this conclusion to ram all types of teacher evaluations into the system, evaluations that are designed to fire not bad teachers, but older teachers that make too much money. The economists and the deformers speak the same language. Instead of talking about kids in the context of their cultures, communities, families and schools, they want to tie kids to teachers and teachers only. It is the problem with all of the social sciences. They take what is essentially an art, whether it is life choices, business or teaching and try to contort it into a scientific study. They create studies that are later used as justification for major policy decisions. Unlike scientists in the physical sciences, the ultimate goal of many social scientists is to have their research politicized by people in power. Each supposedly objective study is really a contestant in the game show of “pick a policy”. The true worth of a study is measured not in the scientific rigor of its findings, but whether or not it shapes policy later on.

Naomi Klein’s The Shock Doctrine describes the role economists have played in privatization schemes around the world. They have been the vanguard of a Neoliberal movement that seeks to privatize every part of the state. Privatization is the favorite policy of Neoliberals everywhere. Public schools are the latest battleground of the Neoliberal push to privatize. The three economists are taking their study on the road, presenting it to journals in hopes it will become a weapon in the policy debate over public schools. Policy makers will point to this “scientific” study as a justification to get rid of tenure and job security in teaching. The general public will unquestioningly embrace it, as they do every fad study that is reported on in the news. There will be more calls for charter schools that do not have things like teacher tenure. In the words of Nobel Prize-winning economist Joe Stiglitz, economics tends to have its fair share of “free market fundamentalists”, people who believe that there should be no public institutions whatsoever.

The three young economists who conducted this study may or not be free market fundamentalists, but they certainly have given the free market fundamentalists in the public school debate a powerful weapon with which to bludgeon public schooling. They are certainly testing fundamentalists and their research assumes the infallible objectivity of standardized exams. In this regard, they are identical to the education deformers.

These economists from this study are education deform ideologues.

Just like deformers, they assume the infallibility of standardized exams. Their research does not even speak to the differences between exams or consider that the exams themselves can be fallible.

Just like deformers, they use that assumption to help conclude that poverty is not destiny. Instead, they all conclude that the teacher is destiny.

Just like education deformers, they say “bad” teachers need to get fired soon, despite the fact that their own research does not necessarily come to this conclusion.

This is not science, it is dogma. What is worse, they use one dogma to prove other dogmas. We will see this study sold as scientific research. What it is really is just another school reform mantra by people with no connection to public school at all. It is just another arrogant set of education deformers who believe their thick assumptions about schools should apply to the education of everyone else’s children.

24 responses to “The Anatomy of Education Deform

  1. Maybe teachers should do a study on the performance of economists and report their findings.
    It’s only fair since they feel they are qualified to ‘study’ and ‘report’ on the performance of teachers.

    • Agreed. We can start with all the economists who said tax breaks for millionaires, bank bailouts and privatization schemes would boost the economy. The message is we need to fire economists sooner rather than later.

  2. Let’s see if Cuomo’s committee will be anything more than deformer puppets.

  3. Pingback: Who is Valentino Crawford Jr.? | eBook Services

  4. I appreciate your comments about the differences between 9th, 10th, and 11th graders developmentally. In my district, we teach Algebra I to 8th graders, and Geometry to 9th graders. Typically, they do very well on the Algebra exam, and horrible on the Geometry exam. I assert it is because they are not developmentally ready for Geometry in 9th grade! Perhaps we have Global and US Histroy mixed up as well?

  5. I really wanted to read this article but the white on black is really hard to read. Your work seems great and I would it to be formatted in a “friendly” manner for reading online. Thanks!

    • Thanks for the suggestion. I have been toying with the idea of changing it, despite the fact that several people advised me that white on black is easier on the eyes. I will post your suggestion on my facebook and twitter feeds and see what people say, if anything. For now, you can read the article in RSS by clicking RSS-posts in the RSS Feed box at the right.

      • I enjoyed your article a great deal but I thought I would add another vote for changing the colour scheme. I wrestled slightly with a strobing effect while I read the last half of the article.

        To the article itself, it’s always too easy for someone to carry out some observations, for which they have quantitative data, and rather than admit that they can’t derive any sound qualitative data – they make it up or draw false conclusions (like that ‘firing teachers’ nonsense). I’m currently working on a higher ed data analysis project of student hand-in patterns and, from my 500,000 pieces of data, I can show you some cool patterns and interesting artefacts, but I can’t yet tell you ‘why’. Answering this question will take a great deal of work, active qualitative exploration, and a long time (2-3 years) before I can even make a stab at why.

        I could, as these economists have done, make some statements that could get me attention and funding, providing I didn’t mind that I had no evidence to back it up or that I was resorting to dogma.

        (To the colour theme point, my own blog is designed to be easy on the eyes – light green background, cool green major image and dark grey-green text. It’s probably not for everyone but it is quite relaxing to read.)

      • Thank you. I will take what you say and your blog’s layout into account.

        And you are right. Data does not really answer the why which, in education, is the most important part.

  6. Pingback: “Round-Up” Of Recent Good Posts & Articles On School Reform Issues | Larry Ferlazzo’s Websites of the Day…

  7. Excellent work exposing the fallacious reasoning of those who believe that “controlling for various factors” leads to a clear determination of causation. Give me an Excel spreadsheet, and data, and I can justify nearly anything. However, as you so clearly state, this is neither science, nor without bias. Friedman should be ashamed of himself, along with all others who attempt to “prove” their dogma with questionable analysis and statistics, as if numbers themselves are sacrosanct.

  8. Great point. It is possible that all the conclusions in that study are due to selection bias. But then, Economics is to the sciences as plane crashes are to the aviation community: A lesson in what to do different.

    • The worst part is that their research has not even been peer reviewed yet. New York Times ran their piece on the front page before anybody else in the economics community had a chance to go over it. You would think that, as the newspaper of record, their standards would be a little higher. Unfortunately, the Times has supported ed deform in the past too.

  9. I would like to see these economists’ former teachers write about how they were as students.

  10. Pingback: The Best Posts On The NY Times-Featured Teacher Effectiveness Study | Larry Ferlazzo’s Websites of the Day…

  11. I like this article of your much better than the other one I read earlier. Excellent points on why the so-called “bad” teachers don’t get the chance to improve and should just get fired. For those of us working in the private sectors, we have annual appraisals where our development goals are determined and the actions towards them are identified. Do these “bad” teachers know they are “bad”? Looks like you have read the report/study. Do you think with 2.5 million data points several of the cultural influences, exams variability and such will be less noise? I don’t think in any scientific study, the data is going to give you straight line and there will be noise. However, the first step will be to look at what the general trend is showing and take an action based on that. and then the improvement cycle continues. From what I have read in your article, it feels like the study is not saying teachers are the sole contributing factors to teenage pregnancy and such, but that they do have a role in helping avoid or promote certain post-course life choices. I don’t need a study to tell me that as kids look for role models from all aspects of life (parents, neighbors, teachers, TV shows, movies…). however, in the world of sensational journalism of breaking news every second and catchy headlines, such reports can be misrepresented. Anyway, good analysis!

    • Some bad teachers know they’re bad and some need an administrator to guide them. Firing them does not help the system when there is no plan to replace them with “better” teachers. Moreover, determining a bad and good teacher is fraught with danger. Good teachers ALWAYS get labeled as bad and vice-versa because of the completely erroneous correlation between teachers and exam scores. The report is based upon a faulty premise that exam scores are accurate representations of teacher quality. This is a premise still debated in the education community, with the preponderance of evidence suggesting that exam scores and teachers have little to no relation to each other.

      Just because there seems to be a correlation between teacher quality and teenage pregnancy in their numbers does not mean there is one. They assume causality without proving it.

      Once you try to measure teaching and learning in numbers, you have lost. There is no way to numerically quantify the art of teaching or the act of learning in statistics. The millions of variables that are impossible to control for make it a fool’s pursuit.

  12. Pingback: walks like a duck « Learning: Theory, Policy, Practice

  13. Pingback: Compassion?: Education Reform’s Separate and Unequal Agenda | assailedteacher

  14. Pingback: How Bad Teachers Nearly Ruined My Life | assailedteacher

  15. Thanks for sharing your thoughts. I really appreciate your efforts
    and I am waiting for your further write ups thank you once again.

Assail me some more by leaving a comment:

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s