Evaluating Teacher Effectiveness
http://thedianerehmshow.org/shows/2012-02-09/evaluating-teacher-effectiveness
In this photo taken Sept. 1, 2011, Jennifer Mojica works with students in her math class at Holmes Elementary School in Miami. In a distressed neighborhood north of Miami's gleaming downtown, a group of enthusiastic but inexperienced instructors from Teach for America is trying to make progress where more veteran teachers have had difficulty: raising students' reading and math scores.
(AP Photo/J Pat Carter)
Most everyone agrees that having a great teacher matters. A recent study by economists at Harvard and Columbia Universities answers the question of just how much. It found having a good teacher may be worth thousands of dollars in extra income over a student’s lifetime. Determining how to measure teacher performance has become a national debate. Some say evaluating teachers primarily based on test scores is unfair and not in the best interest of students. Others say it’s a great incentive and rewards the best teachers. Diane and her guests discuss how best to determine teacher effectiveness.
Guests
Thomas Toch
Senior Fellow, Director of Washington Office, Carnegie Foundation for the Advancement of Teaching
Jonah Rockoff
Professor of Business, Columbia Business School. Co-author of study titled, "The Long-Term Impacts of Teachers: Teacher Value-Added and Student Outcomes in Adulthood."
Jason Kamras
Chief of Human Capital, Washington, D.C. Public Schools
Stephanie Black
Former public school teacher
Program Highlights
Results of a recent study found that having just one high-quality teacher in elementary or middle school can improve a student's quality of life for years to come. But how can we measure teacher effectiveness? Are standardized tests the only answer? A panel of education experts discusses different approaches to educational assessments and improving the U.S. public education system.
Evaluating Teachers
"It's not clear though that all of the attributes that we seek in teachers are captured in a basic skills test score, for example, the ability to teach kids to think critically, to reason analytically," Toch said. "The other thing these test scores don't capture is the ability of teachers to teach the intangibles like tenacity and resilience, what researchers today are calling 'learned optimism' which is particularly important for disadvantaged students who don't come to school from backgrounds where education is assumed to be important, where it is in fact a struggle to be successful in school."
Improving The Tests
Some suggest that if we're not currently testing the things we really care about, we should be getting using - or making - better tests. Rockoff said that "value-added" evaluations attempt to measure how well teachers are performing in the classroom in a new way. Using vale-added parameters, students could conceivably still fail a standardized test, but if the class as a whole show tremendous improvement from the previous to current year, the teacher could be identified as a high value-added teacher. "It's an important development because in our efforts to evaluate teachers, which has traditionally been done very superficially in America public education, this allows us to look at what matters most, student achievement," Toch said.
Teacher-Led Cheating
USA Today recently did an investigation which found high erasure rates in some schools on standardized tests, indicating possible instances where teachers manipulated students' scores. "It's unfortunate, because it's cheating kids to a large degree," Toch said. In D.C. public schools, Kamras said they take the very rare indications of teacher-led cheating very seriously, but that he believes in the inherent morality of the staff. "The overwhelming majority are working hard every single day playing by the rules to do great things for kids. And I think that's what we need to remember when we talk about these kinds of things," he said.
Challenges From A Teacher's Perspective
A caller, Stephanie Black, spoke to Diane about her experiences teaching in public schools from 2007 to 2011. Ms. Black left teaching (and is now a math tutor) because she didn't feel that the extreme focus on test scores was preventing her from becoming a better teacher. Ms. Black said she feels attention to teacher training is very important. "I think we need to move away from this idea that the only way to decide if a teacher is great is to use a standardized test," Ms. Black said.
You can read the [full transcript here]
(http://thedianerehmshow.org/shows/2012-02-09/evaluating-teacher-effectiveness/transcript).
Comments
Please familiarize yourself with our Code of Conduct and Terms of Use before posting your comments.
I am sure that pay for performance will come up in this conversation. Here are a few issues.
Overall politicians love to say. How bad teaching to the test is. So, what are they going to use to measure performance? How are they going to make sure people do not cheat? It is a whole different ball game when you put money in the mix.
How would all of those special education teachers get evaluated?
There many more factors that weigh into how a child is doing in school it will never just be the teachers.
In a resent study they found that adolescents spent 20% of their time doing things classified as entertainment and only 3% of there time on home work.
Best idea give parents a Tax cut if their child shows progress year over year. The more progress they make the bigger the tax cut. Maybe we could double that 3% number.
Here is a crazy idea why not just pay Teachers a livable wages and support your local schools and Teachers and let politicians go mess something else up.
Why not let public schools kick kids out that are not performing. Maybe they would take school more serious. Just like private schools. That makes it fair, right? I really do not think this way. It was just to get people thinking. Public schools are a gift to this country and have made it great.
Thank you for covering this topic.
Diane,
Thanks for what I'm sure will be a refreshing perspective on a truly pressing topic.
I'm a Teach for America alumnus (2 years teaching 1st grade in the Bronx) and am still teaching in an elementary classroom, but not my placement school. I have accumulated LOTS of questions around the idea of effectiveness, and more importantly, teacher retention. Here’s a sampling.
1. I was not a great teacher my first year, at all, and I'm still flabbergasted at the messages TFA pumped into us, that I was making a difference. I wasn't really. But my data looked good. Kids learned math and they could read better. The only way that year is “worth it” in the grand scheme of children’s lives is the fact that I’m still teaching, and now am good at it. How can “effective” be measured by easy tests or self-selected data? And do individuals in the education reform world have a habit of naming instruction effective when it is really just less bad than instruction down the hall?
2. Teach for America has a clearly marketed organizational goal to get alumni into the political realm. This agenda is heavily pushed during corps members’ 2nd year, and after. Individuals who stay in the classroom get countless recruitment emails for positions within TFA, opportunities outside of TFA, to go into administration, law school, etc. If my contribution as a (now) effective teacher is so vital to education reform why am I being recruited out of it?
3. In my current position teachers are on the receiving end of hundreds of thousands of dollars worth of professional development from people my school flies in from New York. My practice has grown tremendously as I’ve been exposed to what true best practice should look like. But the school is not consistent with instructors or with behavior expectations. The bar for accountability changes based on context. How far does effective teaching go in an inconsistent school?
Hope I'm forgiven for going over the character limit and posting again.
Just throwing two more questions out into the ether of ed reform discussion...
4. How are enough effective teachers going to have the energy to stay in the classroom in an incredibly demanding position? I love teaching the kids I teach. It’s vastly rewarding, but HARD. I work 60 hour weeks for inadequate pay and some of the worse health insurance in the country (for the education field). There are many kids in my class who have shockingly high needs, both emotionally and educationally. At my current school it’s impossible to be good at your job and a really good parent to your kids, while teaching rich kids is easier and pays better. What’s going to keep me, and others like me, in the classrooms we're in?
5. I don’t mean the above to sound snarky, just a real question I ask myself daily. My answer for now is how incredibly rewarding it is to see these kids grow. Who gets to cry at work when a child shares her stirring writing to her absent father and can take a bit of credit for bringing that writing into the world? And does that count as effective?
Looking forward to this. I hope that the guests will talk about the use of in-class observations of teachers at work. We need to move beyond "drive by" observations that don't provide much value in helping teachers to improve. Instead, policymakers should be using observation tools with research behind them -- tools that provide detailed information on how teachers are interacting with their students.
At the New America Foundation, we've been examining the use of these tools for the non-tested grades of pre-K, K, 1st and 2nd grade, as well as 3rd. For more information, see this op-ed in the LA Times:
http://earlyed.newamerica.net/publications/articles/2012/pushing_past_me...
and here's more information from an event we hosted recently, titled Watching Teachers Work:
http://earlyed.newamerica.net/events/2012/watching_teachers_work
There's lots more work to do -- and many tough issues to untangle. But I hope that observation-based assessments will get some attention. Thanks.
My step mother is a teacher and we have had numerous conversations about this very issue. Her contention is that the teacher has little control over the child once it leaves the classroom and therefore cannot be held accountable for the ultimate success of the child. This is the same argument that industry had concerning metrics when they were introduced, and after properly figuring in the known rate of expected failure the metrics are adjusted. No one is suggesting that every child will be at or above average, unless they have no idea what an average is, in which case their math teachers need to be looked at. Proper rating based on reasonable expectations born out of reasearch and experimentation will give usable statistical data that can be used to create reasonable and effective metrics. It has worked in every other field on the planet, it will work here. Oh and just so you teachers out there know, everyone thinks their industry is different and that what works for others wont work in their case, so don't bother with that excuse.
In reaction to the posts on Teach For America:
As a teacher educator for a public university, there have always been two things that disturb me about Teach for America:
1) There seems to be an underlying discourse of TFA that it is somehow "OK" to put non-licesenced teachers in high poverty / low performing schools. Why is it OK to place less qualified teachers in 'poor schools' but not placed in middle class or wealthy schools? My sense is that middle class parents would mutiny over having an "unqualified" placed teacher in their children's classrooms. Why should low-income parents be asked to settle for "unqualified" teachers?
2) Second, I frequently have undergraduates who are close to completing their licensure program who WANT to Teach for America. They want to take the knowledge and skills they have acquired; they want to work with low income populations, and they believe in the mission of TFA - but are turned away. ! Shouldn't organizations like TFA work with state licensure programs to place motivated, qualified new teachers in high need districts?
Heather Davis, Associate Professor
NC State University
One thing I've found that frequently gets overlooked when the "good" or "bad" teacher debate happens is the lack of recognition that the chemistry between student and teacher is constantly changing. A bad teacher for one student will frequently be the great, pivotal teacher for another student. Students and teachers are not widgets that can be externally examined by an inspector to see if they meet some set of physical attributes. A teacher that inspires one student will not reach every student.
Whether or not to use tests is not about fairness - it is about whether it is a reliable and valid measure of teacher effectiveness. Is there causation? Please check out this research:
http://roundtheinkwell.files.wordpress.com/2011/10/a-validity-argument-a...
I wholeheartedly agree about the value of having great teachers, and I've been fortunate enough to have had several. But bringing into it the question of evaluation by standardized testing is comparing apples to oil-rigs. The tests simply don't even try to evaluate the qualities that make the kind of teacher who positively impacts his or her students.
Please talk about the role of administration and the atmosphere in the school in the success of students. A discordant atmosphere or a less than supportive supervisor can make a difference in teachers' attitudes and abilities. Near great teachers can be great with an administration that loves students and people makes a tremendous difference. An administration more interested in climbing the ladder can destroy a school.
I taught for 42 years and just recently retired. My students still keep in touch so I know I wasn't too bad.
As for fairness, here is an example of a problem we haven't talked about.
http://literacygurl.blogspot.com/2011/03/teens-with-axe-to-grind.html
What if the kids don't take the tests seriously?
We were just talking about this at home; the only teachers I remember are the strict ones (well, I do remember some of the weak, but with a bitter taste). My stepson just graduated from high school last year (in NJ). He is fairly smart, intelligent, absolutely unchallenged. He was given the answers to the tests by some of the teachers. The main objectives are scores, not KNOWLEDGE. He has to take 3(!) courses before he can start with college level Math. What he should have gotten for "free" (it is still our taxes, right? not exactly free) he has to pay out of his pocket. But the main thing is; he has not learned anything; he has no idea about history, geography, politics...what a waste of perfectly good young mind and time.
He is now at a community college in Boston, and I already see the same patterns - the teacher is going very slowly, waiting for everyone to get their books, lowering the expectations...
David, thank you for the link! Sad.
The focus on growth in this study is very important but the focus on the kinds of tests we use both for kids and for teachers are all too unsophisticated to get us where the study may encourage us to go. We know from Neuro and Cognitive research that learning and teaching is social, emotional, dynamic and sophisticated. Trying to realize a real definition of institutionally effective teaching and learning according to metrics rooted in the industrial revolution is a doomed model.
HannahBee,
As someone whose main job is to train teachers within the public school system, I am always curious about TFA. May I ask, what data did TFA have that showed you were making a difference your first year teaching when you yourself feel that you weren't? I am swayed by their performance data and wonder what's wrong with it.
Thanks for sharing your story.
I am very concerned with the impact on my children with the high stakes component of testing. This study looked at the test given in the mid 1990's to see if the teacher quality had a longterm effect on student achievement. Would this even apply now given the current testing climate and the current tests?
Our students are getting a very different education now- with NCLB. The teachers teach to the test as they are evaluated based on test scores. We are not resolving the problems that are more impactful to acheivement- namely low SES.
Also, how can this study apply today when our students are having less and less individually developed curriculum by their teachers? Teachers have a lot less autonomy than in the 1990's due to the high stakes testing.
Having a great teacher would be great if that teacher taught about our cultural evolution that began with the American Revolution, and the problems facing the task of forming a more improve union.
Teachers are not the problem with public education. It's what they teach. If they used such books like "The Lies My Teacher Told Me" by James W. Loewen, or "A People's History" by Zinn, they would quench the thirst of students who need to know the facts about our culture, and hopefully inspire them to read and study the Constitution within the context the Declaration of Independence and the Articles of Confederation provide.
When people study Shakespeare more than the Constitution we end up with a cultural impact that inspires us to seek out comic relief form the economically oppressive nature of Britannica, or the repressive nature of a democracy without majority rule. Those kids who can't afford tickets to Shakespeare plays end up dramatising them in real life.
From what years are your data set drawn? I ask this, because testing has changed considerably in this era of accountability. The tests themselves are of questionable quality in some states. The trend toward test prep (drill and practice on test-taking skills) is far more prevalent than it was a decade or two ago. In addition, there is far less teacher control over what is taught, when it is taught and the manner in which it is taught today (especially in states and districts who rank near the bottom nationally). I think it is chancy to draw conclusions about the effectiveness of today's accountability measures by examining the results from a decade or two ago. Colleen Megowan PhD Arizona State University (teacher educator, former high school physics teacher for 20+ years)
A national or state sponsored program to evaluate teaching effectiveness should employ those teachers who have retired as well known respected educators in thier field. Any negative evaluation should trip a reevaluation by one or two other competent former teachers in the Evaluation group.
Thank you for your comments. I taught in a district in Texas for 27 years that is relying on Teach for America for many positions. The administration acknowledged that it helped their bottom line and helped fill difficult positions in a large urban minority majority district. Too often those of us who stayed had to pick up the pieces when the TFA teachers left.
Standardized Testing does not promote for child's growth. It simply puts 25-35 kids in a room that they can't leave and forces them to take tests to evaluate their progress on how to take tests.
Instead of using these Standardized Testing like the AZVAP, why not simply have a board of advisers create the final exams and the midterms. That way teachers have a focused goal of what they need to teach the students.
Having a 2 hour timed test that goes over everything from Math, Reading, Writing, Social Studies, and Science that's all jumbled up doesn't allow for a student to focus on what they are working on. Especially since they have to now study for both the State Standardized Test and their Midterms, Finals, SAT, PSAT and dozens of other tests like this.
I myself failed most of these tests only to go on to pass my GED test fairly well in a short amount of time and then I went on to College where I had the highest grade in my class. Now I am employed as the Graphic Design Coordinator at the Grand Canyon and have fantastic job security and am getting ready for my first child. When she goes to school I plan on having her come to my office after school so she can do her homework and have her dad there to teach her things that her teacher didn't.
Good Morning, In regards to evaluating Teachers, test scores only tell us if they are academically proficient, but does not tell how the teacher's ethic values are. It takes just one unethical teacher to harm a student's interest in academics. It is to my understanding teachers are not prepared to handle all posible scenarios that could present itself with the variety of students they handle in one classroom. Also, there are teachers who have been known to lose their temper, and verbally diminish a child under her/his care during school hours. School Board should see that teachers have not only good test scores, and clean background, but also strong ethical values. Children are sponges and the way they see their Teachers handling a situation for them it will look as okay, since it is the Teacher doing it.
The "no child left behind" Act helps to achieve academical levels towards a test, but does not prepare a student for higher levels of education. In my area a high percentage High School students moving onto College have proven to not be fully prepared to fulfill assignments nor the proper atitude that at a College level is requierd to have.
Sad but true, we have accepted for far too long how things are being handled and at the end of the day it is our young memebers of our community the ones paying the consequences.
Thank you for reading my blount opinion.
I am sad to hear the basis of these discussions is to promote evaluation of learning merely through standardized testing. As being a part of the education system it is clear to me that these individuals have no idea what really goes on within the classroom setting. The unfortunate outcome of all this is that little learning takes place in the classroom as teachers are forced to “teach to the test.” Yes students may be able to “answer” the questions on these tests, but they have not achieved the highest level of Bloom’s Taxonomy of critical thinking skills, and thus they are not independent thinkers. It may make schools and teachers look good on paper, but did we really achieve success? Sadly no!
Why has no one asked teachers what resources and supports they need in order to teach effectively? Teachers are the ones in the classroom who have experience working with kids. I use the word kids because children are not adults. Another reason why standardized testing is so often invalid is that children guess, get tired and just fill in random circles, put their answers on the wrong part of the answer sheet, and misunderstand terms they are unfamiliar with. Rather than asking how we evaluate teachers, the question should be why do we mistrust teachers to the point that we don't even allow them to be involved in setting standards and making policy decisions about their own profession? Why does everyone else think they know more about teaching than teachers do?
I do not condone in any way teachers who facilitate cheating on the student evaluations. That said, have the panel considered that this activity at least partly stems from the fact that so many teachers disagree with the premise that how students do on a test directly corresponds to the teachers' effectiveness? Doubtless you have many teachers and administrators who feel - possibly correctly - that in many situations the tests simply do not tell much about the quality of teaching. If educators felt ownership of the testing and the consequences, then there would be far less incentive to cheat, and far more likelihood that other educators would turn in the cheaters.
Brian Wells
Baltimore
It is imperative that we have a universal agreed upon professional standards developed by teachers, managers, families and students. We need them now. Student focus groups in urban districts tell us a "good teacher" is one that uses high standards, "... is strict, never gives up on you, listens to you and expects you to listen and learn."
Kevin
School psychologist
In Columbia Missouri there are two high schools. Both have diverse student populations.
The last statistics I saw regarding graduation rates were a graduation rate of 45% for blacks 75 % for whites, 85% for Asians. Same schools, same text books, same classrooms, same teachers.
How can a teacher change a culture?
OR - there is a 30 year history of research on test anxiety (particularly with regard to high stakes testing). What happens when the student take the tests too seriously and it affects their performance. When we view aggregate scores, we know little about the internal (motivation, anxiety) factors that affected the score - nor the external factors (such as teacher preparation / effectiveness) that affect their performance.
HD
Does this study start making the case (long overdue in our society) in public conversation that teachers are important or is it something else? I'm curious about the fact that researchers from Business Schools are having such a big voice in the conversation about education. The Top- Down mandates system in a bottom-up delivery system within a profession that is so reliant on face-to-face social interaction seems at least as important for reform as anything else in education. No?
Re Elroy: "... Same schools, same text books, same classrooms, same teachers. How can a teacher change a culture?"
I think the emphasis on 'sameness' is part of the 'culture' dilemma. Too often we think that giving the same pedagogy, same resources is fair. Yet, any special educator will tell you that students have different needs and that equity is based on meeting students needs (sometimes that means that to get students to the same level of high performance we have to provide more time, more resources and employ different pedagogies).
HD