Evaluating Teacher Effectiveness
http://thedianerehmshow.org/shows/2012-02-09/evaluating-teacher-effectiveness
In this photo taken Sept. 1, 2011, Jennifer Mojica works with students in her math class at Holmes Elementary School in Miami. In a distressed neighborhood north of Miami's gleaming downtown, a group of enthusiastic but inexperienced instructors from Teach for America is trying to make progress where more veteran teachers have had difficulty: raising students' reading and math scores.
(AP Photo/J Pat Carter)
Most everyone agrees that having a great teacher matters. A recent study by economists at Harvard and Columbia Universities answers the question of just how much. It found having a good teacher may be worth thousands of dollars in extra income over a student’s lifetime. Determining how to measure teacher performance has become a national debate. Some say evaluating teachers primarily based on test scores is unfair and not in the best interest of students. Others say it’s a great incentive and rewards the best teachers. Diane and her guests discuss how best to determine teacher effectiveness.
Guests
Thomas Toch
Senior Fellow, Director of Washington Office, Carnegie Foundation for the Advancement of Teaching
Jonah Rockoff
Professor of Business, Columbia Business School. Co-author of study titled, "The Long-Term Impacts of Teachers: Teacher Value-Added and Student Outcomes in Adulthood."
Jason Kamras
Chief of Human Capital, Washington, D.C. Public Schools
Stephanie Black
Former public school teacher
Program Highlights
Results of a recent study found that having just one high-quality teacher in elementary or middle school can improve a student's quality of life for years to come. But how can we measure teacher effectiveness? Are standardized tests the only answer? A panel of education experts discusses different approaches to educational assessments and improving the U.S. public education system.
Evaluating Teachers
"It's not clear though that all of the attributes that we seek in teachers are captured in a basic skills test score, for example, the ability to teach kids to think critically, to reason analytically," Toch said. "The other thing these test scores don't capture is the ability of teachers to teach the intangibles like tenacity and resilience, what researchers today are calling 'learned optimism' which is particularly important for disadvantaged students who don't come to school from backgrounds where education is assumed to be important, where it is in fact a struggle to be successful in school."
Improving The Tests
Some suggest that if we're not currently testing the things we really care about, we should be getting using - or making - better tests. Rockoff said that "value-added" evaluations attempt to measure how well teachers are performing in the classroom in a new way. Using vale-added parameters, students could conceivably still fail a standardized test, but if the class as a whole show tremendous improvement from the previous to current year, the teacher could be identified as a high value-added teacher. "It's an important development because in our efforts to evaluate teachers, which has traditionally been done very superficially in America public education, this allows us to look at what matters most, student achievement," Toch said.
Teacher-Led Cheating
USA Today recently did an investigation which found high erasure rates in some schools on standardized tests, indicating possible instances where teachers manipulated students' scores. "It's unfortunate, because it's cheating kids to a large degree," Toch said. In D.C. public schools, Kamras said they take the very rare indications of teacher-led cheating very seriously, but that he believes in the inherent morality of the staff. "The overwhelming majority are working hard every single day playing by the rules to do great things for kids. And I think that's what we need to remember when we talk about these kinds of things," he said.
Challenges From A Teacher's Perspective
A caller, Stephanie Black, spoke to Diane about her experiences teaching in public schools from 2007 to 2011. Ms. Black left teaching (and is now a math tutor) because she didn't feel that the extreme focus on test scores was preventing her from becoming a better teacher. Ms. Black said she feels attention to teacher training is very important. "I think we need to move away from this idea that the only way to decide if a teacher is great is to use a standardized test," Ms. Black said.
You can read the [full transcript here]
(http://thedianerehmshow.org/shows/2012-02-09/evaluating-teacher-effectiveness/transcript).
Comments
Please familiarize yourself with our Code of Conduct and Terms of Use before posting your comments.
Anyone who advocates classroom 'evaluations' of teachers, and 'rubrics' about 'value added' by use of standardized tests as keys to improve our education is purely in the realm of fantasy. Tony Wagner's book The Global Achievement Gap covers much of what is wrong with US education today. A lot of it is overly expansive curriculum, especially in math, he advocates Bill Gates supported projects using extensive videotaping of teachers, and the use of these in groups of teachers to spread good technique from experienced teachers to less effective ones as a routine program in schools.
Any student will tell you when an evaluator sits in the classroom, most teachers do not behave anything like they do normally. Just last week, a student in Oklahoma was suspended for taking a cell phone photo of a teacher sleeping at their desk when they were supposed to be teaching. My kids have had teachers who routinely openly belittle the whole class as being dumb or lazy, in a well funded school district that is supposed to be the best in the state.
I have never seen a school that has a set policy on feedback on homework or testing, some teachers do not return homework for weeks, often testing the subject before returning the homework. When I confronted a principal on this nothing was done. When no one in a school district cares, or holds a teacher to the simple objective of returning homework in a timely fashion, all the testing and a few days of evaluators filling out paperwork will be useless in improving education.
I think the greatest thing that can be taught to children is critical thinking- a process of analysis that enables students to evaluated the world around them in a rigorous way. How would this be measured in looking at teacher competency?
I was appalled to hear the educator from Washington call employees Human Capital.
The old term, Human Resources, is bad enough, but this is worse! What has been done to other resources in this country? Exploit, abuse and pollute. To use people in the same way is bad. What is done with capital in this country is worse, and using people like that will lead to worse things than low test scores.
The problem with education and systems that think they have to rely on financial measures is that the goal of those systems is to increase levels of wealth. That is not necessarily compatible with a much more beneficial goal of increasing levels of civilization.
First, there aren't enough private school slots in America to accept every child who might want to attend; but, if every private school in the US closed today, guess where all those students would end up?
Private schools can choose their students, public school can't!
For decades now, we have had one side of our political class telling us education should be a local issue- that Washington shouldn't tell us how to educate our kids. The result of that are 51 different educational standards, and with all due respect, the worst standards are in places that consistently listen to those 'leaders'.
They tell us how bad teachers are, how bad educated people are, and how people need to take a stand against them.
Well, first, why shouldn't we have national standards for education? Why shouldn't the core curriculum be the same nationwide? Allowing for regional dialects, shouldn't the basics [reading, writing, arithmetic, social studies/history- the Civil War is not the war of northern aggression, the south started it] be the same for everyone? Our worldwide industrial competitors have national standards and it shows in their ability to compete.
Second, all those politicos 'dissing' education- every one of them went to college; so, education is bad for you but not for them? Let's face it, with their agenda, they need their constituents desperate (no jobs, struggling to make ends meet) and dumb (poor education and educational opportunities) to keep getting the votes they get.
People who swallow that crap had better get a clue, and an original thought in your heads, and stop listening to the talking points of folks who have shown that they've chosen sides- and it's not yours!
Where is the responsibility of the college teaching programs? Many teachers are not ready to take a class at the time of graduation.
Teacher and administration-led cheating does happen and more reliance on testing increases the pressure to do. I've had to 'clean up' with students whose previous teachers MUST have done something, as the test results were quite different from student performance.
Dallas ISD's answer has been to move teachers around on test day, but that would work better if the teachers from the next grade shifted down- someone who has a vested interest in realistic scores.
I also like the different ideas for evaluation, and not just the principal. Having principals observe other schools could help, as long as it's not the teacher's former principal (often the reason for changing schools). Evaluation by specialists makes a great deal of sense.
Thank you for the insightful talk.
I have missed most of today's broadcast but during the first half hour one of your guest mentioned testing the child to compare to their own progress. He said end of year to end of year comparison. I don't believe that is valid, but BOY to EOY is - after allowance for a two to three week refresher at the beginning of the year. The fall off in achievement between the end of one year and the beginning of the next can be quite significant and some students lose much of what they learned during the summer hiatus - depending on what their home circumstances may be and the ability of parents to enrich their experiences during the summer months. Obviously, some students will progress further over the summer but that is probably not the norm.
As an ex teacher (five years in the early 70's), I see two things that have diminished education overall through the years:
1 - No Child Left Behind and the push to fit every peg into the same hole, no matter what their skills and abilities - the lowest achieving students need the most diversity of methods to find what works for them, so teaching to the test is counter productive.
2 - Homogenious grouping. This m ay benefit the most able of the students but it a crime against the general success of schools. For low qchieving students, being grouped at all times with only other children who are less successful means that they never have a 'model' for success. In addition, having better abled students in the mix allows them to help/tutor their peers and learn empathy. Homogenious grouping in all subjects/classes (and magnet schools) run the inherent risk of developing an elitist attitude among the higher achieving students.
In response to the "loving kids" and "loving teaching":
I think we are hearing the tension between what we call dispositions for teacher education and preparation to be an effective teacher. The National Council for Teacher Education has struggled with what it means to identify teachers with dispositions toward teaching "all students," dispositions to be accountable for having your students meet high standards etc...
And that you can love kids, love teaching and still be unprepared for the demands of teaching. Teaching requires an incredible amount of pedagogical, content, and technological knowledge as well as interpersonal skills to manage 20+ relationships or more (per class). Most individuals who want to go into teaching need instruction and support to acquire these skills. And, initial licensure programs should only be considered the first attempt to develop effectiveness.
I loved the comment about how the standards for effectiveness SHOULD look different for a teacher in their first three years of teaching, 3-10 yrs, and beyond 10 yrs...
Wouldn't we want to see growth over time?
HD
One caller made the point that students jailed on test day scored a "0", bringing his/class/school's evaluation score way down. The converse is that significantly bringing up those very bottom scorers can significantly raise a school or teacher's total evaluation score.
I know of a principal, who in attempt to bring up that "bottom", brainstormed with teachers, and came up with a voluntary "catch up" class for selected students. Staffed by a highly qualified, enthusiastic teacher, with a low student/teacher ratio, these students made astonishing progress in basic skills. Some even caught up to grade level, and re-entered regular classrooms the next year. Total scores jumped up the next year. It worked!
But, alas, this successful innovation was short-lived. The district administrator allowed other schools to "dump" their lowest performers on this school, rather than encouraging them to start their own "catch up classes". The innovative principal soon left that district for a more rewarding professional opportunity elsewhere.
Perhaps in addition to beefing up classroom teacher accountability, we also need higher professional standards, and far better public accountability for district superintendents and supervisory staff. We also need at least minimal professional standards for school board members. In my state, school board members don't even have to be able to read and write.
I want to share a great example of good teaching. My sister, Kay Wehner, who will be 89 in July, was a very gifted 5th and 6th grade teacher in the Oakland, CA schools for many years (during the difficult days when they were bussing students to desegrate schools).
As a result a group of her students took her out to lunch last year to thank her for her inspiration to all of them..They wrote to many classmates who could not be present, but who wrote her beautiful thank you letters for the difference she made in their lives. She is a poet, as well as a teacher, and she inspired them to do their best in every subject, stressing creative writing, taking them on many field trips, etc. I am so proud of her!
Jean Lutterman
From 1997-2006 I worked as a principal and director in Montgomery County, Maryland, which has a superb process for evaluating teachers. I had the privilege of being the co-chair of the Peer Assistance and Review Panel for the first six years of its existence. This panel of eight principals and eight teachers oversaw the work of teacher evaluation. The evaluation process included a set of teacher standards based upon the rigorous national teacher standards. The process provided consulting teachers for all first year teachers and for all teachers evaluated as underperforming by their principals. The purpose of the process was to provide support first. If the support was not successful, then the panel would make recommendations to the superintendent regarding employment. Test scores were considered as but one of multiple data sources for a teacher's evaluation. The process was fair, rigorous, and effective. During my forty-five years in education, I have not seen a better process. I only wish that this process would expand to many districts throughout the country because it serves children well.
I am a 35 year veteran of teaching. I retired, not because I especially wanted to, but because of personal needs at the time. I considered myself an excellent teacher. Because of that (others also told me I was an excellent teacher), I feel I know what is wrong with our schools. I tried calling to get on the show, but the line was always busy. Perhaps, that is just as well, as I had a great deal to say. First, and, in my opinion, foremost, I would like to see you do a show that features TEACHERS! How about a show that features only teachers---ask them the same questions or similar questions that you asked your guests today. I rarely hear programs or see them that anyone asks them.
Secondly, the problems of public education are soooo complicated, so interconnected that to address one problem is not really going to "fix" the whole problem.
I had more to say, but could not include it in 2000 characters. I delineated 9 areas that show problems with the "system."
Pat House
I am also a Teach For America alum, but while I taught high school English for three years in North Philadelphia, I also learned with veteran public school teachers at the University of Pennsylvania. These teachers taught me certain things about "good teaching" that are not valued by the current discourse on education: sticking it out (retention), empathy, critical thought, service-learning, critical inquiry. I think the effort to standardize teacher evaluations and hold teachers across America accountable to the same numeric goals on tested subjects reveals a gross misunderstanding of the history of education in America.
American public education has from the outset been about being able to teach what was locally important with some basic agreements on what should be taught but also with the demand to teach for what is valued to the community. Today, an English teacher, an “effective” teacher, who teaches a story about gay teenagers in Virginia can still be harassed and threatened with impunity. In the national rhetoric on teacher evaluation, the value-added human growth in empathy, kindness, tolerance, and/or citizenship have been lost. Education reformers suffer from the political allergy to looking at complex issues as interconnected. 50% of low-income students who go to college do not finish with a degree; one of the greatest indicators of finishing college is parental income. Education and poverty cannot be addressed as if they are not related. Poverty haunts my former students in the form of– dead mothers, fathers, brothers, asthma, racism, classism, and negativity. The focus of NCLB is Adequate Yearly Progress. But what is adequate twenty-year progress? What kinds of citizens and society do we want? If 99% of citizens were “proficient” in reading and math, would that be enough?
I am also a Teach For America alum, but while I taught high school English for three years in North Philadelphia, I also learned with veteran public school teachers at the University of Pennsylvania. These teachers taught me certain things about "good teaching" that are not valued by the current discourse on education: sticking it out (retention), empathy, critical thought, service-learning, critical inquiry. I think the effort to standardize teacher evaluations and hold teachers across America accountable to the same numeric goals on tested subjects reveals a gross misunderstanding of the history of education in America.
American public education has from the outset been about being able to teach what was locally important with some basic agreements on what should be taught but also with the demand to teach for what is valued to the community. Today, an English teacher, an “effective” teacher, who teaches a story about gay teenagers in Virginia can still be harassed and threatened with impunity. In the national rhetoric on teacher evaluation, the value-added human growth in empathy, kindness, tolerance, and/or citizenship have been lost. Education reformers suffer from the political allergy to looking at complex issues as interconnected. 50% of low-income students who go to college do not finish with a degree; one of the greatest indicators of finishing college is parental income. Education and poverty cannot be addressed as if they are not related. Poverty haunts my former students in the form of– dead mothers, fathers, brothers, asthma, racism, classism, and negativity. The focus of NCLB is Adequate Yearly Progress. But what is adequate twenty-year progress? What kinds of citizens and society do we want? If 99% of citizens were “proficient” in reading and math, would that be enough?
I was driving while listening to your show today, and could not call in a comment. I just had to second the comment that teacher evaluation should be geared toward teacher improvement rather than punishment.
The problem is that principals and supervisors too often do a lousy job with teacher evaluation. I taught for twenty years before I got a helpful evaluation. I will never forget it. A new principal told me my problem was time management. I sort of knew this, but did not know what to do.
He did not stop with the critique. He gave me a book to read, recommended a course to take and gave me 3 goals to work on for the next evaluation period. No principal had EVER given me this kind of personal help. It made a lasting difference in my effectiveness as a teacher.
Training for principals and supervisors is sadly lacking. Most principals and supervisors are promoted from classroom teaching. I hate to say it many of them were not good teachers and got the advanced degrees to move up and get out of the classroom.
In reply to MDavey. So true. I was a school librarian for several years. One year the school board was "touring" our school, which seemed to consist of walking down all the halls and glancing into the rooms. One school board member stopped outside my door. I invited him into the library and his reply was "No thank you. I never did care for books and reading." Need I say more.
Diane: Please think about expanding this discussion. From the responses you received, it is obvious that real teachers need a voice. Not the voice of a union or an adminstrator. These comments were educators asking for help. I was a history teacher at a low performing school, and I did receive the value added bonus. The total emphasis of the district was data. My school's population was mainly students who performed at a 5th or 6th grade level even though they were classified as 10th or 11th grade. The teachers had the task of preparing this group for state test that usually happened at the end of April. So much remedial teaching had to be done. Still the expectation was to have the students ready for April. Many comments touched on the one size fits all. My school is one of probably hundreds that did not fit that model. The diversity in public schools is so broad. With all the emphasis on data and value added, the questions might be how to keep the good teachers and elevate the weaker ones. If data is becoming the main goal, how much real teaching will take place? Teachers who are in the classroom are not being heard. There are more influences on students than teachers, and that is a discussion which should be added to this mix. Is there any possibility that you would consider a conference of regular teachers with the value added presenters or some sort of continuation of the issue? This subject is critical. If America does not have an educated populace, we cannot stand. With dropout rates that exist now, it does not appear that the problem of how to educate is being addressed appropriately.
I am a recent PhD graduate who has just begun adjunct teaching in a large state university, and I can assure you that the attitude of students has shifted dramatically since I was in school. I believe this is because of the focus on evaluating all education through testing. The students have been trained to take tests, think in terms of testing, and ignore anything that will not be tested. Furthermore, they expect the "testable" items and information to be clearly identified during class so they know what to ignore when they study. When I tell them that everything in class is testable, they act as if I am expecting insane things from them. Since when was a class not about learning all the course information? Why in the world would I waste time telling them things that aren't important? Knowledge and learning clearly have been de-emphasized in their high school educations, and I'm sure many of these students came from "high performing" schools. I was speaking to a high school teacher about this matter a few months ago, and he told me the college professors need to hold the line on insisting on knowledge acquisition from students because their hands are tied. College professors have enough on their hands without having to coax infantile 18-year-olds into adulthood where they must take responsibility for their own work.
Speaking as someone who considered going into the teaching profession (in college from 2005-2009), I was hesitant to pursue that degree when I saw what a thankless job it was and how steeply the demands had increased, both on students and educators. Instead, I earned my degree in public relations from a public university in New Jersey, specialized in school public relations, and started working for Character Education Partnership after researching the connections between academic achievement, parental involvement, and comprehensive character education.
Most agree that the current standardized-test-based measure of success defined by NCLB does not capture the true essence of successful education, and does not measure learning with a wide-enough lens.
Through my research and work with CEP, I have learned what leads to success in school, and it's not one answer, or a quick fix. It's a network of things, and it starts with students feeling like they have a purpose and teachers, the community, and parents reinforcing it.
I was happy to hear today about the movement to revamp teacher effectiveness evaluation. It's important to keep social interactions human, and administrators going in to observe a teacher's success and progress throughout the school year is important to being able to assess the teacher's effectiveness.
I think if more people recognize that we are a society that needs to work together and support each other to achieve a greater good, instead of disconnecting from and isolating ourselves from each other, schools will be better able to reach their goals of producing well-educated, socially-responsible members of society. Just one girl's thoughts.
Also, if anyone is interested in learning more from our CEO, he wrote a blog post about cheating and whether teaching to the test actually hurts our kids. www.character.org.
First off - I am 71 so when I speak of my experiences, you'll have an idea which generation I belong to.
The current state of our educational system took over 50 years to get to.
The problems are so entwined that trying to solve it from one direction will be impossible. That being said, certain directions will yield better results than others. The problem started as an unintended consequence of "women's lib". I AM NOT ASSIGNING BLAMING NOR AM I AGAINST WOMEN'S LIB. When I was in public school 1st - 6th grade (mid to late forties) all but two of the teachers were women. In those days female college grads found it very difficult to get into business, law, engineering, and many other professions. What was readily open was nursing and teaching. The best and brightest went into those professions. Women's lib opened up whole new worlds to women and the best and brightest no longer were funneled into the classroom. One direction from which the educational mess must be attacked from is getting more of the best people back into the classroom. To do this, salaries must be made attractive. Businesses continually justify huge salaries by claiming it is the way to attract the best people. WHY NOT SO in education? My thinking assigns 70% of the blame for the current state of education onto the PARENTS. The remaining 30% can be distributed among the "system", the teachers, and the children themselves. I have not been able to figure a way to make the parent more responsible (remember all of the entwined problems?). IF I were a KING, I would require an amount of money put into an account by each parent at the start of each term. Money would be withdrawn by the school each time their child failed a test, each time their child was sent home, etc. WAIT,WAIT, DON'T TELL ME - I'm aware this probably violates some laws, could be considered unfair, etc, etc.
In response to Linda Wilson:
TFA collects self-reported data on student reading growth and student mastery of the math standards because I taught 1st graders who weren't taking state tests yet. If teachers don't adminster those assessments effectively, or with a true understanding what excellent work looks like, the data can end up skewed. I also think that a big difference between truly effective teaching, and mostly competent teaching is how well kids retain the knowledge over the summer versus backsliding a lot or acting like they don't remember math at all. My first year they backslid more than they should have over the summer, and the paper based math assessments I now know weren't really a rigorous assessment for those skills.
I also think that a lot of state tests are not rigorous enough, so just relying on that data doesn't mean a child is getting an excellent education.
You might find it interesting to look into the curriculums of some of the high performing charter networks if you haven't already (KIPP and here in denver West Denver Prep and DSST). The curriculum for much of the instruction IS the test, versus a balanced curriculum that teaches lots of strategies and then teaches kids how to apply them to the test.