7

Should the company promise to never again conduct a survey of this sort? Should it go even further and explicitly ban research intended to manipulate the responses of its users?

Cousins Jeri Lynn DeBose, Tish Hoover, and Josephine (Joey) Parks looked forward to meeting up during the Christmas holidays to compare notes on the results of midyear teacher evaluations. All were public school teachers in districts scattered over the state. In the pressured search for new levels of teacher accountability demanded by legislators, the state department of education joined 16 other States in implementing a new teacher evaluation system. The goal is to hold teachers account-able for student learning progress in the classroom. Under the guidance of the National Council for Teacher Quality, criteria varies by State, but in most cases, 40 percent of each teacher's accountability score would be based on the principal's evaluation and ranking based on personal observation, 30 percent would be based on personal observation by a master teacher from outside the district, and the Other 30 percent would be based on student test score gains. The state department of education would set a performance goal each school district, and the principal would set a performance goal for each teacher. In preparation, the State conducted intensive training sessions for principals and designated master teachers who would conduct the evaluations based on four class observations per teacher. Officials used standardized achievement tests to derive value-added scores that measure student learning over the year. Teacher ratings were I —5, with I am being the lowest and 5 representing near perfection. The publication of the first year's evaluations stirred interest and controversy, particularly among teachers who worried about the possible long-term effects on job retention and tenure. Now, with the first-year evaluations in hand, the three cousins pored over their experiences. The three represented different types of school systems within the state. Jeri Lynn worked for a metropolitan system in the state capital. The system included many low-income students whose first language was nor English, and several schools within the system were teetering on the brink of State takeover if improvement in student scores didn't materialize this school year. Tish worked in a county System dominated by upper-income residents, and Joey taught in the rural community in which all three grew up. The rural community had high unemployment, and a low percentage of graduates went on to college. As a result, the cousins came to the table with differing teaching experiences. "The numbers are all over the place," Jeri Lynn remarked as she studied the pages. "The whole system is flawed, and they need to make changes," Joey said. " It's too subjective. The principal and master teacher observations are subjective because there are personal factors that affect a true outcome." "Yeah, look at the numbers from your upper-income district," Jeri Lynn said to Tish. "How can 60 percent of the teachers score 5s?" Tish chuckled. "Yeah, lucky us. Our schools are overflowing with children from wealthy families. These are the kids who will apply to Ivy League schools. I can tell you that the principals are going to avoid confrontation on all fronts. No principal is going to give any indication that their students are receiving an education that's less than perfect, and that means cramming the rankings with 5s. They claim a higher level of motivation for students, and thus the selection of an elite team of educators. So with those pressures, I don't think we get personal feedback that IS accurate." "At the other end of the spectrum, we have my rural district," Joey said. "The big problem is that the principals know everyone and have longstanding relationships with everyone in the county, so I think scores are based on personal history. We could almost predict who would get high or low scores before the observations. For principals, it can go back as far as his daddy and my daddy hated each other in high school, and now I get to evaluate his daughter "I think that in many cases, principals feel pressure to align scores with state expectations. The stare expected my district to have high scores and expected rural schools such as yours to be lower," Tish said. "But isn't that partially offset by lower goals for the rural school distracts responded Joey. "The key to the accountability system is the principal in each school," Jeri Lynn suggested. "With several of the schools in Metro teetering on the edge of state takeover by the end of the year, we had lots of strict principals who wanted to hold our feet to the fire with lower scores." "l thought the whole idea was to provide the teachers With feedback so that we would know the areas where we need improvement," Tish said. "The principals were supposed to conduct two observations in the fall and two more in the spring," Jeri Lynn said. think that's asking too much of them when they already have so much on their plates. I think a lot of them are skimping on their visits. know I only had one observation last semester, and I'm sure Mr. Talley just faked the second set of numbers. The master teachers make only two observations a year, which may be more objective but counts for less." "I'm wondering, too, how a principal measure performance in a course area outside his area of expertise, such as math," Joey said. "If the guy has a phobia about math, anything the teacher says or does is going to 100k brilliant—thus a 5." Tish and Jeri Lynn looked at each other and laughed. Maybe we picked the wrong subjects," Tish said. "My question is one of perception," Jeri Lynn said. "A large percentage of my students are ELL. That affects their scores. How do you measure a 3 in my situation against a 5 for Tish? At the end of the school year, little Carlos is thrilled that his reading in English has improved, but there's no Big Bang here. a slow steady improvement that may not actually show up in big strides for a couple of years: "So, the question is how do they create a system that is fair?" Tish asked. "And accurate," added Jeri Lynn. What do you see as the major strengths and flaws in the feedback control system used in the schools in this scenario? What changes do you recommend to overcome the flaws?

Cousins Jeri Lynn DeBose, Tish Hoover, and Josephine (Joey) Parks looked forward to meeting up during the Christmas holidays to compare notes on the results of midyear teacher evaluations. All were public school teachers in districts scattered over the state. In the pressured search for new levels of teacher accountability demanded by legislators, the state department of education joined 16 other States in implementing a new teacher evaluation system. The goal is to hold teachers account-able for student learning progress in the classroom. Under the guidance of the National Council for Teacher Quality, criteria varies by State, but in most cases, 40 percent of each teacher's accountability score would be based on the principal's evaluation and ranking based on personal observation, 30 percent would be based on personal observation by a master teacher from outside the district, and the Other 30 percent would be based on student test score gains. The state department of education would set a performance goal each school district, and the principal would set a performance goal for each teacher. In preparation, the State conducted intensive training sessions for principals and designated master teachers who would conduct the evaluations based on four class observations per teacher. Officials used standardized achievement tests to derive value-added scores that measure student learning over the year. Teacher ratings were 1-5, with I am being the lowest and 5 representing near perfection. The publication of the first year's evaluations stirred interest and controversy, particularly among teachers who worried about the possible long-term effects on job retention and tenure. Now, with the first-year evaluations in hand, the three cousins pored over their experiences. The three represented different types of school systems within the state. Jeri Lynn worked for a metropolitan system in the state capital. The system included many low-income students whose first language was nor English, and several schools within the system were teetering on the brink of State takeover if improvement in student scores didn't materialize this school year. Tish worked in a county System dominated by upper-income residents, and Joey taught in the rural community in which all three grew up. The rural community had high unemployment, and a low percentage of graduates went on to college. As a result, the cousins came to the table with differing teaching experiences. "The numbers are all over the place," Jeri Lynn remarked as she studied the pages. "The whole system is flawed, and they need to make changes," Joey said. " It's too subjective. The principal and master teacher observations are subjective because there are personal factors that affect a true outcome." "Yeah, look at the numbers from your upper-income district," Jeri Lynn said to Tish. " How can 60 percent of the teachers score 5s?" Tish chuckled. "Yeah, lucky us. Our schools are overflowing with children from wealthy families. These are the kids who will apply to Ivy League schools. I can tell you that the principals are going to avoid confrontation on all fronts. No principal is going to give any indication that their students are receiving an education that's less than perfect, and that means cramming the rankings with 5s. They claim a higher level of motivation for students, and thus the selection of an elite team of educators. So with those pressures, I don't think we get personal feedback that IS accurate." "At the other end of the spectrum, we have my rural district," Joey said. "The big problem is that the principals know everyone and have longstanding relationships with everyone in the county, so I think scores are based on personal history. We could almost predict who would get high or low scores before the observations. For principals, it can go back as far as 'his daddy and my daddy hated each other in high school, and now I get to evaluate his daughter." "I think that in many cases, principals feel pressure to align scores with state expectations. The stare expected my district to have high scores and expected rural schools such as yours to be lower," Tish said. "But isn't that partially offset by lower goals for the rural school distracts responded Joey. "The key to the accountability system is the principal in each school," Jeri Lynn suggested. "With several of the schools in Metro teetering on the edge of state takeover by the end of the year, we had lots of strict principals who wanted to hold our feet to the fire with lower scores." "I thought the whole idea was to provide the teachers With feedback so that we would know the areas where we need improvement," Tish said. "The principals were supposed to conduct two observations in the fall and two more in the spring," Jeri Lynn said. think that's asking too much of them when they already have so much on their plates. I think a lot of them are skimping on their visits. know I only had one observation last semester, and I'm sure Mr. Talley just faked the second set of numbers. The master teachers make only two observations a year, which may be more objective but counts for less." "I'm wondering, too, how a principal measure performance in a course area outside his area of expertise, such as math," Joey said. "If the guy has a phobia about math, anything the teacher says or does is going to 100k brilliant—thus a 5." Tish and Jeri Lynn looked at each other and laughed. Maybe we picked the wrong subjects," Tish said. "My question is one of perception," Jeri Lynn said. "A large percentage of my students are ELL. That affects their scores. How do you measure a 3 in my situation against a 5 for Tish? At the end of the school year, little Carlos is thrilled that his reading in English has improved, but there's no Big Bang here. a slow steady improvement that may not actually show up in big strides for a couple of years." "So, the question is how do they create a system that is fair?" Tish asked. "And accurate," added Jeri Lynn. Is a 1-5 grading System by principals and master teachers a valuable part of a feedback control system for teachers? Why?