Provide your opinion regarding the impact that poor evaluation questions can have on the outcome of a project. Support your position with an example. From the case study, examine the preliminary considerations one takes into account when preparing to evaluate performance. Select an assessment measurement and provide a detailed suggestion as to how Tina can use it for the K-12 laptop program. Provide one (1) example of another setting where one could use the selected assessment measurement.
Case Study 6 Tina Sears
Evaluating the Impact of a K–12 Laptop Program
by Michael M. Grant, Deborah L. Lowther, and Steven M. Ross
Andersen was a rural, southern community tucked in at the base of the Blue Ridge Mountains in upstate South Carolina. Like other small, close-knit towns in the state, more than half the students in the four-school district qualified for free or reduced lunch. Furthermore, a number of students were minorities, at least in part because a large number of Mexican American families had recently moved into the area.
Hillendale Textiles was an anchor of the community. Not only did many of the parents work one of the three shifts, but CEO and owner Bradley Cook had often stepped in to provide the school district with funds for special projects, such as the high school’s football stadium and new band uniforms. It was easy to see why the rotund Mr. Cook had the ear of the school administrators and the school board.
The Pilot Laptop Program
Another special project was now in the works: In late October, Mr. Cook received a call from Darren Chaude, a business associate at Toh, Inc., an aggressive computer systems firm headquartered in Southeast Asia but well established in the United States. (Hillendale Textiles used Toh, Inc. hardware for its fabric manufacturing lines.) Darren called because he knew that Mr. Cook was an active supporter of his local school district and might be interested in Toh, Inc.’s laptop program for K–12 schools. Toh, Inc. had just become highly visible in portable technologies and was vying for market share with extreme competitive pricing. Darren shared his opinion that the program had been shown to not only improve students’ learning, but also to prepare them for the high-tech job market. He further explained that the program included 60 hours of teacher training from a national expert, free teacher laptops, full technical support, and discounted prices on volume purchases of laptops. Mr. Cook was very impressed with Darren’s description of a recent visit he had made to a fifth-grade laptop classroom. Darren said he had never seen students with such a high level of computer skills or who appeared so motivated to learn.
The conversation piqued Mr. Cook’s interest, so he decided to investigate the possibility of funding laptops in Andersen County School District. After some initial investigation, and following further discussion with Darren, he decided on a pilot program with the eight fifth-grade classrooms. He did not want to buy laptops for all grades, particularly if the program proved to be unsuccessful. His two primary reasons for funding the program were to improve student learning and to increase employee loyalty by supporting a program that benefited their children. Of course, it wouldn’t hurt that he’d get a great tax write-off as well!
Capriciously, Mr. Cook skipped consulting the district before signing the contract with Toh, Inc. He wanted to present the news at the Winter Teacher Meeting, which was attended by all teachers and administrators—and the newspaper. With a certain amount of shock (which was to be expected) and an equal amount of gratitude (which was also to be expected), Andersen County School District set about to make Mr. Cook’s vision a reality.
As District Technology Coordinator, Tina Sears was appointed project director. Plucky and adventurous, Tina, with a small committee of committed parents, fifth-grade teachers, and the school principals, ordered the equipment from Toh, Inc., scheduled the teacher professional development sessions, and prepared to document the changes in teaching and learning. The committee wrote some program goals and distributed them in a flyer to the parents and school board (see Figure 6–1).
Figure 6–1 Pilot Laptop Program Goals and Professional Development Plan.
|Andersen County School District Pilot Laptop Program|
|The purpose of the pilot laptop program is to:|
|1. Equip each 5th-grade classroom with high-performance laptop computers2. Ensure students and teachers are computer literate
3. Increase student achievement through the use of laptop computers
|PROFESSIONAL DEVELOPMENT PLANIn order to achieve the program goals, the 5th-grade teachers will participate in a comprehensive two-week professional development (PD) summer seminar and monthly three-hour sessions during the school year. The training activities will engage teachers in three types of activities:experiencing classroom simulations, creating materials and resources to support technology integration efforts, and practice modeling scenarios to build implementation and management skills. The specific PD objectives are below.
The teachers will be able to:
· Create lesson plans that engage students in effective use of laptop computers as a tool to improve achievement on the district’s standardized assessments
· Integrate effective student use of laptop computers into everyday teaching and learning
· Manage classrooms that have students using laptop computers
Teacher Professional Development
With the equipment on the way and the summer holiday steadily marching by, Tina spoke with Toh, Inc.’s teacher trainer, Mark Waters, a nationally recognized expert in the field of K–12 technology integration, about running the professional development workshops for her teachers. His full-time job was as a professor at a large mid-Atlantic university; he also consulted for Toh, Inc. with schools that were beginning new laptop programs. Mark’s approach centered on encouraging teachers to engage students in using the computer as a tool during problem-based activities. In essence, the teachers wouldn’t just learn how to use computers, they would learn how to teach while using computers to meet their objectives. Mark emphasized that computers wouldn’t just be an add-on to the curriculum; they would be integral to achieving the objectives outlined in the curriculum. Computers would not be a reward for the “smart kids” or for finishing work early; every student would use computers to support and enhance classroom activities.
Essential to the two weeks of professional development in the summer was a series of simulation activities in which Mark modeled the role of a fifth-grade teacher while the teachers assumed the roles of fifth-grade students. For example, while learning about plate tectonics, the teachers/fifth graders would use concept-mapping software, or while learning about similes, the teachers/fifth graders would use presentation software. Mark began each lesson, “Today, we’re going to learn about…” and filled in with “science, social studies, math, or language arts.” Then he would say, “The point of this lesson is not to learn everything there is to know about…” and filled in with “Word,” “PowerPoint,” “Excel,” or some other popular software package. Throughout the two weeks, Mark illustrated how the software applications had specific uses, and how teachers should match each application’s use to their curricular objectives.
Implementing the Laptop Program
As school began in mid-August, the heat didn’t give in, and neither did Tina and her newly trained fifth-grade teachers. During the school year, three-hour follow-up workshops were conducted monthly to maintain the momentum generated during the summer workshops. The teachers planned lessons collaboratively, and Tina guided them through making the best uses of technology for learning.
Documenting progress continued to nag at Tina. She couldn’t let the evaluation slip her mind. To move this task along, Tina called on a friend from a neighboring district who had received computers for her library from a local literacy council. Her friend reported that she had used student, teacher, and parent surveys, but had also wanted to tell the “true” story of how the computers were being used. To do this, she regularly video recorded the students as they were accomplishing different activities, and then created an edited version to show at the annual literacy council’s fund-raising dinner. The video recording was an incredible hit and resulted in the library receiving 10 more computers.
Tina decided she would definitely include video, but she wanted to go even further. So she decided to hold focus groups with the teachers and students to obtain information that wouldn’t show up on a survey. She also added technology to the district’s annual teacher evaluation form, which was a huge improvement because the evaluation hadn’t been changed in years. And, of course, she needed to include the Iowa Test of Basic Skills (ITBS) scores as part of the data collection as well. She dashed off these descriptions in a fax to Andersen County’s school superintendent, Dr. Tammy Burns (see Figure 6–2).
It was unseasonably warm for January, and Dryer Elementary was alive with activity for fifth-grade parent night. Tina beamed throughout the whole evening. The shiny laptops were on display, and streaming videos of students using the laptops were projected in the classrooms. As the images on the screens changed, Tina recalled seeing students retrieve census data from the web and then place it in Excel to make predictions on future population trends. She also saw students team-writing a paper using a web-based word processor. The video revealed that students used technology tools seamlessly to support their learning, including the thesaurus, online dictionaries and encyclopedias, concept mapping software, and graphing calculators. The students also instantaneously searched the web when questions arose during discussions or problem-solving exercises.
It was obvious to Tina the changes were not just with the students; video clips affirmed that the teachers were changing, too. They were not glued to the front of their classrooms. They moved about, talking with individual students and small groups.
The next day it seemed cooler—closer to normal for January—as Tina met with the fifth-grade teachers and recorded their pride from parents’ night. As expected, a photo of Mr. Cook surrounded by laptops and excited fifth-grade students made the newspaper front page.
Spring announced its arrival early in March, and now, everyone was sweating again. Tina was pleased that the time in her muggy, yellow office was abbreviated today. She had scheduled video recording two of the fifth-grade laptop classrooms. This was always a highlight in her day because of the amount of progress that had been made in many of the classrooms. In the most innovative classrooms, students were so engaged in their assignments that they hardly noticed the digital recorder (an enthusiasm she wished would transfer to all the classrooms). Teachers were busily moving from group to group answering and asking questions. Students and teachers frequently used the digital projector to share and discuss their work.
With a subtle smile on her face, Tina reminisced. All the classrooms’ activities and excitement reminded her of when she was selected as the only teacher in the district to have computers placed in her classroom. Back then, students were limited to basic programming and using rudimentary software like Logo. Many students were motivated with turtles and geometry. Yet, even with her limited resources, she used the computers to create a learning environment where more students wanted to learn. Her class was even on the front page of the newspaper, which probably contributed to her appointment as district technology director. Her pluckiness surely helped, too.
Tina collected the last of the parent, teacher, and student surveys, which had straggled in. It was a 70% response rate; the phone calls to homes had eked out a remarkable number of returned surveys. She felt quite pleased that almost everyone marked “Yes” to her two key questions: (1) Do you think students are learning more? and (2) Have the laptops made students more interested in learning?
Figure 6–2 Tina Sears’ Evaluation Plan.
|FAX Transmittal Form|
|To:||Dr. Tammy Burns
Andersen County School District
District Technology Director
No. of Pages:
|Pilot Laptop Program Evaluation
See the evaluation plan below for the pilot laptop program for the eight fifth-grade classrooms.
|Pilot Laptop Program Evaluation|
Mr. Bradley Cook, CEO of Hillendale Textiles, generously donated laptop computers to our eight fifth-grade classrooms. He has agreed to provide laptops for all elementary classrooms, if evidence can be provided to show that use of the laptops improved student learning. Therefore, the purpose of this evaluation is to demonstrate how providing laptops for each fifth-grade classroom improves student learning.
Our district has planned the following activities to demonstrate the positive impacts of having laptop computers:
|1. Add “Technology Use” to the district annual teacher observation form.2. Throughout the year: Collect videotapes of successful laptop use.
3. At the end of the year: Conduct teacher focus groups to see how the teachers feel about using laptops.
4. At the end of the year: Give teachers, students, and parents a survey to see how they feel about the use of laptops.
5. Compare last year’s ITBS scores with this year’s scores.
Initial Evaluation Results
School had been out for a couple of weeks now, and so had Tina’s air conditioning. She carefully transferred the data—the bulk of which were positive survey results and teacher focus group responses—to the final report that she had written. For example, one teacher said, “I’ve never seen so many of my students so excited to learn.” Another commented, “I’ve been teaching for 15 years and have rarely seen so many student products that were written so clearly and demonstrated such deep levels of learning.” Tina also created an edited version of the videos that showed multiple examples of classroom environments that were improved because of the use of laptops. She was confident that the report and video would please Mr. Cook and would be a great springboard to expand to the other grades. She printed out the report and dropped it in the mail, along with the videotape. The smile on her face showed her elation over the report; the sweat on her brow divulged her prayer for the HVAC repairman.
Torrents of rain pounded outside. The almost 100-degree weather was a constant for summer in South Carolina, and Tina’s small air conditioner strained to keep up. Adding insult to injury, it was only hotter after it rained.
The e-mail alert was a welcomed distraction. Tina quickly opened the message and followed the lines of text across the bright screen (see Figure 6–3).
The message was from Dr. Burns and included an original message from Mr. Cook, sponsor of the laptop program. It was obvious that Mr. Cook was unhappy with the evaluation results.
As she read Mr. Cook’s message, Tina’s usual effervescence quickly diminished. How wrong could she be? It had been a whirlwind year, but how could Mr. Cook think that the program was not a success? He had focused on the lack of change in the test scores and had discounted positive survey results. He had also dismissed her reports from the teacher focus groups and the video. The fifth-grade teachers had accomplished so much; she had witnessed it first-hand. Luckily Mr. Cook conceded funding for a formal evaluation the following school year, postponing his decision about discontinuing the laptop program.
Maybe the information in the report had not been definitive enough. Maybe the data she had collected were not convincing enough to warrant another significant expenditure from the textile manufacturing company’s foundation. Cook’s threat to kill the funding flooded her mind. “I’ve got to do something about this evaluation,” she thought. “I can’t let the other teachers and students down. I’ve seen so much progress with the laptops.” She began to question herself, and now she needed to host a meeting with all the stakeholders.
Without a doubt, she needed help with the requested evaluation. Fortunately, Mr. Cook was willing to pay for it, so she should get some outside help. That would relieve some of her pressure and help mitigate her bias. She decided to contact the Toh, Inc. consultant, Mark Waters, who helped with the teachers’ professional development. He had been a breath of fresh air with the teachers and the professional development. Somewhere in the back of her mind she recalled him mentioning something about program evaluation.
Figure 6–3 E-mail from Superintendent, Dr. Tammy Burns, to Tina Sears, District Technology Director.
|Subject:||FW: Laptop Evaluation Report|
|To:||Tina Sears <firstname.lastname@example.org>|
|From:||Dr. Tammy Burns <email@example.com>|
Below is an e-mail from Mr. Cook regarding your laptop report.Begin to schedule the stakeholders’ meeting so that we can have an evaluation
plan in place for Mr. Cook by October 1.
|Subject:||Laptop Evaluation Report|
|To:||Dr. Tammy Burns <firstname.lastname@example.org>|
|From:||Bradley Cook <email@example.com>|
|Tammy,I am disappointed in the lackluster results documented in the Pilot Laptop Evaluation Report submitted by Ms. Tina Sears, Andersen County School District Technology Coordinator. The report documents the positive attitudes by teachers, students, parents, and administrators, and the video clips were moving. But most important to me, this report documents no improvements in student scores on the ITBS. I am considering pulling the funding and dropping the expansion to the other grade levels.
However, if the district can produce a solid evaluation plan that better demonstrates the impact of laptop computers on student learning, I will fund a full evaluation to occur during the next school year. If this report yields positive results, I will extend the laptop program to the remaining elementary grades. For me to consider this new evaluation to be thorough, the school district will need to host a meeting to discuss points of view from teachers, principals, parents, and parent-teacher associations. In addition, the Hillendale Textiles foundation board and myself will need to be present as the sponsors of this program.
|Let’s talk soon,Bradley|
Planning the Future Evaluation
School started back. Mark had been a gold mine. Tina couldn’t believe her luck. Mark worked within a center on campus that helped local and nationwide schools with documenting changes from instructional interventions, just like her laptop pilot program. The unassuming director of the center, Dr. Lisa Colm, was a nationally recognized and well-respected leader in the field of school evaluation. She had backgrounds in educational psychology, statistics, and psychometrics, but her personality tempered audiences’ reactions to these sometimes challenging topics. While discussing program results with legislators, school boards, and other bigwigs, she often asked, “Is this in the best interest of children?” She made the results and numbers real—because they represented real teachers and real children.
After some discussion, the center agreed to handle the laptop pilot’s external evaluation. Together with a team from the center, Dr. Colm agreed to facilitate a meeting, a week later, among Andersen County School District’s stakeholders: teachers, principals, parents, the PTA president, community members, Mr. Cook, and the Hillendale Textiles foundation board, as well as the school board members. “A tough crowd for sure,” Tina sighed. She was just relieved she didn’t have to lead the meeting.
In the small, cramped boardroom at the school district office, all of the chairs were filled. Tina sat quietly in the back and fanned herself. While Dr. Colm expertly maneuvered the discussion around the laptop pilot program’s purpose and goals to the various attendees, three members of the evaluation team scrawled copious notes. The purpose of the project, how the teachers and students had used the laptops, anecdotes from teachers and parents, and emphases on student test scores were all covered.
As purses and yellow notepads were picked up, sweat-soaked tissues were tossed, and the various stakeholders filed out, Tina approached Dr. Colm and the evaluation team. She was apprehensive; she knew these folks were her best help, or the project was sunk. Dr. Colm smiled and reassured Tina: “I’m confident we can help.”
The Next Day
Back at the university, Dr. Colm and the evaluation team met in their cool, blue conference room to debrief the stakeholder meeting from the day before. The relaxed team compiled all their notes onto a large whiteboard wallpapering one side of the room (see Figure 6–4).
Figure 6–4 Whiteboard Notes from Dr. Colm’s Evaluation Team Meeting.
|Do teachers teach differently when lessons include student use of computers?||How do students’ scores change, if at all, on the ITBS?||Do students behave differently when they use the laptops?|
|In what ways do parents support the laptop program?||In which subject areas do student ITBS scores increase the most?How often do students use the laptops?
In which subjects do students use the laptops the most?
|How confident do teachers feel to integrate students’ use of laptop computers into their instruction?|
Reviewing the whiteboard, Dr. Colm sighed. She was confident that her team could readily craft a proposal within the timeline. However, addressing all the stakeholders’ questions, particularly those focused on the ITBS test scores, would be more difficult to achieve, and it would be even harder to make others understand why it was difficult. Yet, with a confidence that came from having addressed these issues many times before, she turned to her team and smiled, “Well, team. Let’s get to work!”
Preliminary Analysis Questions
Implications for ID Practice
PLACE THIS ORDER OR A SIMILAR ORDER WITH MY ONLINE PROFESSOR TODAY AND GET AN AMAZING DISCOUNT
The post Provide your opinion regarding the impact that poor evaluation questions can have on the outcome of a project. Support your position with an example appeared first on MY ONLINE PROFESSOR .