Remarkable Improvement to Title I Mathematics Reporting
The annual Title I grant comes with accountability requirements. The Title I grant directs that documentation be provided to demonstrate that students in Title I schools are improving in reading and mathematics. In my school district this resulted in the development of assessments given to students to demonstrate acquired skills and learning. These assessments were given at three intervals during the school year (late fall, winter, and spring). The assessments were scored and reports generated to document the learning achievement and how the use of Title I money produced improved understanding for students attending schools designated as “poverty” schools. These assessments and reports were seen only as benchmarks toward educational progress. But they needed to be more—much more. They needed to help teachers understand student learning and develop instructional strategies to improve student achievement.
At first, the assessments were given to students, scored, and results tabulated to produce required reports. This was all accomplished using manual processes by teacher specialists in the Title I schools. It proved to be very time consuming and not very helpful for producing analysis that would impact instruction. How could this process be improved? That was the challenge that Remark Office OMR and our Microsoft Access analysis reports were to tackle.
The Office of Compensatory Education (Title I Office) was charged with assessing student progress in the areas of reading and mathematics for the Title I schools in the district. A previous Tom Hays Corner article discussed the reading assessment and the associated reports used to determine student progress in reading. This article will discuss the trimester assessment of students in the area of mathematics.
Three times a year (November, January, and May) students in the Title I schools were assessed in the area of mathematics. The purpose of the assessments was to demonstrate the progress toward success in the area of mathematics by students in grades 1 through 5. Prior to electronic data collection and analysis, the mathematics assessments were developed, administered and the data collected and analyzed manually. This process was very time consuming. The results of the data were intended to produce the required trimester reports to the state and federal government. Classroom teachers and school administrators were not the intended audience, and as a result were given only cursory reports regarding overall success.
The model of the reading assessments and reports that moved the data beyond the state and federal reporting to the classroom teachers and school administrators was successful and well received by school personnel. Therefore, that same model was used for the mathematics assessments.
The mathematics assessments developed and administered to students were continued with some revision for electronic data collection. The assessment items were not changed, but the format of the assessment was changed to allow for mark sense data collection. This is to say that areas on the paper were created where individual item scores could be recorded in a machine readable (bubble) format. An individual assessment for each student needed to be printed. This assessment form included a bar code that would include the student identification number as well as human readable student information—identification number, name, teacher and school. By having the student identification number on each individual student assessment form, the data gathered about the assessment could be linked to a student database that would enable additional data disaggregation for reporting. This was not possible with the manual data analysis previously used.
The best way to explain how the assessment forms were used and how the data reports provided some impetus for instructional improvement is to look at these documents. The assessment form included with this article is for the second trimester for grade 4. Each grade (1 through 5) had similar forms created. The assessment items were developed around the curriculum taught during the previous months in school. Teacher specialists selected the items that would best reflect the concepts taught from the beginning of the school year. The assessments were printed on 11 x 17 inch paper in a landscape format that was folded to create four 8 ½ by 11 inch pages for the assessment items and scoring marks. The assessments were checked and point values marked by the classroom teacher.
In addition to the individual student assessment forms an answer guide was printed. The answer guide was an exact replica of the student assessment except that the answers were provided. This enabled the teachers to quickly check the student assessment responses and record the points awarded for each. This did not relieve the teacher of the checking process, but it did ease the process and enabled the teachers to have data analysis of student learning.
Another advantage of providing the assessment forms was the ability to revise the forms from year to year to reflect the changes in instructional content and emphasis. If an item was found to be not useful for determining student learning or if the wording of an item was seen to be unclear to students, the next year’s assessment could be modified to accommodate the new changes required. This was accomplished with minimal effort.
After the teacher marked the student papers, the answer sheets were sent to the Title I Office for scanning and reporting. It was important that reports be printed in a timely manner to help the classroom teacher understand student learning and be able to make instructional modifications to improve teaching the students. Scanning took place and class reports were printed and returned to teachers within 48 hours of receipt of the assessment documents. At the same time, school reports were delivered to principals and teacher specialists. Data collected from all Title I schools was then aggregated into a total Title I data set that was used by the school system and to produce state and federal reports.
The original classroom report provided was printed in black and white with the entire class list on one sheet of paper. This report identified each student, the number of correct responses, and the percent of correct responses. Most importantly the description of the assessment item and its outcome reference as well as the number of points each student earned for each item was listed. At the bottom of the classroom report was a class average for total score as well as average for each item. The item average indicated the overall class understanding of a concept being measured. A point 6 (.6) average for a 1 point item indicated that reteaching to the entire class might be appropriate. Finally, the Student Counts line indicated the number of students in each range of scores as well as the number and percent of students deemed passing the assessment.
Eventually, the classroom report was color coded to better help teachers visualize which students and which items were in need of attention. Pink was used to indicate immediate and complete attention was needed, yellow was a cautionary color to indicate some attention may be needed, and a white background indicated success. This reporting improvement was greatly appreciated by teachers.
The school summary reports that were provided to the school administration were also used in the Title I office to complete the state and federal reports and are included with this article. The breakdown of the data allowed for disaggregation on the basis of several measures: poverty status, gender, race, special versus regular education status.
All of the reports were the result of the ability to electronically collect data and link that specific data to individual students via a student demographic database. Remark Office OMR made the entire process easy and user friendly. It specifically made the ability to connect the scanned data to a student demographic database extremely easy. The real power of data collection resides in the resulting reports. Getting useful data to teachers in a timely manner is very critical. The reports helped teachers know how each student did on an assessment, and clearly showed instructional implications. Teachers could then easily direct teaching to help students understand the mathematics concepts they may have misunderstood.
The mathematics trimester assessments and the accompanying reports were able to fulfill the requirement of determining student improvement and providing the documentation required by the Title I grant. Because of the improved method of electronic data collection and reporting, the task of providing documentation was moved from a time consuming manual process to an efficient electronic process. The electronic process provided additional benefits of available reports rapidly given to classroom teachers. These reports were more than just as a measure of student achievement; they also were used as an instructional tool to identify areas that warranted additional instruction to the class as well as individual students. Remark Office OMR was the software program that enabled this process to become an electronic reality. This change from a manual process that only provided data to fulfill the Title I grant requirements to an electronic process that impacted classroom instruction was truly a remarkable addition to the Title I schools. This success provided the impetus to move to more ambitious projects in the area of mathematics instruction and student achievement.
Return to Tom Hays' Corner.
