Sample Files
Teacher directions for completing scan sheets
Fall data collection scan sheet
Spring data collection scan sheet
Fall class listing of language arts score profiles
Fall class listing of mathematics score profiles
Fall individual student profile showing language arts and mathematics scores
Spring class listing of language arts score profiles
Spring class listing of mathematics score profiles
Comparison of Fall/Spring readiness for one school by each teacher’s class
Comparison of Fall/Spring readiness by school and system wide
Tom Hays
Data Coordinator for Title I
(Name withheld) Public Schools
My first article chronicled the beginning of my relationship with Remark Office OMR as I developed a data collection system to determine the efficacy of an intervention program in reading for grades 1 and 2. In this article I will discuss a remarkable addition to the data collection efforts: Evaluation of the kindergarten program through two assessments.
The Early Childhood Office had been involved with evaluating the kindergarten program through the use of two assessment instruments. One assessment was given at the beginning of the school year, prior to the end of the first 3 weeks of school. The second assessment was given about 3 weeks prior to the end of the school year. The first was to determine the school readiness in the areas of reading and mathematics of the children entering the kindergarten program, the second to determine the knowledge and skills learned as a result of the kindergarten program. The intent of the two assessments was to determine the effectiveness of the kindergarten program with regard to all students. Of particular interest was how well the kindergarten curriculum was being accessed and acquired by students who were assessed as below the expected readiness level. All of the data had to be collected and then hand analyzed. Needless to say, this process was very tedious and error prone, and extremely late in providing the results.
After the first experience working with the reading program, it was obvious to me that the kindergarten data collection could be accomplished with the same testing process, but a different collection and analysis method. Because this was a new idea to the supervisor of the Early Childhood office, a pilot project would be used to show the feasibility of collecting the data. The pilot for the project began with just 5 of the 30 elementary schools.
The first thing to be accomplished was a redesign of the assessment document so the data could be easily scanned and placed into a database. Because the teachers would still be giving the same assessment that they had used in previous years, there would be little change in the assessment process. However, due to the change in the method of recording the results of the assessment, teachers would require a little training on properly filling in the scannable document. Marking bubbles on individual assessment sheets instead of recording numbers on a grid for a class of children required about 15 minutes of training. This training was handled by the Early Childhood office.
The pilot project convinced the supervisor of Early Childhood of the feasibility of the project. The real clincher for both the Early Childhood office and the school principals was the rapid turn around of the data. Although there was no beginning of the year assessment to compare with this end of year data in the project, the fact that there was no hand tabulation of data and the reports could be back in the hands of the teachers, principals, and supervisor within just a few days was a pleasant surprise to them. This pilot project and the initial reports were given to the Director of Elementary Education for approval for use by all 30 schools. There was no question about should we go forward, but could we get it in place for the beginning of the next school year?
Of course, I could get the documents and database in place, but training all kindergarten teachers and convincing them the value of this process would need some thought. The pre-school meetings for the school year would be the time and place for this training. In addition to training the teachers to fill in the bubbles, the benefits to the kindergarten teachers would also need to be addressed. The demonstration reports proved to be the most useful item to get the teachers on board with the process. These reports showed the data in several different ways. One report was a class list arranged by reading score, another was the class list arranged by mathematics score, and another was an individual student report showing both the reading and mathematics scores along with the sub-test scores. The individual report was seen by the teachers as a very useful report for the back-to-school meetings with individual parents. It provided the teachers with talking points to help parents realize the strengths and needs of their children. (This proved to be a very good public relations tool for the teachers.) After the first full year of implementation of the data collecting and reporting system, both the teachers and administration agreed that the time and effort produced results that warranted the continuation of the program.
Attached to this article are two scan forms (fall and spring), directions provided to teachers for completing the forms, and samples of Fall and Spring reports. The Fall scan form is on an 8 ½ by 14 inch page that records the language arts tests and subtest scores on the left side and the mathematics scores on the right side. Again, the small amount of space required for the bar code that links to the student demographic database was critical to allow for this design. The Spring form is on an 8 ½ by 11 page, but using both sides of the paper – one side for language arts, the other for mathematics. On both forms, the adult assessor (since each child was assessed individually) was to initial the assessment score recorded. Different adults were responsible for assessing children in mathematics and language arts. Also, only the fall assessment collected data on pre-kindergarten educational experiences of children. Remark Office OMR’s ability to export the scanned data to various database products made the data useful to provide information to teachers and administrators about the Kindergarten program and individual students.
The real power of a data collection and reporting system is in the information that is provided to teachers and administrators for instructional decision making. In addition to the samples of the Remark Office OMR scan sheets, I have provided samples of some reports. Additional reports that disaggregate data in a variety of formats were created when special requests were made. It needs to be noted that the database program I used for analysis and reporting is Microsoft Access® since I am familiar with this program. However, any relational database would work just as well for a person familiar and competent with the database program. That is a great feature of Remark Office OMR—its ability to export data to just about any database format.
- Teacher directions for completing scan sheets
- Fall data collection scan sheet
- Spring data collection scan sheet
- Fall class listing of language arts score profiles
- Fall class listing of mathematics score profiles
- Fall individual student profile showing language arts and mathematics scores
- Spring class listing of language arts score profiles
- Spring class listing of mathematics score profiles
- Comparison of Fall/Spring readiness for one school by each teacher’s class
- Comparison of Fall/Spring readiness by school (only one shown) and system wide
The teacher directions were sent to each of the teachers along with his/her class set of fall and spring scan sheets. Within one week of receiving the completed scan sheets from the schools, teachers received from my office a language arts and a mathematics class listing profile and an individual student profile for each student. The Early Childhood office would receive a report showing the number and percent of readiness students for each teacher and each school. This report was also forwarded to the school principal. In the spring, the comparison reports are prepared and sent to the school principals, the Early Childhood office and the Director of Elementary Education. These reports have been helpful in informing instruction and in modification of the curriculum. Take a look at the Fall/Spring Comparison of School/Systemwide report. You can see that in the fall, 29.6% of the students were not ready for Language Arts in kindergarten, but by spring only 12.8% of the students were not ready for the 1st grade Language Arts curriculum. Also, 10.6% of the students were advanced in the fall, but by spring 39.9% were advanced in the Language Arts area. By far, the biggest change was in the area of Mathematics. Fall registered 51.2% of the students behind in mathematics, but by spring only 19.9% were still behind. This gives evidence that the kindergarten curriculum is taking students that are behind and advancing them to on grade level. Not all students, but more than 30% more are where they need to be.
The individual teacher reports shown in the examples are listed in alphabetical order by student last name. However other reports were created using the assessment total score. These would be ordered in ascending total score order to assist teachers with grouping students. The color coding on the class listing profiles helped to draw attention to specific areas of student need. For example, on the Fall Language Arts report, you will notice in the "Language in Use" section, the "Describe Obj" column draws the reader’s attention by all of the pink and yellow color background. This is an indication that the entire group of students needs instruction on this objective. On the same report, notice the student named "Brandon" has quite a few objectives unmet. A teacher seeing this report can begin planning the kind of instruction that will meet the needs of the students and avoid planning and teaching areas the students already have mastered.
The power of the data is in information provided by the reports, but it was the ease of collection made possible by Remark Office OMR that made the reports available in a timely and usable format. Together, Remark Office OMR and Microsoft Access database reporting system transformed data into useful educational information that moved instruction forward for children.
In my next article, "Remarkable Changes to Instruction," I will discuss how Remark Office OMR was moved into the formative assessment arena in the reading/writing area of language arts in the Title I schools.
Thomas Hays is a retired educator who served a 50,000 student school district in Maryland as assistant supervisor for accountability and data coordinator for compensatory education from 1996 through June 2007.
Institutional Research - Check out how Savannah College of Art and Design handles an internally developed survey given annually for 2500 respondents using Remark Office OMR and Remark Web Survey.
Institutional Effectiveness - Find out how Susan Evans was able to collect over 2000 responses and analyze data over a 9 day period using Remark Office OMR and Remark Web Survey.
Quality Assurance Program - St. Patrick’s College was undergoing an important quality assurance program involving both internal and external evaluators. Learn how they used Remark Office OMR to process their data.
Using Remark Office OMR for Course and Instructor Evaluations- Find out how Tiffany Martinez at the University of New Haven uses Remark Office OMR to process course evaluations in a timely, cost-effective manner.
Croatian University Uses Remark to Grade Thousands of Tests - Find out how the University of Rjeka corrects 50,000 tests every three weeks using Remark Office OMR.
Evaluate Impact of Professional Development Programs - Learn how Rebekah Nix was able to replace using transparent answer keys and reduce costs with Remark Office OMR.
Grading Biology Exams at a Large State University - See how Michael Wagenheim used Remark Office OMR to scan over 12,000 multiple choice biology tests per year and save money.
No such thing as a perfect form - Margaret Rutherford at Victoria ISD using Remark Office OMR to collect data for a grant.
Grading Assessments for Title IIB Grants - Learn how Capital Area Intermediate Assessments using Remark Office OMR to collect and analyze data from a variety of assessments, surveys, and evaluations under very tight deadlines.
Collect and Analyze Test Data for Title I - See how Tom Hays reduced the amount of time to collect data for Title I using Remark Office OMR.
Remarkable Changes to Instruction - After Tom Hays proved the feasibility and usability of Remark Office OMR for collecting data, his next challenge was to improve instruction in the elementary schools.
Remarkable Addition to Data Collection - Tom Hays discusses the remarkable addition to the data collection efforts: Evaluation of the kindergarten program through two assessments.
The Beginning of a Remarkable Relationship - The beginning of Tom Hays with Remark Office OMR.
Scanning Technology Conquers Common Testing Dilemmas - See how Dr. Bill Palya, Professor of Psychology at Jacksonville State University used Remark Classic OMR to enhance teaching and learning by enabling more frequent testing, improving test result analysis, and decreasing a student's ability to cheat.
How a Leading Hospital Uses Remark Office OMR® for Regulatory Compliance - Stephanie P provides overall management for the Patient Care Services Performance Improvement program that includes seven Quality Indicators. Patient data is analyzed and benchmarked against other hospitals. Reports are then provided to internal management and outside regulatory agencies.
Gravic's Contribution to Volunteerism - The challenge was to collect data from thousands of evaluation forms completed by participants in the Presidential Summit on Volunteerism held in Philadelphia.
Little League Assessments - Creating fair and balanced little league teams is a very complex process. Glenmoore Eagle Little League strives to create teams with a 50/50 win-to-loss ratio, making the season enjoyable for all the children and parents. In the past this goal was achieved by manually entering talent ratings of every child into a spreadsheet. This was a painstakingly slow process, so three years ago they started using Remark Office OMR® to scan and process player evaluation forms saving an enormous amount of time and effort for the volunteers who perform these functions.
Football Pool - The End Zone Bar & Grill is a sports bar that runs both a weekly and an annual NFL football pool. Employees had to manually check the weekly sheets, then sort and enter data from these sheets to keep a tally for the yearly pool. This process was enormously time-consuming, as there were 500-600 sheets per week to be processed.
