Michael Leih
Associate Professor
School of Business
Trevecca Nazarene University
Nashville, Tennessee
The Challenge:
Trevecca Nazarene University is hosting a state wide high school math competition with 350 of the top high school students from the Nashville area. These students are competing in six different categories ranging from Algebra I to Calculus and Advanced Math Topics for over $20,000 in scholarship awards and bragging rights as the smartest math student for the year. The competition is scheduled to start at 9:30am. Students are given 80 minutes to complete the exam and then take 90 minute lunch break before the awards ceremony. During the 90 minute lunch break, the University must collect, sort and score each exam using a complex scoring formula. After the scoring is complete, each score must be sorted from highest to lowest in each exam category and the top 10 winners are announced during the awards ceremony with the top three winners in each category receiving significant scholarship awards to the University. The stakes are high and the time frame is short to complete an accurate scoring of each of the 350 exams.
We had considered using the University’s Scantron systems, but the system did not allow us to electronically transfer results into a database to manage the sorting and complex scoring of the exams. We needed a system that would provide an accurate and quick way to complete the final ranking of winners for the awards ceremony.
The Solution:
After a quick Internet search for OMR software, we discovered Remark Office OMR 8.0 by Gravic. We downloaded the free demo version of the software and began evaluating the product. The application appeared to have everything we needed. First, it allowed us to create custom answer sheets (using a standard word processor) with the exact number of response location to perfectly match each exam type. Second, it supported both OMR and barcode reading capability to accurately read student identification numbers and exam responses. Third, it supported a complex scoring formula so no additional calculations were needed after the application completed the scoring the exam. Finally, student information and scoring results could be quickly exported to a database where results could be sorted and student information transferred to an award certificate using a mail merge feature to quickly print out the awards to be handed out during the awards ceremony.
We began by developing our custom answer sheet and running a few tests. The application was simple to learn and the support team at Gravic responded quickly to questions we had about the Office OMR application. The support team also provided help in creating an answer sheet format which would provide us with the most accurate scoring process. Using a multi-function printer, we were able scan in our test set of answer sheets into a multi-page tiff format image file. Then we created an exam key and loaded it into the application before we read in the test exams. During our testing, the application worked perfectly. We were able to process over 100 test exams quickly and accurately. After a few more test we felt confident the application would perform well.
On the day of the competition, we handed out the custom answer sheets and the high school students completed their student information and took the exam. As we were collecting the answer sheets, we noticed the wide variety of bubble marks on the answer sheet. Most students had taken care to fill in the bubbles carefully, but many were of varying shading and completeness. We were a little nervous that software would not accurately score the exams with less than perfect markings. However, the process went perfectly. Within 45 minutes we were able to scan in and score the 350 exam answer sheets. During the processing of the exams, the software allowed us to actually view the scanned image of the exam to evaluate any double answer issues and correct them.
After getting the results from the Office OMR application, we manually graded the top 10 scores in each of the six exam categories. During the manual scoring we found 6 exams that appeared to not have been scored correctly by the software. However, after double checking our manual scoring we found the person doing the manual scoring made the mistake and in all but 1 of the test. The single answer sheet that was not scored correctly by the Office OMR software was filled in so lightly the scanner did not pick up several of the bubble marks. We discovered later that this can be corrected by adjusting the contrast settings on the scanner to better pick-up very light markings.
Overall, we are very pleased with Remark Office OMR software and believe it would have been very difficult to accurately score and rank all the student exams within the time limit we had.
Favorite Features
Image storage, Review mode, Complex scoring support, Data export.
Institutional Research - Check out how Savannah College of Art and Design handles an internally developed survey given annually for 2500 respondents using Remark Office OMR and Remark Web Survey.
Institutional Effectiveness - Find out how Susan Evans was able to collect over 2000 responses and analyze data over a 9 day period using Remark Office OMR and Remark Web Survey.
Quality Assurance Program - St. Patrick’s College was undergoing an important quality assurance program involving both internal and external evaluators. Learn how they used Remark Office OMR to process their data.
Using Remark Office OMR for Course and Instructor Evaluations- Find out how Tiffany Martinez at the University of New Haven uses Remark Office OMR to process course evaluations in a timely, cost-effective manner.
Croatian University Uses Remark to Grade Thousands of Tests - Find out how the University of Rjeka corrects 50,000 tests every three weeks using Remark Office OMR.
Evaluate Impact of Professional Development Programs - Learn how Rebekah Nix was able to replace using transparent answer keys and reduce costs with Remark Office OMR.
Grading Biology Exams at a Large State University - See how Michael Wagenheim used Remark Office OMR to scan over 12,000 multiple choice biology tests per year and save money.
No such thing as a perfect form - Margaret Rutherford at Victoria ISD using Remark Office OMR to collect data for a grant.
Grading Assessments for Title IIB Grants - Learn how Capital Area Intermediate Assessments using Remark Office OMR to collect and analyze data from a variety of assessments, surveys, and evaluations under very tight deadlines.
Collect and Analyze Test Data for Title I - See how Tom Hays reduced the amount of time to collect data for Title I using Remark Office OMR.
Remarkable Changes to Instruction - After Tom Hays proved the feasibility and usability of Remark Office OMR for collecting data, his next challenge was to improve instruction in the elementary schools.
Remarkable Addition to Data Collection - Tom Hays discusses the remarkable addition to the data collection efforts: Evaluation of the kindergarten program through two assessments.
The Beginning of a Remarkable Relationship - The beginning of Tom Hays with Remark Office OMR.
Scanning Technology Conquers Common Testing Dilemmas - See how Dr. Bill Palya, Professor of Psychology at Jacksonville State University used Remark Classic OMR to enhance teaching and learning by enabling more frequent testing, improving test result analysis, and decreasing a student's ability to cheat.
How a Leading Hospital Uses Remark Office OMR® for Regulatory Compliance - Stephanie P provides overall management for the Patient Care Services Performance Improvement program that includes seven Quality Indicators. Patient data is analyzed and benchmarked against other hospitals. Reports are then provided to internal management and outside regulatory agencies.
Gravic's Contribution to Volunteerism - The challenge was to collect data from thousands of evaluation forms completed by participants in the Presidential Summit on Volunteerism held in Philadelphia.
Little League Assessments - Creating fair and balanced little league teams is a very complex process. Glenmoore Eagle Little League strives to create teams with a 50/50 win-to-loss ratio, making the season enjoyable for all the children and parents. In the past this goal was achieved by manually entering talent ratings of every child into a spreadsheet. This was a painstakingly slow process, so three years ago they started using Remark Office OMR® to scan and process player evaluation forms saving an enormous amount of time and effort for the volunteers who perform these functions.
Football Pool - The End Zone Bar & Grill is a sports bar that runs both a weekly and an annual NFL football pool. Employees had to manually check the weekly sheets, then sort and enter data from these sheets to keep a tally for the yearly pool. This process was enormously time-consuming, as there were 500-600 sheets per week to be processed.
