Success Story 

 COMPASS Brings Advanced Student Testing Technology to Nigeria 

Abuja, Nigeria October 2006  Dozens of Nigerian scorers pored over heaps of pupil workbooks in a large conference hall--about the only space that could embrace a task as daunting as the scoring of 16,044 student achievement tests. When USAID and the Government of Nigeria launched the Community Participation for Action in the Social Sector (COMPASS) Project in mid-2004, the partnership took on the challenge of implementing the largest integrated intervention to date in Nigeria's education and health sectors. The project also heralded new possibilities for evaluating its impact on the quality of basic education in Nigeria through comparing performance trends in schools over the five-year life of the project.

According to UNICEF, most countries rely on academic examinations to determine student competency. Only a few measure student achievement to assess and improve their educational systems. Nigeria recently joined this league through the COMPASS Project. For the first time ever in Nigeria, rigorous scientific student achievement testing was conducted to assess the impact of interventions in improving math proficiency and English literacy. Carried out by COMPASS through EDUCAN Consultants in May - June 2006, the assessment took place in 170 schools in the three COMPASS states of Kano, Nasarawa and Lagos, and in the Federal Capital Territory. 32,088 pupils, 888 teachers, and 170 head teachers participated in the recent exercise, producing massive numbers of pupil workbooks and questionnaires for scoring.

The testing was conducted to assess how pupil literacy and numeracy performance in COMPASS-supported schools compare with control schools, and how performance trends compare from one grade to the next throughout the life of the project.

Launched in the second year of the project, the process involved developing testing tools (which collated 66,000 elements of concrete information and anecdotal notes during a pilot exercise), analyzing preliminary data captured during field-testing, and developing valid test items and testing methods to measure literacy and numeracy achievement of primary 1, 3, 4, 5, 6 pupils.

Explaining the school testing selection process, education testing expert Gilles Fournier of EDUCAN noted that 130 schools in Kano, Nasarawa and Lagos states, where COMPASS implements basic education interventions, participated in the assessment. He added that 40 schools from the Federal Capital Territory were randomly selected as control schools.

The process also involved five separate training events and preparation of a step-by-step checklist and scoring guide to be distributed to test administrators and scorers.

According to education testing expert Dr. Denis Mildon, also of EDUCAN, the consultants took several measures to assure the reliability and consistency of scoring by teams. Periodically scored papers were reviewed to assure that scorers were using the scoring guides in a consistent manner. said Mildon, “Overall reliability scores were calculated,” “and scoring patterns revealed an 85% level of consistency.”

To verify consistency of scoring from year to year, 200 pupil workbooks from each grade level were set aside for rescoring at the next scoring session.”

“Beyond implementing a variety of interventions, community-based education projects should be able to demonstrate how assessing literacy and numeracy can improve the quality of education,” said COMPASS Senior Education Advisor Semere Solomon.

The results of the student achievement testing will provide insight into the effectiveness of the COMPASS/GoN partnership which targets interactive radio instruction (IRI) as a primary component of the project, along with teacher training interventions, scholarships targeting Primary Education students, development and distribution of instructional materials, school health and nutrition interventions, and subgrants and capacity-building assistance to Parent-Teacher Associations.