What it means to standardise examination

Fred Matiang'i (left), the Cabinet Secretary for Education, Science and Technology, with Prof George Magoha, the chairman of the Kenya National Examinations Council. PHOTO | WILLIAM OERI | NATION MEDIA GROUP

What you need to know:

  • The council may not be compelled to standardise the results, but if that has been the tradition, then a sudden shift needs to be explained to all the stakeholders in the education sector so that none considers the other with suspicion and malice.

  • If the previous results were "massaged" before being released, the council should come out clearly on what massaging was all about, why it was necessary, for how long it has been a practice and state categorically that from now henceforth, there will be no massaging of results and that students earn grades commensurate with their effort.

The 2016 Kenya Certificate of Secondary Education examination results have continued to elicit mixed reactions from different stakeholders in the education sector.

Kenya National Union of Teachers (Knut) has demanded that the results for all the 574,125 candidates be recalled immediately.

Knut says due process was not followed in marking and releasing of the results. The union’s leadership has argued that the results released last month do not reflect the true performance of the candidates, and have cited clear breaches of marking processes that were overlooked by the Kenya National Examination Council (Knec).

When he announced the results in Mombasa, Education Cabinet Secretary Fred Matiang’i termed them “credible and honest”.

In the letter to the Clerk of the National Assembly Justin Bundi and Jeremiah Nyegenye of the Senate, Knut demands immediate recall of KCSE results and that due process of “moderation and grading” be followed. The union argues that the examination was not moderated during marking, claims the marking scheme was not discussed and adopted by the examiners.

The union further claims that one uniform grading system was applied in all subjects, which is a disaster in itself and an action that must be reversed. It further argues that this has disadvantaged the mathematics and science students thereby disadvantaging male candidates whom, traditionally in many schools, opt to take three sciences and mathematics.

FEW UNDERSTAND

Examination standardisation, however, is a process that very few people understand, hence the resultant confusion. Examination standardisation is a statistical process that is specifically designed to remove variable elements from test scores and allow the candidates to be compared equally. In other words, it is a way of giving equal value to the results of each test, regardless of the number of questions and the time allocated for completing them. For instance, an A in Mathematics cannot have equal value with an A, say, in religious education.

Standardised marks are more useful measures than raw scores (the number of questions answered correctly). This is done world over for three main reasons:

First, examination standardisation places the candidate’s scores on a readily understandable scale. Test scores are not made readily understandable by just converting them to percentages, because percentage on its own is not related to the average score of all the candidates or on how spread out their scores are. On the other hand, standardised scores are related to both these statistics.

Tests are usually standardised so that the average, nationally standardised score automatically comes out as 100, irrespective of the difficulty of the test and so it is easy to see whether a candidate is above or below the national average.

'STANDARDISED DEVIATION'

The measure of the spread of scores is called the “standardised deviation” and this is usually set to 15 for educational attainment and ability tests, and for many occupational tests. This means that irrespective of the difficulty of the test, about 68 per cent of the candidates in the national sample will have a standardised score within 15 points of the average ( between 85 and 115), and about 96 per cent will have a standardised score within two standard deviations (30 points) of the average (between 70 and 130). These examples come from a frequency distribution known as “the normal distribution”.

In England, for instance, examination standardisation is done in educational tests so that an allowance can made for the different age of the pupils. Almost invariably, in ability tests taken in the primary and early secondary years, older pupils achieve slightly higher raw scores than younger pupils.

However, standardised scores are derived in such a way that the ages of the pupils are taken into account by comparing a pupil only with others of the same age. An older pupil may in fact gain a higher raw score than the younger pupil, but have a lower standardised score. This is because the older pupil is being compared with other older pupils in the reference group and has a lower performance relative to his or her own age group.

MEANINGFULLY COMPARED

Examination standardisation is also done so that scores from more than one test can be meaningfully compared or added together. Standardised scores from most educational tests range from 70-140. Hence a student’s standing in say, mathematics and English, can be compared directly using standardised scores. It is not, therefore, meaningful to add together raw scores from tests of different length or difficulty.

These, and many other factors that are unknown to the public – the relative difficulty of the examination, the ability of the cohort, among others, allows for the calculation of a standardised score directly from a raw score.

In order to create a standardised score, a reference table called a “look-up table” is created for each test paper that is written and the table is specific to that test paper because it takes account of the difficulty of the paper.

The minimum standardised score is derived from the look-up table and the actual number will vary, depending on the average score of all those taking the test.

Dr Matiang’i expressly said there was no "massaging" of the results. "Massaging" in this context could be taken to mean standardisation.

BIGGER TASK

Recalling the results for yet another process could be water under the bridge. The bigger task, however, is now left to the teachers and Knec over future examination results. The teachers need to go back to the students with a different approach and make them understand that examination standardisation is not a process penned down in law or statutes of Knec. It is a deliberate attempt which the council can decide to undertake without coercion from any end. This will ensure the results do not elicit empathy with a feeling that the learners got the shorter end of the stick when future results are released.

Knec, on the other side, needs to weigh between getting into a completely new way of handling results or going as per the tradition or practice of the council. There are decisions that are guided by set practice or tradition of the institution and reference is often made to such decisions in current similar circumstances.

The council may not be compelled to standardise the results, but if that has been the tradition, then a sudden shift needs to be explained to all the stakeholders in the education sector so that none considers the other with suspicion and malice.

If the previous results were "massaged" before being released, the council should come out clearly on what massaging was all about, why it was necessary, for how long it has been a practice and state categorically that from now henceforth, there will be no massaging of results and that students earn grades commensurate with their effort.

Ken Ndori is an examiner of English.