Objective
Most health literacy measures rely on subjective self-assessment. The Critical Thinking about Health Test is an objective measure that includes two multiple-choice questions (MCQs) for each of the nine Informed Health Choices Key Concepts included in the educational resources for secondary schools. The objective of this study was to determine cut-off scores for passing (the border between having and not having a basic understanding and the ability to apply the nine concepts) and mastery (the border between having mastered and not having mastered them).
Design
Using a combination of two widely used methods: Angoff’s and Nedelsky’s, a panel judged the likelihood that an individual on the border of passing and another on the border of having mastered the concepts would answer each MCQ correctly. The cut-off scores were determined by summing up the probability of answering each MCQ correctly. Their independent assessments were summarised and discussed. A nominal group technique was used to reach a consensus.
Setting
The study was conducted in secondary schools in East Africa.
Participants
The panel included eight individuals with 5 or more years’ experience in the following areas: evaluation of critical thinking interventions, curriculum development, teaching of lower secondary school and evidence-informed decision-making.
Results
The panel agreed that for a passing score, students had to answer 9 of the 18 questions and for a mastery score, 14 out of 18 questions correctly.
Conclusion
There was wide variation in the judgements made by individual panel members for many of the questions, but they quickly reached a consensus on the cut-off scores after discussions.