Dubai teacher creates computer programme to grade GCSE short answers

The system is currently performing with 94 per cent accuracy, which is in line with human competency

Stuart Forsyth, a computer science teacher at Dubai College, created a computer programme that can correctly mark GCSE short-answer questions with around 94 per cent accuracy. All photos by Pawan Singh / The National

Mr Forsyth's computer programme is called Sama (short-answer marking agent), which could help free up more time for teachers.

Sama process answers from one word up to a few sentences.

Mr Forsyth says the system is currently performing with 94 per cent accuracy, which is in line with a competent human.

The data set has been created for the computer science curriculum, but Mr Forsyth hopes to expand the system for other subjects in the future, including science and mathematics.

A teacher in Dubai has created a computer programme that can evaluate GCSE short-answer questions with about 94 per cent accuracy.
Stuart Forsyth, a computer science teacher at Dubai College, said the system, which he named Sama (short-answer marking agent), could help free up more time for teachers.
He wrote the programme as part of his thesis for an MSc he is studying in Robotics at Middlesex University Dubai.
"I have done a lot of data collection from lessons taught at my school," he said.

I'm a teacher and the reality is, we don't like marking as it is very time consuming. To have an automated marking system in place would help to free up more time for things like lesson planning and pupil interaction

"I had an initial data set of 300 answers from around 40 different questions and I got the system trained up and developed based around those answers.
"I expanded that data set later on to test how effective the system actually was and I collected another 1,500 answers from pupils and the question set was expanded to just over 100 questions."
In most GCSE examinations, answer sets come in three forms: multiple choice, essay and short answers.
Sama has been developed to evaluate the latter and can process answers from one word up to a few sentences.
The answers typically ask candidates to state, describe, suggest or explain reasoning behind a question where there is an objective criterion for right and wrong.

Natural Language Processing or NLP enables computers to understand human language and meaning by analysing the syntax and semantics of sentences and comparing them to existing bodies of text, which are already understood.

Stuart Forsyth says the programme is currently being used as a revision tool. Pawan Singh / The National

Mr Forsyth, who has been teaching Years 7 to 13 at Dubai College for four years, said the system was currently performing with 94 per cent accuracy, which was in line with a competent human grader.
While the data set has been created for the computer science curriculum, he hopes to expand the system for other subjects in the future, including science and humanities.
A former systems analyst for the banking sector, Mr Forsyth said he came up with the idea to create the programme about six months ago.
"I'm a teacher and the reality is, we don't like marking as it is very time-consuming," he said.

"To have an automated marking system in place would help to free up more time for things like lesson planning and pupil interaction.
"My school has been so supportive throughout the whole process and as part of our teacher training we focus a lot on innovations in tech in education."
Currently, Sama has an accompanying website called SQuiz and pupils at the school are using the system as a revision tool.

In the future, with more tweaking and development, Mr Forsyth said it could be used as an examination or tutoring and assessment tool, not just a revision tool.

But said he would need venture capital investment to be able to work full time to achieve this before someone else did.