Glasgow Times:

SCOTLAND’S poorest schools were four times more likely than private schools to have passing Higher grades changed to a fail by the moderation system behind this summer’s SQA results scandal, The Ferret can reveal.

Due to the coronavirus pandemic, exams were cancelled for the first time ever in Scotland, and the SQA was tasked with awarding grades using teacher estimates and a system of moderation.

But some 75,000 pupils had to be issued with new grades after the SQA moderated teacher estimates for children from poor areas more harshly than those in affluent postcodes.

Now, new analysis of the overall figures for both upgrades and downgrades has found large disparities across Scotland, with pupils in the wealthiest schools the primary beneficiaries each time.

According to one academic, the data shows that the system originally in place to award 2020 exam results was “biased against the teacher judgements of socially deprived students”.

The Ferret asked the SQA to release information for every secondary school in Scotland showing what percentage of Higher grades in each centre had been increased, what percentage had been decreased, and what percentage had been reduced from a pass to a fail.

READ MORE: Homelessness Crisis: Begging, beaten and robbed - the dangers of living rough on the streets​

This data was combined with information on the proportion of pupils in each school in receipt of free school meals, with private schools included as a separate category.

The analysis was carried out by Barry Black, a University of Glasgow researcher and associate of the Urban Big Data Centre, and reviewed by Professor Catherine Lido of Glasgow University.

The data shows that in schools where 40% or more of pupils receive free school meals, the average number of Higher grades moved from pass to fail was more than 20%.

In schools with 0-10% of pupils receiving free school meals the figure was 9%, and in private schools fell to just 5.1%.

Clear links to deprivation levels were also found when analysing the average rates at which all grades were either increased or decreased.

Schools with 0-10% of pupils in receipt of free school meals saw an average of 21.6% of Higher grades lowered during the moderation process, with this figure climbing to 39.9% in most deprived schools.

Private schools had the lowest number of average downward changes at 18.3%.

Only a small proportion of Higher grades were moved upwards by the SQA moderation process, with pupils in the wealthiest schools more likely to benefit than others.

Those in schools with 30-40% of pupils receiving free school meals fared worst, with an average of just 1.5% of Higher grades being increased from initial teacher estimates.

Although all downgraded results were eventually cancelled and reissued, increases in awards were not reversed.

In the recently published independent report into the 2020 exam results, the authors noted that the SQA had “no regret” about applying the moderation process, only that the appeals system “was not allowed to run its course”.

Black told The Ferret: “The model that was developed by the SQA engrained the inequalities which pupils at more deprived schools experience.

“These findings show in much greater detail what has been known since results day – that the SQA Alternative Certification Model negatively impacted those in the most deprived schools the most. They illustrate – in the starkest of terms – the inequality that is baked into our Scottish education system.

“It is important to remember that downgraded results were due to the past attainment of a pupils’ school and not any aspect of their individual performance.

“The goal now must be to learn from this period and re-imagine how we go about closing the educational attainment gap in Scotland – and never again formalise it.”

Black’s findings were echoed by Lindsay Paterson, professor of education policy at the University of Edinburgh. He said the research confirmed “beyond doubt” that the SQA’s model was biased against teacher judgements of socially deprived pupils.

Paterson added: “This careful research has confirmed beyond doubt that the SQA’s algorithm was, in effect, if not by intention, biased against the teacher judgements of socially deprived students by pulling down these teacher estimates.

“Although the results of that algorithm were abandoned by the Scottish Government a week after the results were announced, the legacy of distrust shown by the SQA towards teachers is likely to take many years to heal,” he said.

READ MORE: Scotland weather: Glasgow, Ayrshire and Argyll set for heavy rain as Yellow warning issued​

“The research also shows for the first time that the algorithm was biased against the socially deprived in a further way, by upgrading their estimates far less often than it upgraded the estimates for students in more advantaged circumstances.

“Because these upgrades were not reversed, the Scottish Government has in effect colluded in this aspect of the bias against students living in poverty.”

Dr Marina Shapira, of the University of Stirling, was part of the team which carried out the review into the handling of the 2020 results process.

She told The Ferret there are “two main reasons why the schools in areas with higher levels of socio-economic disadvantage were downgraded more than schools in more advantageous areas”.

She referred to the recently published report, explaining that schools in disadvantaged areas “historically have on average lower levels of attainment”, meaning that “high performers at historically low attaining schools would be disproportionately affected by moderation based on historical attainment records of schools, with their grades being arbitrarily downgraded”.

Shapira also pointed out that existing research shows that it is more difficult to accurately estimate the grades of lower attaining pupils, and therefore their grades are often overpredicted by teachers.

However, the SQA defended its system, criticising the report’s methodology.

A SPOKESPERSON said: “We do not believe the methodology here is robust. The overall position shows clearly that local authority schools saw proportionately more upgrades than independent schools through moderation.

“In local authority schools, 1.85% of entries were moderated upwards and in independent schools, it was 0.79%.”

The SQA said its certification model, including moderation of teacher estimates, was considered necessary to “maintain standards over time”. The body argued that “almost three quarters of estimates were unchanged”.

“Following the cancellation of the 2020 exams, the SQA was commissioned by the Scottish Government to develop an alternative certification model, based on teacher and lecturer estimates, to maintain standards over time.

“Given the estimates we received, we considered some moderation of teacher estimates was necessary, however almost three quarters of estimates were unchanged.

“Every school and college was in scope for moderation and all were treated in the same way. Data was anonymised throughout this process.

“The Equality Impact Assessment included statistical analysis of available data from 2016 onwards based on the Scottish Index of Multiple Deprivation (SIMD).

“It demonstrates that after moderation, there was an increase in attainment for those learners living in Scotland’s most deprived areas and a narrowing of the attainment gap between those in the most deprived and least deprived SIMD bandings compared with previous years.

“So what this analysis fails to show is that attainment levels rose and the attainment gap narrowed even after moderation.

“Following the ministerial direction issued to SQA on August 11, the results generated by this approach have been replaced by school and college estimates.

“The deputy first minister has set out the measures for the delivery of qualifications in the 2020-21 session and SQA is now focused on taking that work forward.”

A Scottish Government spokesperson said downgraded awards had been withdrawn and re-issued “based solely on teacher or lecturer judgement, or SQA moderated teacher and lecturer estimates where these were higher”.

“We accepted that the risk of undermining the value of qualifications was outweighed by a concern that young people, particularly from less advantaged backgrounds, may have been adversely affected,” the spokesperson said.

“We will look to learn lessons from the process to awarding qualifications this year that will help to inform any future actions.”

They also confirmed that next year’s exams results will “reflect an individual’s work rather than a statistical model or the past performance of their school.”

“Decisions on the 2021 exam diet have been informed by Professor [Mark] Priestley’s recommendations, widespread consultation by the SQA on the exams timetable and course assessment modifications, and by listening to the views of pupils, teachers, parents, education experts, local authorities and other stakeholders.

“Teacher and lecturer judgement will be at the centre of the approach to assessment of National 5 qualifications in 2020-21.

“Thus will reflect an individual’s work rather than a statistical model or the past performance of their school.”