Statistical working paper
Following the rapid growth of academy chains after the Academies Act 2010, and repeated pressure from the Education Select Committee, the Department for Education (DfE) is taking a significant step towards rating the performance of academy chains and local authorities on an ongoing basis. Although there is a wide range of data available at school level through the school performance tables, at present, the DfE does not publish a summary of performance at academy chain or local authority level.
The DfE has published a statistical working paper on measuring the performance of schools within academy chains and local authorities. The DfE feels that publishing this data is a challenge because simple aggregations of attainment or other measures can lead to spurious conclusions about performance within an academy chain or local authority since they will often reflect the type of schools that have entered into the chain or that remain with a local authority.
An effective summary could simplify the assessment of progress, which in turn will help academy chains and local authorities understand their own performance, help parents make choices about schooling and inform the DfE’s decision-making. Indeed, the Education Select Committee recommended that the DfE should publish performance information about academies and free schools on a chain-by-chain basis as it was not satisfied the DfE had been able to hold chains sufficiently to account.
Academy chains vs local authorities?
The DfE concludes that academy chains should only ever be compared with other academy chains, and similarly local authorities should only be compared with other authorities. The DfE believes that academy chains and local authorities have very different roles and responsibilities and therefore are not directly comparable. Academy chains often take on underperforming local authority schools, so results will be poor from the start.
The DfE also concludes that, since overall performance relates to multiple factors such as pupil outcomes, financial management and quality of leadership, no single measure is likely to capture every element of performance or impact.
For the purposes of these new performance measures: an “academy chain” is a group of five or more academies in either a ‘sponsor-led’ chain or a ‘non-sponsor led’ chain comprising five or more academies in a Multi Academy Trust (“MAT”); and a “local authority” is a group of five or more schools that are community, voluntary aided, voluntary controlled or foundation schools.
Attainment vs value added?
The working paper considered the strengths and weaknesses of measures of attainment, e.g. the proportion of pupils who achieved five good GCSEs including English and mathematics, arguing that taken in aggregate in an evolving school system they can be a poor measure of the effectiveness of chains and local authorities. Attainment measures do not account for the pupil intake of a school; the school may be highly effective in terms of the progress that pupils make but still have attainment that is relatively low.
Value added is a measure of the performance of pupils relative to those with similar prior attainment nationally. The DfE feels that value added is a better measure of the effectiveness of schools in terms of the progress of pupils. However, there is an assumption that schools with the same value added have the same propensity to demonstrate improvement. Yet, there remains a range of characteristics over which the school has no control which may affect the propensity of a school to demonstrate improvement. To aid interpretation of the scores, these new measures will sit alongside a range of contextual information covering the number and types of school included and the average level of disadvantage, special educational needs and prior attainment.
The working paper proposes two measures that capture how schools within chains and within local authorities are currently performing and how that performance has changed over time.
1. Current school value added
This measure captures the average current value added scores within chains and local authorities. This measure is an average of value added scores within the chain or local authority weighted by number of pupils and the length of time the school has been with the academy chain or local authority.
2. Change in school value added within academy chains and local authorities
This measure captures the change in school value added scores between a baseline year and the current year in comparison with similar value added in the baseline year. The baseline year is taken as the last year as the predecessor school (if applicable) or five years ago, whichever is more recent.
The working paper includes scores for 100 local authorities and 20 academy chains.
What do you think?
While the strengths and weaknesses of value added and attainment as measures of performance are well known to the sector, the DfE believes that value added measures are fair. However, the working paper clearly acknowledges the flaw that two schools with the same value added score can have very different characteristics that may affect rates of improvement. Moreover, the working paper recognises that no measure can ever capture the range of individual circumstances in every school, academy chain or local authority. The proposal to include a range of contextual information with the scores will help to deal with this issue.
If you wish to provide views on the DfE’s working methodology, including any suggestions for alternative approaches, please send your comments to the following email address:firstname.lastname@example.org