The National Pupil Database (NPD) is a database controlled by the Department for Education in England, based on multiple data collections from individuals age 2–21 in state funded education and higher education. Data are matched using pupil names, dates of birth and other personal and school characteristics, including special educational needs, disability, and indicators for free school meals, a child in care, and families in the armed forces. Personal details are linked to pupils' attainment and exam results over a lifetime school attendance.
In October 2018 the database contained over 21 million individual named pupil records. It is deemed by the department to be "one of the richest education datasets in the world". This is just one of the distributed datasets that the Department for Education controls, and separate from the further Individualised Learner Record (ILR) in the Learning Records Service, for example.
Schools use Management Information Systems (MIS) to collect and analyse pupil level information at local level. Data from these systems are used to complete the termly school census returns provided to Local Authorities (regional) or directly to the Department for Education (national) three times a year. The National Pupil Database has expanded in its scope of the items collected, and from children of a wider age range over time. Data once stored in the National Pupil Database, are never deleted.
The Higher Education Statistics Agency passes students' personal confidential data collected from universities to the Department for Education, where it is linked to individuals' school records in the National Pupil Database, expanding the lifetime record for millions of people that the department retains indefinitely.
Scope of the NPD The National Pupil Database covers only pupils in state (or partially state-funded) schools in England. However similar systems operate across the rest of the United Kingdom.
For Wales, the Welsh Government holds pupil level data back to 2004.Welsh data For Scotland, the Scottish Government holds electronic records for children in Scotland back to 2002 and allows third party access to this data.Scottish data For Northern Ireland, data is available from approximately 1,200 schools, 400 pre-schools and individual level records for over 300,000 pupils each year. The Northern Ireland Schools Census includes data going back to 1990.Northern Ireland data
Sources of pupil data in the NPD Details of all data sources contained within the linked set of data which form the National Pupil Database, and the coverage of children within each source.
Data Source AgesSchool Census/PLASC 2-18PRU Census [historic to 2013] 2-18Early Years Census 2-4Alternative Provision Census 2-18Early Years Foundation Stage Profile 2-4Reception Baseline Assessment 4-5Year 1 Phonics Test 5Key Stage 1 SATs 6-7Multiplication Times Tables Test 8-9Key Stage 2 SATs 10-11Key Stage 4 Awarding Body data 14-21Key Stage 4 Achievement & Attainment Tables data 15-16Key Stage 5 Awarding Body data 14-21Key Stage 5 Achievement & Attainment Tables data 16-18Individual Learner Records (ILR) 14-21HESA data 17-21Children Looked After 0-18Children In Need unborn-18PLAMS Post-16 Learning Aims 16-18NCCIS National Client Caseload Information 15–24Independent Specialist Providers (ISP)
Data types held The pupil level data are personal confidential data which include sensitive personal data as defined by the Data Protection Act 1998. The National Pupil Database contains:
Identifiers: the pupil, school and local authority identifiers. With effect from 2010/11, the pupil's UPN (unique pupil number) remained consistent throughout their time in school and remain permanently on the record. Fixed pupil characteristics which are expected to stay the same, such as date of birth. However some characteristics which may be expected to be static, for example ethnicity, may be inconsistent within an individual record. Ethnicity was found in 2016 to have been ascribed by school staff in some counties, even to the point of overriding parental and pupil choices to refuse to provide the optional information as recommended to schools by Brighton and Hove City Council ahead of the autumn school census. This option was changed in system providers' design to remove fields 'ascribed by' as a result. Time-varying pupil characteristics: these may change in each sweep of the census because they describe the circumstances of the pupil at that point in time. Key stage test results and other attainment data: data at the ages of five, Early years foundation stage profile or (FSP) seven (KS1), eleven (KS2) 14 (KS3), 16 (KS4 or GCSE)and 18 (KS5). School type characteristics: these describe the school the pupil attends at each sweep of the census. School staff data is not included in this census for the National Pupil Database, but through the separate School Workforce Census. Workforce personal data includes identifiers such as names and National Insurance numbers and characteristics such as ethnic group, employment contracts and remuneration details, qualifications and absence information.
There are about 400 possible variables to collect on individual pupils. The full national code sets of all the items of data that can be collected on individual children can be downloaded from the Department for Education, listed in the common basic data set (CBDS), including health and SEND (special educational needs and disability).
Use of pupil level data For uses of Key Stage attainment datasets and School Census dataset see also England: School Census. Raw pupil level personal data are held in the Department for Education National Pupil Database (NPD). The linked datasets contain data which are identifying, and too sensitive or disclosive to be published, although these data are given out to third parties in raw form.
David Cameron announced in 2011, the government would be "opening up access to anonymised data from the National Pupil Database […]." This was an expansion to other third parties, since these data had already been used for many years and extensively by academic public interest researchers.
Since 2012, Secretary of State has had powers to share raw data from National Pupil Database under terms and conditions with named bodies and third parties who for the "purpose of promoting the education or well-being of children in England are conducting research or analysis, producing statistics, or providing information, advice or guidance", and who meet the Approved Persons criteria of the 2009 Prescribed Persons Act, updated in 2012/13.
The data when released, however are not anonymised, but are sensitive and identifying. "According to centrally held records at the time of writing, from August 2012 to 20 December 2017, 919 data shares containing sensitive, personal or confidential data at pupil level have been approved for release from the National Pupil Database. For the purpose of this answer, we have assumed the term sensitive, personal or confidential uses of information to be data shares classified as either Tier 1 or Tier 2 as set out in the National Pupil Database area on GOV.UK. [In addition] There were 95 data shares approved between March 2012 and this classification system being introduced."
In a presentation to the NPD User group in September 2016, the Director of the DfE Data Modernisation group acknowledged the release of sensitive data: "People are accessing sensitive data, but only to then aggregate. The access to sensitive data is a means to an end to produce the higher level findings."
The data items for release are classed into four tiers by the Department for Education, as described in the NPD User Guide. Following the change of legislation, releases of the data since 2012 from the Department for Education to third parties have not been anonymous, but have been of identifiable and highly sensitive (Tier 1), identifiable and sensitive (Tier 2), aggregated but may be identifying due to small numbers (Tier 3) and identifying non-sensitive items (Tier 4). Raw, closed data are released on a regular basis to third parties, and the majority of releases are of Tier 1 and 2 data.
A list of completed National Pupil Database Third Party Requests and those in the pipeline, are published on a quarterly retrospective basis.
Government uses of the data are based on a model of data sharing, passing raw data from one location to another, which is viewed by some as 'obsolete'. Intra departmental transfers of data include to the Cabinet Office for preparation of Electoral Registration Transformation work in 2013, to match participant data in the National Citizen Service, and for use in the Troubled Families programme, as well as arm's length bodies such as NHS Digital for a survey "What About Youth" mailed home to 300,000 15 year olds in 2014. Not all government uses of the data are recorded in the Third Party Release Register, such as internal use. The volume of Police and Home Office use first made public through Freedom of Information requests in 2016, were first officially published by the department, in the Third Party Release Register in December 2017, under "External Organisation Data Shares". Police requests were only documented going as far back as July 2015. This omits police access to records before this date, as noted in a ministerial correction (HCWS272) made by Nick Gibb, Minister of State for School Standards, on the numbers of pupils data released to the Home Office and police. "Information supplied by the Data Modernisation Division of the DfE has been identified as containing incorrect facts in the response provided to Parliamentary Questions concerning the volume of children's records passed onto the police and the Home Office (PQ48634, PQ48635 and PQ52645) and in figures quoted during a House of Lords Debate on the 31 of October 2016 on the Education (Pupil Information) (England) (Miscellaneous Amendments) Regulations 2016."
Of the documented 887 requests for identifiable data that have been through the DMAP request process in March 2012 – December 2016, only 29 have been for aggregated data, according to analysis by the NGO defenddigitalme. There were 15 rejected applications between March 2012 and September 2016, including a request "by mistake" from the Ministry of Defence to target its messaging for recruitment marketing. Approved uses include identifying and sensitive data released to Fleet Street papers, "to pick interesting cases/groups of students," and about 60% of applications approved (as distinct from volume of data used) for identifying and sensitive, pupil level data, were from think tanks, charities, and commercial companies.
The Telegraph newspaper was granted identifying and sensitive data in 2013, for all pupils in the KS2, KS4 and KS5 cohorts for the years 2008–2012.
Academic uses of school census data make up about 40% of the requests for identifying, pupil level data, processed through and approved by the DMAP process. The raw data are sent to the requestor's own location. There is no charge made for fulfilling requests. "DfE does not charge for data (and has not since the NPD process began), nor does DfE charge for the processing and delivery of extracts to customers."
There is however no transparency of the volume of how many children's data have been given away in approved uses either, because "the Department does not maintain records of the number of children included in historic data extracts." (PQ109065)