Pupil data: national pupil database numbers
Dear Department for Education,
Please could you confirm:
1. As a snapshot of data on the date of December 28th 2015: the total number of unique individuals' data stored in the NPD - (and separately the similar/equivalent higher education databases if the total number is higher) may I suggest this is ascertained by asking for the total quantity of unique pupil numbers (UPN) associated with individual pupil records in the National Pupil Database (please confirm that this is UPN not pupil reference matching numbers or list the quantity of both).
2. Provide the document / terms of reference that state the retention period of the records (please state the retention period simply in free text if no document exists).
3. Please provide the table of charges which an applicant must meet in order to receive data, for example standard or bespoke extracts of NPD data.(For reference please see the health data model equivalent: http://www.hscic.gov.uk/media/14839/DARS... )
I would be very grateful if you are able to provide these as soon as possible and within the maximum 20 days permitted under the FOI act (ref: https://ico.org.uk/media/for-organisatio...).
I appreciate your time and effort.
Thank you for your support.
Yours sincerely,
Jen Persson
Dear Ms Persson
Thank you for your recent enquiry. A reply will be sent to you as soon as possible. For information; the departmental standard for correspondence received is that responses should be sent within 20 working days as you are requesting information under the Freedom of Information Act 2000. Your correspondence has been allocated reference number 2015-0057632.
Thank you
Department for Education
Ministerial and Public Communications Division
Tel: 0370 000 2288
Dear Ms Persson,
Ref: 2015-0057632
Thank you for your request for information, which was received on 30/12/2015. The request has been dealt with under the Freedom of Information Act 2000 (“the Act”).
You asked the following questions:
Please could you confirm:
1. As a snapshot of data on the date of December 28th 2015: the total number of unique individuals' data stored in the NPD - (and separately the similar/equivalent higher education databases if the total number is higher) may I suggest this is ascertained by asking for the total quantity of unique pupil numbers (UPN) associated with individual pupil records in the National Pupil Database (please confirm that this is UPN not pupil reference matching numbers or list the quantity of both).
The total number of Unique Pupil Numbers (UPNs) in the NPD as at 28/12/2015 was 19,807,973. This covers pupil records since 2000.
The UPN is not collected in further and higher education.
2. Provide the document / terms of reference that state the retention period of the records (please state the retention period simply in free text if no document exists).
DfE does not have a documented retention period for the NPD records it holds. Initially, the data is used both for operational purposes (for example, to allocate funding or to enable effective operations in schools) and analytical purposes, after which it is retained over a longer timeframe for historical, statistical or research purposes. The Data Protection Act contains an exemption (s33(3)) that allows us to keep the data indefinitely for historical, statistical or research purposes.
A retention period is set for any extract of the NPD we provide to third parties and is usually between 6 months to 3 years depending on the aims of the project.
3. Please provide the table of charges which an applicant must meet in order to receive data, for example standard or bespoke extracts of NPD data.(For reference please see the health data model equivalent: http://www.hscic.gov.uk/media/14839/DARS...)
DfE does not charge for data (and has not since the NPD process began), nor does DfE charge for the processing and delivery of extracts to customers.
The information supplied to you continues to be protected by copyright. You are free to use it for your own purposes, including for private study and non-commercial research, and for any other purpose authorised by an exception in current copyright law. Documents (except photographs) can be also used in the UK without requiring permission for the purposes of news reporting. Any other re-use, for example commercial publication, would require the permission of the copyright holder.
Most documents produced by a government department or agency will be protected by Crown Copyright. Most Crown copyright information can be re-used under the Open Government Licence (http://www.nationalarchives.gov.uk/doc/o...). For information about the OGL and about re-using Crown Copyright information please see The National Archives website -http://www.nationalarchives.gov.uk/infor... .
Copyright in other documents may rest with a third party. For information about obtaining permission from a third party see the Intellectual Property Office’s website at www.ipo.gov.uk<http://www.ipo.gov.uk/>.
If you are unhappy with the way your request has been handled, you should make a complaint to the Department by writing to me within two calendar months of the date of this letter. Your complaint will be considered by an independent review panel, who were not involved in the original consideration of your request.
If you are not content with the outcome of your complaint to the Department, you may then contact the Information Commissioner’s Office.
our correspondence has been allocated reference number 2015-0057632. If you need to respond to us, please visit: https://www.education.gov.uk/contactus and quote your reference number.
As part of our commitment to improving the service we provide to our customers, we are interested in hearing your views and would welcome your comments via our website at: https://www.education.gov.uk/pcusurvey.
Yours sincerely
Martin Johnson
Web: https://www.education.gov.uk<https://www.education.gov.uk/>
Twitter: https://www.twitter.com/educationgovuk
Facebook: https://www.facebook.com/educationgovuk
Dear Department for Education,
thank you for the information received in part. Please may I ask for the missing information from my original request which is incomplete:
1. The total number of individuals in the higher education databases as this was not included, it only listed the NPD total: "(and separately the similar/equivalent higher education databases if the total number is higher)"
3. You note that the "DfE does not charge for data (and has not since the NPD process began), nor does DfE charge for the processing and delivery of extracts to customers."
May I suggest / request you provide me, for example the details of all charges, through provision of copies of the invoices or other notice for payment terms issued to the Telegraph Media Group (related to the February 20th, 2013 data request), Newscorp and News UK (26th September 2013 data request), or Pearson plc in 2013 (January 4, 2013 data request) and BBC (request 21st August 2014) or any party acting on their behalf.
A list of all charges that data recipients pay in order to receive data, may be broader in scope than only the processing and delivery of extracts to customers, or for data itself.
Thank you. I much appreciate your support.
Sincerely,
Jen Persson
Dear Ms Persson
Thank you for your recent enquiry. A reply will be sent to you as soon as possible. For information; the departmental standard for correspondence received is that responses should be sent within 20 working days as you are requesting information under the Freedom of Information Act 2000. Your correspondence has been allocated reference number 2016-0009307.
Thank you
Department for Education
Ministerial and Public Communications Division
Tel: 0370 000 2288
Dear Ms Persson,
Ref: 2016-0009307
Thank you for your request for information, as a follow-up to 2015-0052976, which was received on 14/02/2016. The request has been dealt with under the Freedom of Information Act 2000 (“the Act”).
You asked the following questions:
Thank you for the information received in part. Please may I ask for the missing information from my original request which is incomplete:
1. The total number of individuals in the higher education databases as this was not included, it only listed the NPD total: "(and separately the similar/equivalent higher education databases if the total number is higher)"
In your original question above, you went on to specifically ask for “the total quantity of unique pupil numbers (UPN) associated with individual pupil records in the National Pupil Database”, which was provided. There are no UPNs in higher education (HE) data as UPNs are only assigned by schools and so the HE part of the question, as asked, could not be answered.
In answering this, we have interpreted your request to cover pupils in both the further and higher education databases. As DfE doesn’t collect the UPNs of students in further or higher education, the number of individuals has been identified by counting the number of unique pupil matching references (PMRs), which the DfE allocates to individuals when their data is matched to the NPD. Using these PMRs, the total number of individuals in the further education (FE) and HE data held by the DfE as at 28th December 2015, in addition to the number of UPNs already identified in the NPD in the previous answer provided, was 1,422,659.
DfE only holds a subset of FE and HE data – if you require information on the total number of individuals held in further and higher education databases by Government you should approach the Department for Business, Innovation & Skills (BIS). DfE does not approve third party access to HE or FE data held by DfE, that approval is provided by BIS, the Skills and Funding Agency (SFA), the Higher Education Statistics Agency (HESA) and/or Higher Education Funding Council for England (HEFCE) as appropriate.
3. You note that the "DfE does not charge for data (and has not since the NPD process began), nor does DfE charge for the processing and delivery of extracts to customers."
May I suggest / request you provide me, for example the details of all charges, through provision of copies of the invoices or other notice for payment terms issued to the Telegraph Media Group (related to the February 20th, 2013 data request), Newscorp and News UK (26th September 2013 data request), or Pearson plc in 2013 (January 4, 2013 data request) and BBC (request 21st August 2014) or any party acting on their behalf.
A list of all charges that data recipients pay in order to receive data, may be broader in scope than only the processing and delivery of extracts to customers, or for data itself.
The DfE does not impose charges of any kind and does not receive any payment from data recipients in order for them to receive data – so do not have any such invoices or other notice for payments from these requestors. HESA, however, does impose charges and receives payment from data recipients for any HESA data the DfE provides. DfE is not involved in this payment and does hold any of these invoices or notices for payment.
The information supplied to you continues to be protected by copyright. You are free to use it for your own purposes, including for private study and non-commercial research, and for any other purpose authorised by an exception in current copyright law. Documents (except photographs) can be also used in the UK without requiring permission for the purposes of news reporting. Any other re-use, for example commercial publication, would require the permission of the copyright holder.
Most documents produced by a government department or agency will be protected by Crown Copyright. Most Crown copyright information can be re-used under the Open Government Licence (http://www.nationalarchives.gov.uk/doc/o...). For information about the OGL and about re-using Crown Copyright information please see The National Archives website -http://www.nationalarchives.gov.uk/infor... .
Copyright in other documents may rest with a third party. For information about obtaining permission from a third party see the Intellectual Property Office’s website at www.ipo.gov.uk<http://www.ipo.gov.uk/>.
If you are unhappy with the way your request has been handled, you should make a complaint to the Department by writing to me within two calendar months of the date of this letter. Your complaint will be considered by an independent review panel, who were not involved in the original consideration of your request.
If you are not content with the outcome of your complaint to the Department, you may then contact the Information Commissioner’s Office.
Yours sincerely,
Martin Johnson
Web: https://www.education.gov.uk<https://www.education.gov.uk/>
Twitter: https://www.twitter.com/educationgovuk
Facebook: https://www.facebook.com/educationgovuk
Dear Department for Education,
Thank you for the clarifications and additional information.
So to summarise clearly and make sure I understand the detail correctly, the additional number of individuals' data held at the DfE on Dec 28th, 2015 on top of the near 20 million in the NPD is a further near 1.5 million (1,422, 659) but this is only a subset of the higher education data held in national databases.
On charges, the NPD data on individuals' stored are given away for free whereas HESA (higher education) is charged for. (The price structure for parts of which is found online here: https://www.hesa.ac.uk/component/pubs/?t... )
And you confirmed that BIS is responsible for HESA data, and will have more information, in particular, the total number of individuals' records stored and sold.
Thank you for your support that completes this request.
Sincerely,
Jen Persson
Dear Ms Persson,
Ref: 2016-0013928
Thank you for your communication, as a follow-up to 2015-0009307, which was received on 21/03/2016.
I would like to offer my apologies for the delay in responding. The Department takes very seriously its obligations under the Act, including the requirement to respond within the statutory deadline of 20 working days. I am afraid that we have not been able to maintain this standard in this instance and while there may be considered to be mitigating circumstances, this delay is not acceptable and I am very sorry for the inconvenience which it may cause you.
You stated the following:
Thank you for the clarifications and additional information.
So to summarise clearly and make sure I understand the detail correctly, the additional number of individuals' data held at the DfE on Dec 28th, 2015 on top of the near 20 million in the NPD is a further near 1.5 million (1,422, 659) but this is only a subset of the higher education data held in national databases.
In answering your previous query we interpreted this to cover pupils in both the further and higher education databases held by DfE. As DfE doesn’t receive the UPNs of students in further or higher education, the number of individuals was identified by counting the number of unique pupil matching references (PMRs), which the DfE allocates to individuals when their data is matched to the NPD. As you indicated above we found that in addition to the pupils, in the previous answer (19,807,973), in the NPD the total number of additional individuals in the further and higher education data held by the DfE as at 28th December 2015, was 1,422,659.
It is correct to say that DfE only hold a subset of the further and higher education pupils held by BIS.
On charges, the NPD data on individuals' stored are given away for free whereas HESA (higher education) is charged for. (The price structure for parts of which is found online here:
https://www.hesa.ac.uk/component/pubs/?t...)
It is correct to say that, the DfE does not impose charges of any kind and does not receive any payment from data recipients in order for them to receive data. It should be noted, however, that there is a rigorous process in place for assessing requests for approval before any data can be provided.
HESA charges customers directly for the provision of HESA data to data recipients, including any HESA data provided by DfE on HESA’s behalf. HESA’s pricing policy can be found here:
https://www.hesa.ac.uk/bespoke-data-serv...
And you confirmed that BIS is responsible for HESA data, and will have more information, in particular, the total number of individuals' records stored and sold.
BIS is a joint data owner of the HESA data alongside HESA and Devolved Administrations. The main legal owner is HESA. BIS will hold more HESA and ILR information than that held by DfE and linked to the NPD.
Thank you for your support that completes this request.
Most documents produced by a government department or agency will be protected by Crown Copyright. Most Crown copyright information can be re-used under the Open Government Licence (http://www.nationalarchives.gov.uk/doc/o...). For information about the OGL and about re-using Crown Copyright information please see The National Archives website -http://www.nationalarchives.gov.uk/infor... .
Copyright in other documents may rest with a third party. For information about obtaining permission from a third party see the Intellectual Property Office’s website at www.ipo.gov.uk<http://www.ipo.gov.uk/>.
If you wish to complain about the handling of the FOI request sent to you on 14 March 2016, you should make a complaint to the Department by writing to me within two calendar months of the date of this letter. Your complaint will be considered by an independent review panel, who were not involved in the original consideration of your request.
If you are not content with the outcome of your complaint to the Department, you may then contact the Information Commissioner’s Office.
Yours sincerely,
Martin Johnson
Department for Education
We work to defend the right to FOI for everyone
Help us protect your right to hold public authorities to account. Donate and support our work.
Donate Now
Fiona Nicholson left an annotation ()
So the information isn't being sold on as a way of making money for DfE but is being provided without charge to support the move to academies, and to assist companies and consultants marketing to schools (who are able to show added value and like for like comparisons)?