Dear Department for Work and Pensions,

Your 2021-22 accounts (https://www.gov.uk/government/publicatio...), published on 7 July 2022, refer at paragraph 48 of the Report of the Comptroller and Auditor General (“the report”) to a “risk model to detect fraud in Universal Credit advances claims” (“the model”).

The model uses a machine learning algorithm to conduct an “analysis” of “information from historical fraud cases” and “predict which cases are likely to be fraudulent in the future”. The Department states that “cases scored as potentially fraudulent by the model are flagged to caseworkers, who then prioritise the review and processing of such cases accordingly".

Please provide the following information on the model:
(1) Who developed and tested the model?
a) Were any pilot studies of the model conducted? If so, who conducted them? Please send any reports detailing the results from the pilot.
b) How often is the model updated?
c) What triggers a model update?
d) If the department does maintenance and updates internally, please disclose the handbook and/or guidance given to data scientists.

(2) The dataset(s) used to train the model.
a) The list of input features and feature importance.
b) The training and test set.
i. In case these data cannot be provided: detailed information on how the train and test data was collected
ii. Distributions for each of the input features as well as distributions of the labels (fraud/no fraud or referral no/referral)
c) The source code and the trained model.
d) Confusion matrices (i.e., number or share of TP, FP, TN, FN) for the test data based on true and predicted labels for each protected characteristic.

(3) Is the model a regression or classification model?
a) If it is a regression based model, is there a threshold above which a case is referred?

In the event that you determine some of the information I have requested to be exempt from disclosure, please redact exempt information with black boxes, instead of snipping or excerpting, and please state which category of exemption you believe applies to the information.

If it is not possible to provide the information requested due to the cost of compliance limit identified in s.12 FOIA, please prioritise information relating to the dataset(s) used to train the model.

Yours faithfully,

Mia Leslie

Dear Department for Work and Pensions,

Please pass this on to the person who conducts Freedom of Information reviews.

I am writing to request an internal review of Department for Work and Pensions's handling of my FOI request 'Universal Credit advance claims - fraud risk model'.

I made my request on 24 October 2022 and I received a substantive response on 21 November 2022.
My request concerned the reference made in your 2021-22 accounts (https://www.gov.uk/government/publicatio...), published on 7 July 2022, at paragraph 48 of the Report of the Comptroller and Auditor General (“the report”) to a “risk model to detect fraud in Universal Credit advances claims” (“the model”), about which I asked you to provide the following information:

1. Who developed and tested the model?
a. Were any pilot studies of the model conducted? If so, who conducted them? Please send any reports detailing the results from the pilot.
b. How often is the model updated?
c. What triggers a model update?
d. If the department does maintenance and updates internally, please disclose the handbook and/or guidance given to data scientists.

2. The dataset(s) used to train the model.
a. The list of input features and feature importance.
b. The training and test set.
i. In case these data cannot be provided: detailed information on how the train and test data was collected
ii. Distributions for each of the input features as well as distributions of the labels (fraud/no fraud or referral no/referral)
c. The source code and the trained model.
d. Confusion matrices (i.e., number or share of TP, FP, TN, FN) for the test data based on true and predicted labels for each protected characteristic.

3. Is the model a regression or classification model?
a. If it is a regression based model, is there a threshold above which a case is referred?

You provided answers to parts 1,1(b) and 3(a) of the request, partially provided an answer to 1(a) but refused to give further details, and did not give a substantive response to parts 1(c) and 1(d) and 2(a),(b),(c) and (d) on the basis of section 31.
· You disclosed that DWP staff developed and tested the model
· You confirmed that pilot studies of the model were carried out by DWP staff
· You disclosed that the model is “updated when DWP deems it necessary such as due to a reduction in effective- ness or a material change in the UC service”

In response to part 1(a) of the request you said you would not give further details of the models or the pilots; in response to parts 1(c) and 1(d) you refused to disclose details of what triggers a model update, guidance given to data scientists or records of the maintenance and updates; and in response to parts 2(a),(b),(c) and (d) of the request, you refused to disclose details of your models input features, feature importance or training datasets. You reason for refusing disclosure was the same for all parts of the request, namely that disclosure would “compromise the effectiveness of [your] response to fraud and [you] would therefore withhold any specific details of such work on the basis of the provisions contained in Section 31 of the Freedom of Information Act (“the Act”), which covers the prevention of crime”.

You acknowledged that there is a legitimate public interest in ensuring the Department gathers and uses information legitimately to check accuracy and eligibility in the award and payment of benefits, but nevertheless, maintained that providing the details requested would “enable a perpetrator to understand” your services the way your IT systems work, as well as where and how you gather information.
You assert that this would enable an offender to “make false claims to benefit, divert public funds, affect the way the government pays benefits to claimants or collects taxes, and could otherwise compromise the provision of essential public services”, which is “not in the public interest”.

I request that you review the decision to refuse disclosure, specifically the determination that prejudice would occur as a result of such disclosure, and the assessment of the public interest test.

First, ICO Decision Notice FS50286261 (available here: https://ico.org.uk/media/action-weve-tak...) emphasised that the onus is on the public authority to explain how purported prejudice would arise and why it is likely to occur.

The exemption you have applied requires you to conduct a prejudice test to demonstrate that there would be a ‘causal link’ between the release of the requested information, and prejudice to the prevention of crime. You have explained what prejudice you believe would occur but have not given an adequate explanation of why such prejudice is likely to occur as a result of the disclosure of the specific information requested. ICO guidance advises that you must then go on to determine the likelihood of this prejudice arising were the information released. ICO advises on the meaning of 'would or would be likely to' (available here: https://ico.org.uk/media/for-organisatio...). Further, the same ICO guidance states that when "[d]eciding whether the prejudice would occur or is only likely to occur is important. In this context the term “would prejudice” means that it has to be more probable than not that the prejudice would occur”.

In your consideration of the likelihood of such prejudice occurring, I ask you to consider the likelihood of a possible perpetrator being able to ‘game’ this model or prejudice the prevention of crime in the way you have articulated. If the input features are immutable characteristics, they likely cannot be affected by a perpetrator (for example age or gender) and, even where input featues are not immutable characteristics, it would be incredibly hard for a non-technical person to understand what exactly they would need to do in order to affect the outcome of the model due to the complex ways in which variables interact with one another in a model.
Please review your decision, and if you reach the same decision, please provide details as to how you identified the likelihood of the above prejudice arising.

Second, the exemption under section 31 is qualified and therefore you must weigh the public interest in maintaining the exemption against the public interest in disclosure. You acknowledged that there is a legitimate public interest in ensuring the Department gathers and uses information legitimately to check accuracy and eligibility in the award and payment of benefits. You said that enabling an offender to make false claims to benefit, divert public funds, affect the way the government pays benefits to claimants or collects taxes, and comprising the provision of essential public services is not in the public interest.

However, you have not weighed the public interest in disclosure against the public interest in maintaining the exemption. You must consider the relative weight of the arguments for and against disclosure, which as you may know, can be affected by inter alia the likelihood and severity of any prejudice, how far the requested information will help public understanding, and whether similar information is already in the public domain.

As articulated above, you have not provided a detailed analysis of the likelihood and severity of any prejudice. In relation to how far the requested information will help public understanding, and whether similar information is already in the public domain, I would like to draw your attention to the fact that very little information regarding this model is in the public domain. As a result, there is very low public understanding of the development, operation, and impact of this model on the public. The lack of transparency surrounding this model is an effective bar to public assessment of whether there is good decision-making by the DWP in relation to the identification of fraud in Universal Credit claims.

There is public interest in transparency and accountability to safeguard democratic processes, in ensuring justice and fair treatment for all, and in upholding standards of integrity. The Justice and Home Affairs Committee’s report on new technologies and the application of the law (30 March 2022, HL Paper 180) (available here: https://committees.parliament.uk/publica...) highlighted that “[w]ithout transparency, there can not only be no scrutiny, but no accountability for when things go wrong.” (pp.3-4). The report also notes that “[t]ransparency, namely public disclosure of what technology is currently being used, where, and for what purpose, is key to enable the sound deployment of technological solutions.” (p.39, para. 90).

There is a public interest in fully understanding the reasons for decisions. The model is used in the process of deciding whose Universal Credit claims are deemed to “look fraudulent”. The determination that a claim fits the criteria of “looking fraudulent” can lead to benefits being suspended whilst investigations are carried out. There is public interest in the public knowing information about a model that supports this important decision making process, that can have a significant impact on the lives of individuals. In Cabinet Office and Christopher Lamb v Information Commissioner (available here: EA/2008/0024 and 0029) the Information Tribunal said at paragraph 82 “…the majority considers that the value of disclosure lies in the opportunity it provides for the public to make up its own mind on the effectiveness of the decision-making process in context”.

I am aware that guidance from the Information Commissioner, in ‘The Guide to Freedom of Information’, says that internal reviews should be completed within 20 working days, or in exceptional circumstances 40 working days should be allowed.

I look forward to receiving your response in line with the Commissioner’s guidance.

A full history of my FOI request and all correspondence is available on the Internet at this address: https://www.whatdotheyknow.com/request/u...

Yours faithfully,
Mia Leslie