Dear Queen Mary, University of London,

The PACE Trial protocol was published in 2007 and included a criteria for 'recovery' from CFS, [1] a secondary outcome which later underwent substantial changes. The paper on recovery rates, published in Psychological Medicine in February 2013, stated that: "We chose domains for defining recovery on the basis of the previous literature and the measures available from the trial. The thresholds defining our criteria for recovery on each domain were based either on population normal ranges, case definitions or trial entry criteria. We changed three of the thresholds for measuring recovery from our original protocol (White et al., 2007) before the analysis, as explained below." [2]

There has been confusion over the timing of these changes and whether they were all approved by the trial management group, the trial steering committee, the data monitoring and ethics committee, or whatever relevant trial oversight bodies can grant approval outside the 'trial management group' (which included the authors), before publication. This confusion has reached the level of parliamentary debate in the House of Lords, and the new thresholds have courted controversy because they overlap with trial eligibility criteria for severe chronic disabling fatigue. [3] Furthermore, the justification for changing the physical function threshold was apparently based on a misinterpretation i.e. a threshold of ≥85 points in SF-36 physical function does not exclude 'approximately half' of the 'general working age population' as asserted in the paper on recovery rates. [4]

Two of the threshold changes are identical to and/or appear to be based on a post-hoc analysis which was previously introduced in early 2011 during the Lancet's peer review stage of the publication process i.e. the 'normal range' in fatigue and physical function. [5][6] It was stated in the Lancet paper that: "The statistical analysis plan was finalised, including changes to the original protocol, and was approved by the trial steering committee and the data monitoring and ethics committee before outcome data were examined." As it was a post-hoc addition, the normal range was not included in the statistical analysis plan [7] and therefore it is unclear whether approval was given. Similarly, the paper on the recovery rates does not state that approval was given for the changes to the recovery thresholds. [2]

The statistical analysis plan states that it "maximises transparency, providing a record of all planned analyses"; "were approved by the Trial Steering Committee (Version 1.2 dated 2 May 2010) prior to database lock"; "according to ICH E9, the statistical analysis plan should be prespecified, completed after the protocol has been finalised but reviewed and possibly updated as a result of a blind review of the data carried out after the completion of data collection"; and that "This document details the presentation and analysis strategy for the principal paper(s) reporting results from the PACE Trial. It is intended that the results reported in these papers will follow the strategy set out here; subsequent papers of a more exploratory nature will not be bound by this strategy but will be expected to follow the broad principles laid down for the principal papers." [7]

As the statistical analysis plan did not include the recovery criteria or mention anything about changes to the recovery thresholds, the report on recovery is an example of a subsequent exploratory paper. It appears plausible that, while changes were made to the recovery thresholds before the analysis was done to determine the recovery rates for that particular paper, these changes were made some time after the general unblinding and analysis of outcomes data for previous principal papers. Therefore, I would like to request the following information to help the public clarify these issues unambiguously and conclusively.

Can you please confirm, deny, or clarify, the following:

a) Confirm or deny that the post-hoc normal range in fatigue and physical function was explicitly approved by the relevant trial oversight bodies before publication of the Lancet paper in February 2011. If it was approved, who approved it?

b) During the peer review stage of the Lancet paper, a reviewer suggested what is now known as the 'normal range'. [6] What did this reviewer suggest should be the normal range threshold in physical function? Did the reviewer explicitly suggest that it should be the mean minus 1 S.D. score of the general population (incorrectly described in the Lancet paper as a working age population [8]) from the ONS Omnibus Survey 1992 sample published by Bowling et al. in 1999? Or was it an open suggestion, such as establishing a threshold using any method and any normative dataset, therefore giving White et al. some freedom to establish this threshold as however they saw fit? What about for the normal range in fatigue? Please note that I am not requesting any personal information about the reviewer in question.

c) Confirm or deny that the change to the physical function threshold for recovery (from ≥85 to ≥60 points out of 100) was made after the normal range was first suggested and calculated during the Lancet's peer review process.

d) Approximate dates for when the three threshold changes (fatigue, physical function, CGI) were made to the recovery criteria, for when the authors were first unblinded to outcomes data, and if possible, for when the authors first conducted the main analyses of the primary outcomes for the Lancet paper.

e) Confirm or deny that any changes to the 2007 version of the trial protocol relating to the primary outcomes (fatigue and physical function) and the criteria for recovery was guided by any data produced from the trial itself, either blinded or unblinded. If so, please specify which changes were guided by trial data.

f) Confirm or deny that the relevant trial oversight bodies had explicitly approved all the changes made to the 'recovery' criteria prior to publication of the Psychological Medicine paper in February 2013. If approved, please specify which oversight body approved them. If some were approved but others were not, please specify which ones were approved or not approved.

References:

1. http://www.biomedcentral.com/1471-2377/7/6

2. http://www.ncbi.nlm.nih.gov/pmc/articles...

3. http://www.publications.parliament.uk/pa...

4. http://www.bmj.com/content/347/bmj.f5963...

5. http://www.ncbi.nlm.nih.gov/pmc/articles...

6. http://www.meactionuk.org.uk/whitereply....

7. http://www.trialsjournal.com/content/14/...

8. http://www.thelancet.com/journals/lancet...

Yours faithfully,

Mr Matthees.

Dear Queen Mary, University of London,

Please pass this on to the person who conducts Freedom of Information reviews.

I am writing to request an internal review of Queen Mary, University of London's handling of my FOI request 'Timing of changes to PACE Trial recovery criteria'.

No explanation or evidence was given for refusing this FOI request as "vexatious", so I cannot offer a specific counter-argument but will instead highlight the ICO's approach to assessing whether a FOI request is vexatious, as considering whether compliance would create a significant burden in terms of expense and distraction, whether the request is designed to cause disruption or annoyance, whether the request has the effect of harassing the public authority or its staff, whether the request can otherwise fairly be characterised as obsessive or manifestly unreasonable, and whether the request has any serious purpose or value.

http://ico.org.uk/foikb/FOIPolicyVexatio...

The ICO's specialist guide titled "Dealing with vexatious requests (section 14)" provides further details and discusses multiple indicators of vexatious requests: abusive or aggressive language, burden on the authority, personal grudges, unreasonable persistence, unfounded accusations, intransigence, frequent or overlapping requests, deliberate intention to cause annoyance, scattergun approach, disproportionate effort, no obvious intent to obtain information, futile requests, frivolous requests.

http://ico.org.uk/~/media/documents/libr...

I do not believe that this FOI request meets these indictors as listed above and described in the document. Although QMUL have no obligation to explain why a FOI request has been deemed "vexatious", the document states that public "authorities should aim to be as helpful as possible" and "the ICO considers it good practice to include the reasoning for the decision in the refusal notice".

The cut off point for evidence that a FOI request is vexatious is the set time limit (normally 20 working days) in which the public authority must respond to that FOI request. Prior to the present FOI request submitted 26 April 2014, I submitted a different FOI request on 24 March 2014 and QMUL subsequently issued a refusal notice on 22 April 2014. That would not be a valid reason to dismiss this present FOI request as vexatious. Submitting multiple FOI requests at one time is technically allowed, but to minimize the burden I waited until the time limit for the first FOI request had expired before submitting this second FOI request.

Section 14(1) of the FOIA can only be applied to the FOI request itself, not the individual who submits it, and it is concerned with the nature of the FOI request rather than the consequences of releasing the requested information. It is also highly unlikely that this FOI request would require more than £450 or 18 hours of staff time to fulfil. A public authority can combine the total cost for all FOI requests received from one person during a period of 60 days so long as they are requests for similar information, but the two FOI requests from myself are for clearly different information without significant overlap.

When reviewing complaints over how FOI requests deemed "vexatious" were handled, the ICO expects the public authority to bear in mind that they primarily look for evidence that the request would have an unjustified or disproportionate effect. Similarly, where the public authority believes context or history strengthens their argument that a FOI request was vexatious, the ICO expects the public authority to provide any relevant documentary evidence or background information to support this claim.

When doing research for this FOI request I noticed that QMUL (when involved with other FOI requests) have alleged that there is an organised hate campaign to harass researchers and unfairly discredit PACE. Whether or not such a campaign exists, it has also become a simplistic excuse or convenient stereotype to dismiss, avoid, or delegitimise, sincere criticisms and difficult questions, as extremist. Such an attitude would be bound to produce some false positives when classifying critical responses as harassment.

However this FOI request is not part of such a campaign, and the PACE trial should stand on its own scientific merit. Assessment of this merit or any criticisms and questions should include relevant behind the scenes context to shed light on any major, questionable, and possibly unapproved, changes to the trial protocol, so the public can be informed about what happened. These changes had the effect of making the tested therapies appear much more successful than they otherwise would have. Most people should be able to understand that using a recovery definition which partly overlaps with the entry criteria for severe chronic disabling fatigue is obviously problematic in a therapy trial promoted as a definitive study settling a major longterm controversy.

A series of circumstances have given rise to suggestions that changes to the recovery criteria were post hoc, introduced after the principal investigators were unblinded to trial outcomes data, and unapproved by the relevant trial oversight bodies who approved the original protocol. In addition to the changes to the recovery criteria described in the FOI request, the changes to the threshold of clinical significance or improvement to fatigue and/or physical function on an individual patient level were also described as post hoc in the 2011 Lancet paper. Due to the circumstances surrounding the 'normal range', questions have also been asked about whether the 'clinically useful difference' were conceived under similar circumstances as those which gave rise to the 'normal range'.

The Freedom of Information Act was designed to give individuals a greater right of access with the intention of making public bodies more transparent and accountable. This FOI request is not intended to harasses or burden QMUL but is a genuine attempt to answer important questions about the redefinition of recovery in the PACE trial in a way which leaves no room for ambiguity. My motivation is resolving and seeking the truth behind the PACE trial controversy for the benefit of the ME/CFS community. 

The important questions asked above would not be necessary if there was enough information in the public domain. Without such clarification and transparency, the controversy and confusion surrounding these issues are highly unlikely to resolve themselves and the reputation of the PACE trial within the ME/CFS community will remain tainted with significant doubts. Again, the confusion arising from the apparent discrepancies has reached the level of parliamentary debate, which reflects the scale and importance of this issue. Withholding the requested details will probably only serve to perpetuate the controversy and confusion.

As all the changes to the thresholds of improvement or recovery (for the primary efficacy measures of fatigue and physical function on an individual patient level) were post hoc, the public interest strongly favours knowing the requested details, about whether these controversial and apparently questionable/unjustified changes to the protocol of a large 'definitive' trial were conceived or introduced after the principal investigators were unblinded to trial outcomes data, and whether these changes were all approved by the relevant trial oversight bodies who approved the original protocol. Without such approval, it defeats the purpose of forming a trial oversight body in the first place and would have a significant impact on how the results should be interpreted.

The PACE trial cost UK taxpayers £5M to QMUL's obvious benefit. According to the document cited above, "Public authorities must keep in mind that meeting their underlying commitment to transparency and openness may involve absorbing a certain level of disruption and annoyance. [...] The public authority should be mindful to take into account the extent to which oversights on its own part might have contributed to that request being generated." I believe that when adequately informed about the relevant circumstances and judging objectively, most reasonable people would conclude that the purpose and value of this FOI request is enough to justify the (relatively minimal) impact on QMUL to answer the important questions I seek to resolve. This FOI request has an important value and serious purpose in terms of the objective public interest in the information sought.

QMUL have clearly stated on their website that: "The Freedom of Information Act aims to promote greater openness about the way public authorities operate and gives members of the public a right to access information held by public authorities. Queen Mary aims to comply fully with its obligations under the Act and to ensure that the service we provide for those wishing to gain access to information is simple, efficient and effective." I hope QMUL will come to realise the advantages of providing the requested information in order to help resolve this ongoing controversy to the benefit of themselves and the ME/CFS community.

http://www.qmul.ac.uk/about/collegeinfo/...

A full history of my FOI request and all correspondence is available on the Internet at this address: https://www.whatdotheyknow.com/request/t...

Yours sincerely,

Mr Matthees.

Dear QM FOI Enquiries,

To whomever handles internal reviews of FOI requests at QMUL,

On the 18th July 2014 (38 working days after the initial refusal notice) I asked for an internal review of the handling of "Timing of changes to PACE Trial recovery criteria", but have received no acknowledgment, answer, or explanation for the delay.

https://www.whatdotheyknow.com/request/t...

40 working days have elapsed since that date:

http://www.work-day.co.uk

The Information Commissioner’s Office recommends that public authorities carry out internal reviews within 20 working days:

http://ico.org.uk/for_the_public/officia...

Similarly, the ICO's guidance on conducting internal reviews states that:

"In view of all the above the Commissioner considers that a reasonable time for completing an internal review is 20 working days from the date of the request for review. There may be a small number of cases which involve exceptional circumstances where it may be reasonable to take longer. In those circumstances, the public authority should, as a matter of good practice, notify the requester and explain why more time is needed. In our view, in no case should the total time taken exceed 40 working days. In such cases we would expect a public authority to be able to demonstrate that it had commenced the review procedure promptly following receipt of the request for review and had actively worked on the review throughout that period."

https://www.whatdotheyknow.com/request/1...

I do not understand how a sincere request for important information which is relatively easy to disclose is "vexatious". In previous correspondence with QMUL I outlined the definitions of vexatious FOI requests and this does not meet those definitions.

In a recent paper, Professor White co-wrote: "Our deliberate policy, to help allay concerns about the trial, was to be as transparent as possible regarding what we did, while protecting medical confidentiality and our staff; this included publishing the protocol and the statistical analysis plan, and paying for open access to all publications."

http://pb.rcpsych.org/content/early/2014...

The statement of being "as transparent as possible regarding what we did" appears to be at odds with the current situation, where questions or concerns still remain over major changes to the trial protocol years after the fact, and where a sincere FOI request for unambiguous answers to conclusively resolve the confusion has been refused as "vexatious". I described the circumstances which resulted in this unfortunate situation, where apparent discrepancies remain and concerns have not been allayed, and asked detailed questions in order to get clear answers which cannot be disputed by others or cause further confusion.

The statistical analysis plan and the timing of events strongly suggest that the authors had a clear view of the distribution of fatigue and physical function scores long before the 'normal range' first appeared and later became the base criterion for 'recovery', among other changes to the recovery criteria which are not mentioned in the statistical analysis plan either, such as the CGI threshold. A concern among the patient community is that post-hoc changes have been made to the recovery criteria after the authors were unblinded to trial data, made without approval by the relevant trial oversight bodies, but presented as pre-determined outcomes or approved changes made before unbinding/analysis was done and without any influence from trial data.

I am not accusing the authors of deliberate wrongdoing, I seek clear answers on these issues, which are not available elsewhere. Fulfilling this FOI request should only take an email to and from whomever knows this information. The whatdotheyknow.com website was used to ensue that any answers would go on public record so there is no longer any confusion. These questions may be awkward, and if so, a ICO guideline on 'When can a request be considered vexatious or repeated?' states: "In the context of a dispute, a request may be a reasonable way to obtain new information not otherwise available to the individual. You should not use section 14 as an excuse to avoid awkward questions that have not yet been resolved satisfactorily."

https://www.whatdotheyknow.com/request/1...

The patient and research community have a right to know the answers to these questions. Please prevent this from dragging out longer than necessary, it has already been about 43 months since the Lancet publication and about 19 months since the Psychological Medicine publication. I hope QMUL uses this opportunity to engage and set the record straight. If QMUL are concerned about the impact that the answers may have, I welcome any relevant context to accompany them.

Yours sincerely,

Mr Matthees.