Selected data on PACE Trial participants
Dear Queen Mary, University of London,
Previous FOI requests [1][2] have asked for the release of PACE Trial results according to the outcome measures laid out in the trial protocol published in 2007 but since abandoned, and for additional summary statistics on the trial participants or at least the subgroup classified as 'recovered' after the 52-week followup period. [3] These requests have been denied because the information was not held in final form and the calculations required to attain them from data that is held would supposedly exceed the limit of £450 (calculated as the estimated cost of one person spending 18 hours in determining whether the information is held, then locating, retrieving and extracting the information). [4] A few of the comments posted in response have raised doubts over whether acquiring the data and performing relatively simple calculations would really take over 18 hours to perform.
In order to help ease the burden of staff having to perform the required calculations themselves once the relevant data is located and retrieved, I would like to request the following selection of baseline and 52-week followup data on all 640 individual PACE Trial participants for which the data exists, in a spreadsheet or equivalent file with separate columns for each variable:
• SF-36 physical function scores (range 0-100 points) [baseline and 52-week followup];
• CFQ fatigue Likert scores (range 0-33 points) [baseline and 52-week followup];
• CFQ fatigue bimodal scores (range 0-11 points) [baseline and 52-week followup];
• Oxford criteria CFS caseness (does participant meet criteria, yes or no) [52-week followup only];
• Participant-rated CGI scores (range 1-7) [52-week followup only];
• Doctor-rated CGI scores (range 1-7) [52-week followup only];
• 6MWT walking distances (in meters) [baseline and 52-week followup];
• The group which each participant was allocated to after randomisation (i.e. either to APT, CBT, GET, or SMC).
If granted, please make sure that each individual row only contains values from the same participant, as is common practice for such data in spreadsheets, so that more than one variable can be analysed at a time. To clarify, I am requesting only 'anonymised' data, I am not requesting any information which can identify individual participants (not even the participant ID numbers if those are deemed to be inappropriate to include, so long as each individual row only contains values from the same participant).
1. https://www.whatdotheyknow.com/request/p...
2. https://www.whatdotheyknow.com/request/p...
3. https://www.whatdotheyknow.com/request/6...
4. http://ico.org.uk/for_organisations/free...
Yours faithfully,
Mr Matthees
FOI 2014/F73
Dear Mr. Matthees
Thank you for your email of 24^th March requesting data from the PACE
trial.
The request you have made is in the same vein as previous requests which
Queen Mary University of London has refused and which the Information
Commissioner has subsequently upheld (see Decision Notices FS50514995 and
FS50484575 available by searching at
[1]http://search.ico.org.uk/ico/search/deci...).
As with those requests and the ones you have cited in your correspondence,
you are similarly requesting data derived from each individual who took
part in the PACE trial and that this be presented across a number of
variables at the individual-participant level. As such, I am afraid that
we cannot supply this to you as it is exempt under s.40(2) and s.41(1) of
the Freedom of Information Act 2000 (FOIA) as the information consists of
sensitive medical data provided in confidence by participants in the PACE
trial.
The information you have requested consists of data derived from each
living individual who took part in the PACE trial. Although this was a
fairly large trial in medical research terms, given the limited pool of
eligible potential ME/CFS participants and the nature of the data sets, it
is unlikely that this data could be sufficiently pseudonymised so as to
render it sufficiently certain that, with its disclosure to the public,
the data subjects would not be identified or identifiable from this data
or from other data that could fairly readily come into your, or someone
else’s, possession. This would mean disclosing sensitive personal data of
participants. Moreover, the information was collected in confidence under
terms which, were they to be broken, would deter people from participating
in clinical trials and breach the doctors’ obligations in respect of the
duty of confidentiality they owe. Queen Mary has an ethical duty of care
towards the participants that extends beyond their participation in the
trial.
Section 40(2)
Since this information effectively consists of sensitive personal data of
third parties, it is exempt from disclosure under Section 40(2) of FOIA
(by virtue of s.40(3)(a)(i)). That is to say, it is personal data, the
release of which to the world at large would not be fair and would thus
breach Principle 1 of the Data Protection Act 1998. This is an absolute
exemption.
For the purposes of determining release of personal data under FOIA,
Principle 1 is the most relevant (1). Principle 1 states that personal
data must be fairly and lawfully processed by a data controller (in this
case QMUL). Given that assurances of this sensitive data’s confidentiality
were given as a condition precedent to trial participation, it would not
be within the reasonable expectations of these individuals that the data
would be released in to the public domain. In addition, it is very likely
that the disclosure to members of the public without these data subjects’
further explicit consent would cause great distress, evidently to their
detriment. It is not fair to process personal data in a manner which leads
to adverse consequences, such as revealing that an individual suffers from
a certain health condition.
When these participants consented to taking part in the PACE trial, they
consented that the collection, use and analysis of their sensitive
personal data would be for a specific limited purpose only i.e. for
disclosure only to individuals associated directly with the PACE trial,
for its research analysis in this trial. Not only did the PACE trial
participants not consent explicitly to the further release of their
sensitive medical data beyond the scope of this limited purpose, they were
given assurances of its confidentiality, often as an essential condition
of their voluntary participation in the PACE trial. All participants
signed informed consent to participate based on the understanding that
"information collected about me for the trial, including my personal
details, a copy of this consent form and all of the questionnaires I
complete for the trial, will be held securely by the local trial staff and
at the PACE trial coordinating centre at Queen Mary, University of
London." Furthermore, they signed the consent form on the basis of a
patient information sheet, which included the following: "Will you keep my
details confidential? Yes. All your details and all recordings will be
kept strictly confidential and held in a locked filing cabinet or on a
secure computer. People on our research team will only see your records if
they need to for the research." The patient information sheet also stated:
"The data and recordings we collect will be securely stored for 20 years
after the end of the trial, for your protection and to follow good
clinical practice (GCP). The same applies to other records gathered for
our study, including your medical notes and the database holding the
collected data for this trial."
The full consent forms (two of them) are provided for your convenience and
the patient information sheet can be viewed at
[2]http://www.pacetrial.org/docs/trialinfo.....
None of the information is in the public domain and none of the
individuals were therefore informed that any disclosures would be made
under FOIA. No consent has been sought by QMUL. Release of individual data
as requested is not consistent with the contract that QMUL has with
participants as outlined in the consent form they signed and the patient
information sheet they read before giving their informed consent, the
relevant parts being quoted above. These factors mean any release would
not be fair. In any case, the Information Commissioner recognises the
following, “disclosure of such information [sensitive personal data] is
likely to be unfair as it comprises information that individuals will
regard as the most private. This means that in the majority of cases it
will be in the reasonable expectation of the individual that such
information will not be disclosed” (2). Given the circumstances in which
the data were obtained and specific assurances, there is no reason to
believe that expectations of participants will have changed.
The Information Commissioner states, “if the disclosure would not be fair,
then the information is exempt from disclosure” (3).
Principle 2 of the Data Protection Act 1998 also does not allow data
controllers to process personal data for further incompatible purposes.
Clearly the release of this sensitive data to a member of the public is
incompatible with the purpose for which explicit consent to its processing
was originally given. There are no other bases which would justify as
necessary the further processing of this sensitive personal data. Thus,
its disclosure here would mean that QMUL would contravene this principle
as well.
Considering the public interest in relation to the ‘fairness’ test, we do
not believe that there is an overriding legitimate interest in disclosure
to the public balanced against the consequences of disclosure. The
Information Commissioner further states in this regard, “the private
interests of the requester, or even a small group of people, are not
relevant in this context” (4). In any case we believe that the rights of
the individuals in question would outweigh these interests because of the
nature of the data, the potential detriment, the explicit assurances of
how the data would be used and because there is, on the contrary, no
expectation of putting this information in to the public domain.
Section 41
Section 41 of the Act exempts absolutely information provided to the
public authority by any other person in confidence if the disclosure
outside of this Act of such information would be actionable in confidence.
QMUL is of the opinion that this exemption is engaged here and meets all
of the elements for actionable breach of the confidence to which QMUL have
committed in the undertakings to these patients as discussed above.
Firstly, this is information that has the necessary quality of confidence
under the traditional tests of confidence under Coco v Clarke [1969] RPC
41, 47, Megarry J. It was disclosed in the context of a confidential
relationship, under a clear obligation of confidence. This is not only
that undertaken specifically here as indicated but as well since this is
data provided in the context of medical treatment, under the traditional
obligation of confidence imposed on medical practitioners. See generally,
General Medical Council, ‘Confidentiality’ (2009) available at
[3]http://www.gmc-uk.org/guidance/ethical_g...
Disclosure would be a breach of this obligation and again likely to the
detriment of these patients.
The public interest in disclosure defence that is built into confidence
would not likely control here. At issue is the protection of the
long-protected relationship with medical practitioners and their patients
which is intended to encourage patients to undertake important medical
treatment, even if in the context of a medical research trial, and to
encourage their full and frank disclosure so that treatment will be
appropriate. The disclosure here which risks the identification of
individual patients does not appear to outweigh the importance of
protecting these confidential relationships merely because the medical
treatment was undertaken in the course of a study.
Rather, the public interest in maintaining the confidence in this
sensitive personal data expected by these patients who were willing to
take the risk and participate in a study for the benefit of others
together with the traditional protection of the confidential medical
practitioner/patient relationship, more than outweighs any justification
presented for its release.
In light of this analysis, we are unable to release this sensitive data
since we believe Section 41 to be engaged.
Moreover, were disclosure to be made here it is also likely to deter
future research. Queen Mary carries out a vast range of research, both
medical and non-medical. It always complies with the sponsor/funder’s
policies on making data available, in this instance the MRC’s ‘Policy and
Guidance on Sharing of Research Data from Population and Patient Studies’
(5). Of relevance to this whole issue, we note that the Medical Research
Council, who funded this study and a recent further follow up study, now
recommends that individual data, properly anonymised or pseudonymised,
should only be released to bona fide researchers, employed by bona fide
research organisations, who will respect the confidential nature of such
data (6).
Releasing data in violation of the Data Protection Act and contrary to the
funder’s policies would be likely to compromise our ability to attract
research funding from these sponsors and deter individuals from
participating in future medical trials if their personal data cannot be
guaranteed to be kept confidential or anonymous. There is a strong public
interest in maintaining the agreed confidentiality with these patients and
their continuing trust in the commitment that their data will not be
disclosed except for further qualified research purposes in light of the
planned further research with this same pool of patients. By breaching the
contract between the research team and participants, harmful effects that
may be compensable may occur, since the knowledge that data about a
particular individual is being released to a member of the public may
upset that individual and cause anxiety that one release of data about
them may lead to more being released. It may also cause an individual
participant to lose faith in current and future investigators of any
medical trial that they might have considered participating in. This will
damage future trials, which is not in the public interest.
This is not a remote or theoretical possibility since Queen Mary has
analysed the outcome data of participants two and a half years after they
entered the trial, which has made the case to examine the effects of these
treatments five years after starting them. This is important so that
patients and their healthcare professionals can learn whether improvements
are maintained or enhanced a long time after treatment, or whether such
patients relapse after stopping treatment.
In accordance with s.17, please accept this as a refusal notice.
If you are dissatisfied with this response, you may ask QMUL to conduct a
review of this decision. To do this, please contact the College in
writing (including by fax, letter or email), describe the original
request, explain your grounds for dissatisfaction, and include an address
for correspondence. You have 40 working days from receipt of this
communication to submit a review request. When the review process has
been completed, if you are still dissatisfied, you may ask the Information
Commissioner to intervene. Please see [4]www.ico.org.uk for details.
Yours sincerely
Paul Smallcombe
Records & Information Compliance Manager
(1) See p.10 at
[5]http://www.ico.org.uk/for_organisations/...
(2) Ibid. at p.14
(3) Ibid. at p.11
(4) Ibid. at p.27
(5) See
[6]http://www.mrc.ac.uk/Ourresearch/Ethicsr...
(6) See
[7]http://www.mrc.ac.uk/Ourresearch/Ethicsr...
References
Visible links
1. http://search.ico.org.uk/ico/search/deci...
2. http://www.pacetrial.org/docs/trialinfo....
3. http://www.gmc-uk.org/guidance/ethical_g...
4. http://www.ico.org.uk/
5. http://www.ico.org.uk/for_organisations/...
6. http://www.mrc.ac.uk/Ourresearch/Ethicsr...
7. http://www.mrc.ac.uk/Ourresearch/Ethicsr...
Dear Mr Smallcombe / QMUL,
Thankyou for the response to FOI 2014/F73, which was refused as exempt under s.40(2) and s.41(1) of the Freedom of Information Act 2000 (FOIA), drawing upon the the Data Protection Act 1998 (DPA), and with references to GMC and MRC guidance: "as the information consists of sensitive medical data provided in confidence by participants in the PACE trial" and cannot be fairly processed by the data controller for purposes other than what it was explicitly collected for. Unfortunately, I am dissatisfied with the response and would like to request an internal review and further clarification with reference to the following:
• It is entirely possible to disclose anonymised trial data without breaching the DPA or fairness of the FOIA, the DPA's definition of personal data cannot be extended to cover situations where the disclosed data does not identify any individual, the disclosure of anonymised data is not a disclosure of personal data even when the data controller holds the key to re-identification, consent is not necessary to release anonymised data when it is unlikely to lead to re-identification, and the DPA does not require anonymisation to be completely risk free but mitigated until the risk of re-identification is remote. [1] This FOI request involves a heavily redacted dataset with most variables removed, or a selection of trial data which has undergone de-identification / anonymisation, and ceases to be personal data at the point of disclosure so the DPA does not apply. [2]
• QMUL have claimed that re-identification from the requested (pseudo)anonymised data is a significant risk. However, there is no obvious or plausible method for a member of the public to do this without additional data that is held securely at QMUL and highly unlikely ever to come into the possession of myself or other members of the public. According to the ICO's Knowledge Base on FOI Policy with reference to the Data Protection Act, the onus is on public authorities to explain how re-identification would occur. [3] Speculative assertions about alleged 'extremists' may reflect the potential existence of a 'motivated intruder', but if such people exist they are highly unlikely to succeed in re-identification and therefore such speculation is irrelevant. In a previous unrelated ICO ruling, the Commissioner attempted to play the role of 'motivated intruder' and was unsuccessful. [4] Any fair attempts to play the role of 'motivated intruder' for the purposes of assessing this FOI request would also fail. The selected data being requested was chosen so that the results could be calculated from the bare minimum of data required (except without the optional criteria for recovery from a clinical perspective, and with the addition of 6MWD data).
• It is also the responsibility of the public authority to establish how disclosure would lead to adverse consequences. QMUL have stated that disclosure without explicit consent is unfair and "very likely" to cause "great distress" to participants to their obvious detriment, with adverse consequences such as revealing that an individual suffers from a certain health condition, and therefore constitutes a probable actionable legal offense. These assertions appear to be based primarily on the assumption that the requested data is strictly personal data that can identity trial participants when disclosed. As the requested data would not plausibly lead to identification of any individuals, the given hypothetical examples of adverse consequences do not apply, and it remains unclear how disclosure would constitute a serious invasion of privacy causing considerable distress if no individuals can be identified. The fairness test can be satisfied by removing identifiable information about individuals and/or by anonymising the data, "for example, by removing the name but leaving the rest of the information". [5]
• Much of the requested data is already one step removed from the information directly provided by trial participants e.g. fatigue and physical function numerical scores are calculated from multiple answers to different questions on a paper questionnaire, the specific answers of which cannot be reliably extrapolated from the summed numerical scores. The requested data consists of either dichotomous or continuous values which are common and/or fluctuating (it is not comparable to genetic information or a National Insurance Number). The data is quantitative not qualitative; it is not for example the written accounts of personal opinions, identifiable contextual experiences about personal lives, or the handwriting of living individuals.
• QMUL stated that explicit consent will not be requested from trial participants for this particular FOI request. Assuming that such consent is actually necessary for the release of non-identifiable anonymised trial data (which according to multiple references no longer qualifies to be classified as personal data), the Ministry of Justice guidance on s.40 also points out that "a public authority should not be able to engineer a situation in which data cannot be disclosed by failing to notify the data subjects". [5]
• QMUL stated that "Principle 2 of the Data Protection Act 1998 also does not allow data controllers to process personal data for further incompatible purposes." However, other public authorities tried to use this argument to refuse the disclosure of anonymised data and failed, because anonymisation and/or redaction did not count as a form of processing. [6]
• The GMC's guidance on confidentiality and MRC's guidance on data sharing, cited by QMUL, did not appear to prohibit the disclosure of non-identifiable anonymised data to members of the public. Obviously this does not mean unrestricted access to all data collected, but it certainly opens up the possibility of releasing selected non-identifiable anonymised data.
• The implications of anonymisation on confidentiality is somewhat less clear, but the s.41 exemption can only be applied if there is a good chance of a successful actionable breach of confidence, [7] which QMUL believes is the case. However, at least 2 expert commentaries suggest that anonymisation of confidential data can replace the need for consent, and can change the nature of the data so that in most contexts it is no longer 'personal data' and thus not subject to the legal duties of data protection. [8][9] It it also appears that the public interest can override the requirement for confidentiality. [10][11]
• The public interest strongly favours disclosure, but the argument is complex and involves specific details about the trial which cannot be easily summarised into a paragraph. Therefore I will add an additional annotation below describing the public interest argument in about 2800 words and providing appropriate references. Patients and clinical commissioners have a right to accurate information about treatments promoted to them as rehabilitative or potentially curative, but it can be demonstrated that the (apparently post hoc and possibly unapproved) redefinitions of improvement and recovery in the PACE trial were too lax (e.g. recovery thresholds overlapping with the trial eligibility criteria for severe chronic disabling fatigue and not guaranteeing no longer having CFS), that these outcomes were inaccurately presented as based on strict or conservative thresholds, and that the most controversial change to the physical function criteria was erroneously based on a misinterpretation of summary statistics from a population study. The FOIA appears to be most plausible method for finding out the results as promised in the original stricter PACE trial protocol. It is doubtful whether disclosure would actually deter future research. Conversely, it could be counter-argued that research candidates may feel discouraged from participating in controversial research topics if previous trials have involved major, questionable, and possibly unapproved, deviations from the pre-approved original protocol.
Yours sincerely, Mr Matthees.
References (abbreviated to save space):
1. http://ico.org.uk/for_organisations/data...
2. http://ico.org.uk/news/blog/2012/anonymi...
3. http://ico.org.uk/foikb/PolicyLines/FOIP...
4. http://ico.org.uk/~/media/documents/deci...
5. http://www.justice.gov.uk/downloads/info...
6. http://www.cloudlegal.ccls.qmul.ac.uk/Re...
7. http://www.justice.gov.uk/downloads/info...
8. http://www.apira.co.uk/userfiles/files/C...
9. http://www.patient.co.uk/doctor/records-...
10. http://ico.org.uk/for_organisations/guid...
11. http://ico.org.uk/for_organisations/guid...
Reasons why the public interest strongly favours disclosure of the requested data:
(FOI 2014/F73) 18th June 2014
Assertions about complete recovery and/or functional remission from any chronic debilitating illness with a poor prognosis that is regarded as difficult to treat should be taken seriously and be based on reasonably stringent definitions. However, a recent 2014 systematic review of studies on recovery from CFS (including the PACE trial) concluded that in general what the literature defined as 'recovery' is better described as modest clinical improvement only. There was no guarantee of 'recovery' per se, as classification was based on limited assessments, less than a full restoration of health, and self-reports lacking objective measures in function which when used in behavioural intervention studies suggested no changes (prompting the authors to conclude that these therapies for CFS were not rehabilitative as often claimed). [1] An earlier 2012 systematic review concluded that a "comprehensive rehabilitation programme only rarely results in full recovery". [2]
The PACE trial has been repeatedly presented as offering 'definitive' answers on the controversial issue of 'rehabilitative' treatments for CFS (e.g. in the official website FAQ, [3] in the trial statistical analysis plan, [4] by the Science Media Centre, [5] and on ABC radio [6]). The published definition of recovery/remission was presented by the principal investigators White et al. as 'comprehensive and conservative' and purported to use stricter thresholds than a previous study on recovery from CFS by Knoop et al. published in 2007. [7] However, multiple significant issues have been identified with the recovery criteria which strongly challenge or contradict these presentations and have not been adequately addressed by the authors. [1,8,9] Disclosure of the requested data will greatly help the resolution of these issues.
The thresholds used for the 'normal range' score of fatigue and physical function inappropriately overlapped with the trial eligibility criteria for 'severe fatigue' and 'significant disability'. The recovery definition allowed participants to be classified as 'recovered' without reporting clinically significant improvements to fatigue and physical function, as such improvements were not required and allowed a 5 point decline in physical function. No longer meeting Oxford criteria for CFS in the trial did not necessarily mean no longer meeting Oxford criteria or suffering from CFS in the clinic, because additional criteria for fatigue and physical function were required, and participants were classified as 'no longer meeting Oxford criteria' if they failed to meet a single one of these thresholds e.g. moving from a score of 65 to 70 points in physical function but remaining unwell. 11% of excluded candidates failed to meet these additional criteria despite otherwise meeting Oxford criteria, which itself also requires fatigue to be the only principal symptom (which is not a requirement of any other CFS case definition. [10] and 80% of candidates who were definitely or provisionally diagnosed with CFS before the trial were excluded from participation, with the most common reason being not meeting Oxford criteria for CFS). Improvement on the clinical global impression scale does not guarantee a recovery from CFS or any improvement in the primary outcome measures of fatigue and physical function. The optional requirements of not meeting CDC criteria for CFS or London criteria for ME were superfluous because these were not an entry requirement, tend to be more difficult to meet than the Oxford criteria in the first place, and were not applied properly in the trial. [7,11]
The relevant trial oversight bodies approved the original 2007 protocol published in BioMed Central, which included a much more stringent definition of clinically significant improvement ('positive outcome') and complete 'recovery'. [12] According to BioMed Central, "publishing study protocols will help to improve the standard of medical research by ... enabling readers to compare what was originally intended with what was actually done, thus preventing both 'data dredging' and post-hoc revisions of study aims". [13] The purpose of pre-publishing a protocol is to avoid accusations of cherry picking the results, but when the protocol is ignored this clearly cannot be guaranteed. The thresholds for clinical improvement on an individual patient level for the primary measures of fatigue and physical function were abandoned and replaced with weaker thresholds which have been criticized for being minimal. [14,15] Similarly, all components of the recovery definition were significantly modified in a manner which made them substantially less stringent and easier to qualify. Of particular note, the threshold for normal physical function was dropped from 85 to 60 out of 100 points, a score low enough that 13% of participants were already within the 'normal range' at baseline despite meeting trial eligibility criteria for 'significant disability' (65 points or less). [16] In contrast, participants originally had to improve a minimum of 20 to 25 points to physical function to be classified as recovered. Other researchers of CBT for CFS have even classified a score of 60 to 70 points as indicative of 'severe' impairments in physical function. [17,18]
Professor White previously requested that the threshold for a 'positive outcome' in physical function (later abandoned) be raised from 70 to 75 points, because the entry criteria had been raised from 60 to 65 to increase recruitment, so a 10 point gap between entry criteria and 'positive outcome' scores was needed to avoid a 'trivial' difference. [19,20] Now there is a 5 point gap in the opposite direction, which cannot not be described as a strict or 'conservative' threshold. Although it has been argued that protocols can change in light of new information, it is unclear how any of these changes could "more accurately reflect recovery" as asserted in the paper by White et al. [7] Furthermore, as the changes to the definition of recovery published in 2013 appear to be largely based on controversial post hoc analyses conducted for a previous paper on the trial results published in 2011, [21] it is unclear whether these major deviations from the protocol were approved by the relevant trial oversight bodies, and this confusion surrounding the timing of changes has reached the level of parliamentary debate in the House of Lords. [22]
As a previous claim made in the Lancet paper about the normative dataset used from a population study had turned out to incorrect, [23] it seemed prudent to examine the justification behind the most controversial change to the recovery criteria. White et al. asserted that the change to the threshold of normal physical function was justified because a score of ≥85 "would mean that approximately half the general working age population would fall outside the normal range". [7] However this is incorrect, as independent analyses of the English normative dataset cited by White et al. revealed that over half score the maximum of 100 points. The median(IQR) score for the general working age population sample is 100(90-100) not about 85 as implied (which suggested an erroneous assumption that the mean and median were equivalent), and only about 18% of the general working age population sample had a score under 85. [24] The original threshold of >=85 points appears to be reasonable and appropriate, as it "represents the ability to carry out moderate activities, such as lifting a table, carrying purchases, or bowling, without limitations". [25] 92% of the healthy working age population score 85 points or more, and 61% score the maximum of 100 points. The mean(SD) and median(IQR) scores for this population are 95.0(10.2) and 100(95-100) respectively, with scores under 80 appearing to be extreme outliers when defined as more than 3 x IQR below the median. [24] It is highly unlikely that the PACE trial participants classified as 'recovered' have a similar distribution of scores compared to a healthy working age population.
White et al. stated that "we derived a mean (S.D.) score of 84 (24) for the whole sample, giving a normal range of 60 or above for physical function" and asserted that this sample was "demographically representative". [7] However, the 'whole sample' was a general population which included the elderly and chronically disabled, [26] with age and illness having a major impact on physical function scores in a way which decreases the mean and increases the standard deviation, therefore lowering the threshold of 'normal'. The mean(SD) age was 48.3(19.0) years, 32% were aged 60 years or more, and 22% reported chronic debilitating illnesses (many of which would have medically excluded candidates from participating in the PACE trial). [24][26] Whereas PACE trial participants had a mean(SD) age of about 38(12) years at baseline, only 3% were aged 60 years or more, [27] and were previously screened for common chronic debilitating illnesses in the population which would have excluded them from a CFS diagnosis.
Although described as a 'conventional' method, [23] White et el. have applied a simple parametric statistical method to a dataset without any apparent consideration for what the authors of the cited paper (Bowling et al.) described as a heavily skewed distribution, [26] which was specifically warned against in a paper previously co-authored by Professor White [28] and has been described elsewhere as a "fundamental misuse of statistics". [29] Furthermore, the use of normative data from a general population sample with important demographic differences to PACE trial participants (age distribution and presence of common debilitating illnesses) has never been justified in any of the publicly available PACE trial literature. It is unclear why the authors did not stop and think twice before using a 'recovery' threshold that was unusually low and overlapped with their own trial criteria for 'significant disability'. A score of 60 points means reporting significant limitations in multiple domains (somewhere between minor limitations for 8/10 questions or major limitations in 4/10 questions), [30] which is unusual for healthy people of working age and an unsuitable threshold for a genuine recovery. White et al. [7] incorrectly claimed that their threshold was more "conservative" i.e. stricter than the previous work of Knoop et al. [28] The latter paper actually used the same mean plus or minus 1 SD formula as PACE did (not mean plus or minus 2 SD as claimed by White et al.), and relied on a healthy population instead of a general population to calculate a higher threshold of 80 points in physical function as the normal threshold for recovered. Similarly, serious questions have also been raised about the suitability of the threshold for normal fatigue and the population used to derive it. [8,31-33]
In response to the paper on 'recovery', Dr Esther Crawley from the University of Bristol said that "Every patient with CFS/ME wants to know how likely they are to recover." [34] Yet, many patients were rather unsatisfied with the major deviations from the previously established protocol, questioned the 'normal range' in particular, and wanted to know the 'positive outcomes' and more importantly the recovery rates as previously defined more stringently. A collection of patient charities made a FOI request for this information in 2011, which included the results according to the original recovery criteria [35] but were refused on the grounds that this information was exempt under s.22 of the FOIA i.e. due for future publication. [36] A similar FOI request in 2012 was refused on the grounds that the information was not held in final form because the definition of recovery had changed with a pending paper and there was no intention on publishing the requested information in the future (the refusal notice also incorrectly claimed that some of the changes made the definition more stringent). [37] Another FOI request in 2013 was refused on the grounds that the raw data required to calculate these outcomes does exist but would require over 18 hours to do so. [38] Therefore, this FOI request is for selected raw data so that these calculations can be done without QMUL.
ME/CFS is regarded as a controversial subject, but this controversy is only further fuelled by the lack of transparency over trial results presented as 'definitive' and the failure to publish the measures specified in the original approved protocol. Given that the published recovery thresholds appear to be fundamentally based on previous post hoc analyses, coincide with less than expected clinical improvements, and are generally at or below the level of the trial entry criteria, it is difficult to believe that the accusations of cherry picking or intentionally misleading vulnerable patients and clinical commissioners (irrespective of whether it is true or not) will simply go away without the publication of these stricter outcomes. It is critical that sufficient data is placed in the public domain to allow patients and clinical commissioners to accurately assess recovery and the sensitivity to any particular threshold.
There have been recent calls for medical research to be more transparent and accessible and accountable, as per the AllTrials campaign (www.alltrials.net). Although this does not necessarily mean unrestricted public access to all the data of a trial, AllTrials calls for "All trials past and present should be registered, and the full methods and the results reported." The Wellcome Trust takes a step further and calls for the full release of all trial data. [39] The public interest in transparency around drug trials has been well established by the European Medicines Agency and the same principles should apply to psychotherapy and/or behavioural interventions. [40] The PACE trial was publicly funded research and the (anonymised) data should be openly available to the maximum practical extent. Answers to remaining questions in science are generally gained from further replication, but the PACE trial cost taxpayers £5m, and due to its high cost and large size, it is highly unlikely that another similar trial will be conducted anytime soon. Therefore, the collected data should be explored to the maximum extent possible. Without voluntary transparency, the task of finding out the results as promised in the original PACE trial protocol depends on members of the public, and the FOIA appears to be most plausible method for seeing this happen in the foreseeable future.
The ongoing confusion and controversy is adding to the suffering of patients, and getting to the bottom of this issue is important whatever the outcome may be. The results and interpretations do not just affect those who are curious about research, but have national and perhaps even global ramifications. Patients and clinical commissioners of this chronic debilitating illness have a right to accurate information about treatments which are promoted to them as rehabilitative and potentially curative. This is required for them to assess and give informed consent for treatments, or make informed decisions about health care. Lax definitions of recovery/remission and clinical improvement lead to unreasonable expectations from patients by those who provide their care. In a similar study known as the FINE trial [41] (which released the results according to its own published protocol and failed to show significant improvements with therapies similar to and sharing elements with CBT/GET tested in the PACE trial), some participants had doubts about the (overly optimistic) treatment rationales, and therapists reported becoming angry and blaming participants as "the bastards don't want to get better". [42]
It is doubtful whether disclosure would actually deter future research. Conversely, it could be counter-argued that research candidates may feel discouraged from participating in controversial research if previous trials have involved major, questionable, and possibly unapproved, deviations from the pre-approved original protocol which made it much easier for the tested therapies to appear successful, coincided with less impressive than expected results, and led to the results being exaggerated. For example, the published rates of trial participants within the 'normal range' for fatigue and physical function (which overlapped with trial eligibility criteria for severe chronic fatigue and significant disability) was presented in 2011 at a Lancet press conference with the principal investigators as returning back to normal, [43] and this was then widely misinterpreted as a complete recovery or cure in the national news media e.g. [44,45] and medical journals e.g. [46,47] The Lancet editorial which accompanied the 2011 paper on the PACE trial results inaccurately claimed that the 'normal range' was a strict criterion for recovery based on scores from healthy people, [48] but the Press Complaints Commission later ruled that this comment was misleading and breached Clause 1 (Accuracy) of the Code. [49] Such repeated misstatements of fact have negative implications for how patients are treated by doctors, how funding decisions are made, and for scientific accuracy concerning recovery from ME/CFS. A poll conducted on the ME Association website during March 2011 revealed that 89% of 751 respondents were significantly concerned that the PACE trial results would adversely affect treatment within the NHS. [50]
Unless the PACE group themselves promptly publish the original protocol-defined 'positive outcomes', the original protocol-defined 'recovery' rates, and summary statistics on those classified as recovered (both versions) compared with appropriate summary statistics of healthy populations with a similar age distribution as trial participants, then the disclosure of the requested data allowing others to do the necessary calculations is certainly in the public interest. Given that the lax definition of 'recovery' fundamentally depends on a threshold for 'normal' physical function which appears to be seriously flawed and inaccurately presented as strict or conservative, with the reason for abandoning the original protocol-defined threshold found to be erroneously based on a misinterpretation of summary statistics from a population study, the requested data will be important to help the public (patients, carers, research community, healthcare staff, et cetera) further assess the degree and nature of improvements in the PACE trial. Please help resolve this controversy once and for all by granting this FOI request.
References (abbreviated to save space):
1. http://www.ncbi.nlm.nih.gov/pubmed/24791...
2. http://www.ncbi.nlm.nih.gov/pubmed/22725...
3. http://www.pacetrial.org/faq/faq2.html
4. http://www.trialsjournal.com/content/14/...
5. http://www.sciencemediacentre.org/expert...
6. http://www.abc.net.au/radionational/prog...
7. http://www.ncbi.nlm.nih.gov/pmc/articles...
8. http://www.ncbi.nlm.nih.gov/pubmed/23363...
9. http://www.meassociation.org.uk/2013/07/...
10. http://www.iacfsme.org/BULLETINWINTER201...
11. http://www.ncbi.nlm.nih.gov/pubmed/21334...
12. http://www.biomedcentral.com/1471-2377/7/6
13. http://www.biomedcentral.com/authors/pro...
14. http://www.thelancet.com/journals/lancet...
15. http://www.thelancet.com/journals/lancet...
16. https://listserv.nodak.edu/cgi-bin/wa.ex...
17. http://www.ncbi.nlm.nih.gov/pubmed/22354...
18. http://eurpub.oxfordjournals.org/content...
19. https://listserv.nodak.edu/cgi-bin/wa.ex...
20. http://www.ico.org.uk/~/media/documents/...
21. http://www.ncbi.nlm.nih.gov/pmc/articles...
22. https://www.whatdotheyknow.com/request/t...
23. http://www.thelancet.com/journals/lancet...
24. http://dx.doi.org/10.5255/UKDA-SN-3660-1
25. http://www.ncbi.nlm.nih.gov/pubmed/9054791
26. http://www.ncbi.nlm.nih.gov/pubmed/10528...
27. http://www.bmj.com/content/347/bmj.f5963...
28. http://www.ncbi.nlm.nih.gov/pubmed/17426...
29. http://www.bmj.com/content/347/bmj.f5963...
30. http://www.rand.org/health/surveys_tools...
31. http://www.thelancet.com/journals/lancet...
32. http://evaluatingpace.phoenixrising.me/a...
33. http://www.meactionuk.org.uk/Normal-fati...
34. http://www.sciencemediacentre.org/expert...
35. http://www.meassociation.org.uk/?p=6171
36. http://www.meassociation.org.uk/wp-conte...
37. https://www.whatdotheyknow.com/request/p...
38. https://www.whatdotheyknow.com/request/p...
39. http://blog.wellcome.ac.uk/2013/08/05/fu...
40. http://www.ema.europa.eu
41. http://www.ncbi.nlm.nih.gov/pmc/articles...
42. http://www.ncbi.nlm.nih.gov/pmc/articles...
43. http://www.meactionuk.org.uk/pacepressco...
44. http://www.ncbi.nlm.nih.gov/pmc/articles...
45. http://www.bmj.com/content/342/bmj.d1168...
46. http://www.guardian.co.uk/society/2011/f...
47. http://www.dailymail.co.uk/health/articl...
48. http://www.thelancet.com/journals/lancet...
49. http://www.pcc.org.uk/news/index.html?ar...
50. http://www.meassociation.org.uk/archive-...
Mr Matthees left an annotation ()
There are a few typos worth correcting:
Re "with scores under 80 appearing to be extreme outliers when defined as more than 3 x IQR below the median". The end part should read "3 x IQR below the 1st quartile". The value of 80 points is still valid.
Re "which cannot not be described as a strict or 'conservative' threshold". The beginning part should be "which cannot be described".
To whomever handles internal reviews of FOI requests at QMUL,
On 18th June 2014 (40 working days after the initial submission) I asked for an internal review of the handling of a FOI request but have received no answer or any acknowledgment that it was received.
https://www.whatdotheyknow.com/request/s...
According to the following website, 23 working days has elapsed since that date:
The Information Commissioner’s Office recommends that public authorities carry out internal reviews within 20 working days:
http://ico.org.uk/for_the_public/officia...
Similarly, the ICO's guidance on conducting internal reviews states that:
"In view of all the above the Commissioner considers that a reasonable time for completing an internal review is 20 working days from the date of the request for review. There may be a small number of cases which involve exceptional circumstances where it may be reasonable to take longer. In those circumstances, the public authority should, as a matter of good practice, notify the requester and explain why more time is needed. In our view, in no case should the total time taken exceed 40 working days. In such cases we would expect a public authority to be able to demonstrate that it had commenced the review procedure promptly following receipt of the request for review and had actively worked on the review throughout that period."
https://www.whatdotheyknow.com/request/1...
Can you please acknowledge receiving the request for an internal review of the handling of this FOI request, and provide any details on what is causing the apparent delay with a rough estimate on when I should expect a complete response. I understand that reviewing the handling of this FOI request may not be the simplest or easiest, so I am willing to wait until 40 working days have elapsed after the initial request for an internal review, 12th August 2014, before concluding that there had been a failure to respond. Please note that I have also posted two annotations which contain a clarification and an erratum.
https://www.whatdotheyknow.com/request/s...
Yours sincerely,
Mr Matthees
Dear Mr. Matthees
Queen Mary's Procedure for Internal Reviews follows the guidelines of the Information Commissioner and states that QMUL will endeavour to conduct these within 20 working days.
Unfortunately it has not been possible in this case, though we will aim to respond within 40 working days.
Yours sincerely
Queen Mary University of London
Dear QMUL FOI Enquiries,
On the 18th June 2014, I requested an internal review regarding the FOI request "Selected data on PACE Trial participants".
After I sent a reminder about the apparent delay, QMUL responded with: "Queen Mary's Procedure for Internal Reviews follows the guidelines of the Information Commissioner and states that QMUL will endeavour to conduct these within 20 working days. Unfortunately it has not been possible in this case, though we will aim to respond within 40 working days."
A record of previous correspondence can be found here:
https://www.whatdotheyknow.com/request/s...
52 working days have now passed since the original request for the internal review.
According to the ICO, "In our view, in no case should the total time taken exceed 40 working days. In such cases we would expect a public authority to be able to demonstrate that it had commenced the review procedure promptly following receipt of the request for review and had actively worked on the review throughout that period."
https://www.whatdotheyknow.com/request/1...
Can you please provide me with an update.
Yours sincerely,
Mr Matthees.
Mr Matthees left an annotation ()
A minor correction for an earlier post.
I wrote: "On 18th June 2014 (40 working days after the initial submission) ..."
It should read: "On 18th June 2014 (40 working days after the refusal notice) ..."
Dear Mr. Matthees
The internal review is still in progress. We apologise for the delay and will respond as soon as we are able.
Yours
Queen Mary University of London
Mr Matthees left an annotation ()
Three of the URL references in the public interest argument did not render properly on this website. Here are URLs which should work, and in the event that they do not, here are the full citations:
14. Jane Giakoumakis. The PACE trial in chronic fatigue syndrome. The Lancet, Volume 377, Issue 9780, Page 1831, 28 May 2011. http://www.thelancet.com/journals/lancet...
15. Bart Stouten, Ellen M Goudsmit, Neil Riley. The PACE trial in chronic fatigue syndrome. The Lancet, Volume 377, Issue 9780, Pages 1832 - 1833, 28 May 2011. http://www.thelancet.com/journals/lancet...
31. Sarah M Feehan. The PACE trial in chronic fatigue syndrome. The Lancet, Volume 377, Issue 9780, Pages 1831 - 1832, 28 May 2011. http://www.thelancet.com/journals/lancet...
QMUL have not complained about non working URLs so the emailed copy they received from this website probably works OK.
Dear Mr. Matthees
Queen Mary has now concluded its internal review procedures on this
request.
The decision of the internal reviewer is to uphold the original decision
to withhold the information you requested.
We apologise for the delay, which was down to staff leave and the
commencement of the new academic year.
If you remain dissatisfied you have the right to appeal to the Information
Commissioner’s Office. Please see [1]www.ico.org.uk for details.
Yours sincerely
Queen Mary University of London
References
Visible links
1. http://www.ico.org.uk/
Mr Matthees left an annotation ()
QMUL have refused this request but I want to post an update for the record.
I still maintain that QMUL have failed to provide convincing reasons why the requested data cannot be sufficiently pseudonymised or how the additional personal identifiers required to enable re-identification could somehow "fairly readily" become public.
1) Regarding this statement I made on confidentiality:
"The implications of anonymisation on confidentiality is somewhat less clear, but the s.41 exemption can only be applied if there is a good chance of a successful actionable breach of confidence, [7] which QMUL believes is the case. However, at least 2 expert commentaries suggest that anonymisation of confidential data can replace the need for consent, and can change the nature of the data so that in most contexts it is no longer 'personal data' and thus not subject to the legal duties of data protection. [8][9]"
Anonymisation definitely has similarly fundamental implications for confidentiality (as it does for the FOIA and DPA). Both the NHS and GMC guidelines or code of practice for confidentiality strongly indicate that the confidentiality and sensitivity of medical information is fundamentally determined by whether that information can identify individual patients. The NHS code of practice for confidentiality points out that effectively or sufficiently anonymised information is not confidential and may be used with relatively few constraints, and once information is effectively anonymised it no longer requires consent.
^ https://www.gov.uk/government/publicatio...
^ http://www.gmc-uk.org/guidance/ethical_g...
^ http://www.gmc-uk.org/guidance/ethical_g...
2) Regarding the issue that the questionable post-hoc revised threshold for normal physical function was derived by using an inappropriate statistical calculation on a non-representative population sample, and that the stated reason for abandoning the protocol-defined threshold was based a factual error (simple misinterpretation of summary statistics from a population study). This information has now been published in a peer-reviewed commentary:
^ Matthees A. Assessment of recovery status in chronic fatigue syndrome using normative data. Qual Life Res. 2014 Oct 11. [Epub ahead of print] PMID: 25304959. http://link.springer.com/article/10.1007...
3) QMUL's apparent assertion that confidentiality contacts explicitly prohibit the sharing of such data with anyone outside the PACE group at all is doubtful and contradicted by the sharing of data to a third party (Cochrane) and the publication of results in PLoS One which expects that all data underlying the published findings is made fully available without restriction with rare exception.
^ http://www.publications.parliament.uk/pa...
^ http://www.plosone.org/static/policies.a...
4) QMUL implied that releasing the requested data would scare off participants from being involved in the five-year follow-up; however, as the 52 week follow-up was completed in January 2010, the five-year follow-up should also be completed.
^ http://www.ncbi.nlm.nih.gov/pmc/articles...
5) Regarding this statement on the The Wellcome Trust's position on data sharing:
"The Wellcome Trust takes a step further and calls for the full release of all trial data. [39] The public interest in transparency around drug trials has been well established by the European Medicines Agency and the same principles should apply to psychotherapy and/or behavioural interventions. [40]"
A correction: That was a blogpost and not the official stance of the Wellcome Trust, which is still in the process of discussing methods to "develop a global solution to increase transparency around trials and develop a model for greater access to trial data, in a way that protects the confidentiality of trial participants".
^ http://www.wellcome.ac.uk/About-us/Polic...
The Wellcome Trust are still negotiating the procedures and guidelines for open data:
^ http://blog.wellcome.ac.uk/2013/08/05/fu...
6) The PACE Trial protocol published in 2007 stated that "The trial will be conducted in compliance with the Declaration of Helsinki,". Both the Declaration of Helsinki and the World Health Organisation's International Clinical Trials Registry Platform (ICTRP) program state that researchers are responsible and accountable for the completeness and accuracy of their reports.
^ http://www.biomedcentral.com/1471-2377/7/6
^ http://www.wma.net/en/30publications/10p...
^ http://www.who.int/ictrp/results/en
The MRC has also signed up to the AllTrials campaign, which demands that "all trials past and present should be registered, and the full methods and the results reported". Unfortunately, after nearly 5 years since the relevant data was collected, QMUL/PACE have still not adequately reported on the outcome of recovery from CFS.
^ http://www.alltrials.net/supporter-orgs/...
Mr Matthees left an annotation ()
A few more links that others may find useful:
The European Medicines Agency (http://www.ema.europa.eu) are now backing public access to clinical trial data: "A key change from earlier versions of the policy is that researchers, and other members of the public, will now be able to download data. [...] In future the EMA will make data down to the level of individual patients available, but it wishes to consult further about how it can do this while protecting privacy."
^ http://blogs.nature.com/news/2014/10/eur...
In a systematic review of the use of the FOIA (UK) for healthcare research published between January 2005 to January 2013, it was reported that 1732 requests were made across the 16 formal studies which met the criteria for inclusion. It was concluded that researchers should make greater use of the FOIA to access the information they need that is not otherwise disclosed. The authors discussed various issues with research of this type and suggest ways it could be optimised. It was also argued that scrutinizing data is a key part of the scientific method and having raw data openly available is a key aspect of academic freedom.
^ Fowler AJ, Agha RA, Camm CF, Littlejohns P. The UK Freedom of Information Act (2000) in healthcare research: a systematic review. BMJ Open. 2013 Nov 27;3(11):e002967. doi: 10.1136/bmjopen-2013-002967. PMID: 24285626. http://www.ncbi.nlm.nih.gov/pmc/articles...
Paul Watton left an annotation ()
Following an appeal to The Information Commissioner, the ICO has now ruled in favour of the applicant's FOI request and details of the judgement can be found in the following report: http://www.meaction.net/wp-content/uploa...
Mr Matthees left an annotation ()
Thankyou for posting that update, Paul Watton.
https://ico.org.uk/media/action-weve-tak...
I appreciate the ICO's support for this FOI request (FS50565190). Unfortunately, today I received an email from the ICO to inform me that QMUL have appealed to the Information Tribunal. I am relieved that a solicitor will now be defending the case with input from me if necessary.
I am still preparing a detailed comment on the recent ICO decision notice in general and on QMUL's speculative assertions about my implied involvement with a malicious campaign of vicious activists who allegedly want to discredit PACE and harass researchers.
The last few months have been very busy and it has been a struggle to keep up with all the recent developments relevant to the PACE trial. There are too many to list in a brief message, but to summarize some examples: Starting with journalist David Tuller's series of articles critiquing the trial, multiple research experts and senior academics are now openly criticizing how the PACE trial was conducted, analyzed, or reported. This includes James Coyne who has given strong vocal support for this FOI request (http://tinyurl.com/o9sfywc), with Jonathon Edwards also offering his support. There is an open letter to the Lancet by six prominent figures, who outline significant problems with the trial and insist on a fully independent re-analysis of individual-level data by qualified experts. Similarly, there is also an independent petition with over 10,000 individual signatures calling for the retraction of questionable claims and the release of anonymised individual-level trial data.
I submitted this FOI request to help resolve the ongoing controversy over the trial results. It is compatible with calls in the research community for increased transparency in clinical trials and open data once methods to protect patient privacy are established (these calls were prompted by awareness of significant problems with how clinical trial research has been conducted and reported). Although the public interests which favour disclosure of the requested data have existed for several years, the recent developments described above are strong additional indicators that the circumstances of the PACE trial demand the disclosure of a selection of de-identified trial data to enable transparent and open (publicly verifiable) re-analysis of the trial results for the benefit of the patient community and scientific debate about the value of the interventions tested.
Walter Irvine left an annotation ()
Are you kidding me? They obviously have something in their study data that they don't want the public to know about
Paul Watton left an annotation ()
In spite of the ongoing controversy surrounding the PACE trial, the NIHR have approved funding for the MAGENTA trial - led by PACE researcher Dr Esther Crawley - studying the use of graded exercise therapy in loosely defined paediatric cases of CFS http://public.ukcrn.org.uk/search/StudyD...
and Bristol University are recruiting a Research Associate / Trial Manager for a fixed term of 5 years at a cost of up to £40k/Yr http://www.jobs.ac.uk/job/AMV070/researc...
Paul Watton left an annotation ()
The First Tier Tribunal has dismissed QMUL's appeal and has therefore upheld the Information Commissioner's Decision Notice dated 27th October 2015, requiring that the PACE trial data to be released.
You can read the First Tier Tribunal Decision Notice here: http://www.informationtribunal.gov.uk/DB...
On a personal note, like so many other ME/cfs sufferers, I am immensely grateful to Mr Alem Matthees for his persistence and hard work in getting to this point. He has done a tremendous service for the preservation of good science and truth.
Mr Matthees left an annotation ()
Thankyou, Paul Watton. Please note that, to my knowledge, the decision notice is still open to corrections and may not be the final version.
Epson left an annotation ()
This data has now been released, confirming what was obvious to anyone interested in looking seriously at the evidence: the results from the PACE trial had been spun. Using the trial's pre-specified recovery criteria neither CBT nor GET led to significant improvements in participant's recovery rates:
http://www.virology.ws/2016/09/21/no-rec...
http://www.thecanary.co/2016/10/02/resul...
Congratulation to Mr Matthees for this impressive victory for those who care about science and the fight against quackery.
We work to defend the right to FOI for everyone
Help us protect your right to hold public authorities to account. Donate and support our work.
Donate Now
Mr Matthees left an annotation ()
Clarifications for the public interest argument above:
• If there is any doubt about the ease of which some trial participants could be disqualified from meeting Oxford criteria at followup (for failing any additional ad hoc criteria bolted onto the Oxford criteria for the purposes of the trial), QMUL previously stated: "A score of 70 or more on the SF-36 sub-scale would mean that the participant did not meet Oxford criteria. Similarly having a score of 5 or less on the Chalder fatigue questionnaire would mean that the participant did not meet Oxford criteria." [1] This is not usually part of the Oxford criteria and means that no longer meeting Oxford criteria at followup in the trial could be achieved by borderline cases improving a single increment on either scale but who could still be significantly affected by CFS and still otherwise meet Oxford criteria outside the trial setting. In the original trial protocol, a fatigue score of 5/11 on the CFQ (bimodal scoring) and a score of 70/100 on the physical function sub-scale of the SF-36 were still both regarded as abnormal scores, and required a fatigue score of 3/11 and a physical function score of 85/100 to be classified as recovered. [2]
• Re: "11% of excluded candidates failed to meet these additional criteria despite otherwise meeting Oxford criteria". This was based on Figure 1 of the 2011 Lancet paper, [3] but it is not entirely clear how many excluded candidates failed to meet each stated reason for exclusion, as only one reason is given for each exclusion but it is plausible that excluded candidates failed more than one. Out of the 2517 candidates who were excluded (all of which had previously been definitely or provisionally diagnosed with CFS), 43% were excluded for not meeting Oxford criteria, 10% were excluded for scoring over 65/100 points in physical function, and 1% were excluded for scoring under 6/11 points in fatigue, suggesting that some excluded candidates still had CFS.
• A video has been made demonstrating how a trial participant could improve a single increment in fatigue, decline a single increment in physical function, report feeling significantly better e.g. due to pain medication, and then be classified as 'recovered' despite remaining significantly unwell with CFS. [4] Some participants would have to improve more to cross the thresholds, but it demonstrates a problem with the lax recovery criteria and that combining multiple recovery criteria does not necessary negate the weaknesses of each individual criteria. The definition of 'recovery' is simply too lax to justify the use of the word 'recovered', particularly without objective measures (which when used have generally shown a lack of clinically significant improvement).
• Re: "The thresholds for clinical improvements on an individual patient level for the primary measures of fatigue and physical function were abandoned and replaced with weaker thresholds...". To further clarify in context with the sentence which preceded it about BioMed Central's statement on post hoc revisions, the latter thresholds were the 'clinically useful difference' i.e. 2/33 points in fatigue (Likert scoring) and 8/100 points in physical function, which did not appear in the final version of the statistical analysis plan, [5] and were described as post hoc in the 2011 Lancet paper. [3] It is unclear when these were introduced and whether they were approved by the relevant trial oversight bodies (see below for why doubts have been raised).
• Re: "as the changes to the definition of recovery published in 2013 appear to be largely based on controversial post hoc analyses conducted for a previous paper on the trial results published in 2011". To further clarify, the methods and thresholds for the normal range for fatigue and physical function used in the revised definition of recovery did not appear in the final version of the statistical analysis plan [5] and are exactly the same as the post hoc analyses that were introduced during the peer review stage [6] of the fast tracked 2011 Lancet publication [3] (i.e. not introduced by the authors themselves). This is why doubts have been understandably raised over whether the changes to the recovery thresholds were approved by the relevant trial oversight bodies and whether the recovery definition was revised after rather than before these post hoc analyses were conducted, which would contradict the published impression that the changes were made to the recovery criteria before analysing any data. [7,8]
1. https://listserv.nodak.edu/cgi-bin/wa.ex...
2. http://www.biomedcentral.com/1471-2377/7/6
3. http://www.ncbi.nlm.nih.gov/pmc/articles...
4. http://www.youtube.com/watch?v=d_7J5ELjArU
5. http://www.trialsjournal.com/content/14/...
6. http://www.meactionuk.org.uk/whitereply....
7. http://www.ncbi.nlm.nih.gov/pmc/articles...
8. https://www.whatdotheyknow.com/request/t...