Software code for air transport models
Dear Department for Transport,
The DfT document "UK Air Passenger Demand and CO2 Forecasts", published January 2009, cites and exhibits the results from a number of economic models.
http://www.dft.gov.uk/pgr/aviation/atf/c...
In particular, the following models and forecasts are mentioned by name, and calculated results given in a non-computer readable format:
* Annex A: Unconstrained Passenger Demand Econometric Models (operated for each of 21 markets).
* Annex B: The National Air Passenger Demand Model, which includes 21 econometric models.
* Annex C: The Passenger Airport Choice Model
* Annex D.3: National Air Passenger Demand Model
* Annex D.5: National Air Passenger Allocation Model
* Annex D.6: CO2 model
* Annex G: Detailed Demand and CO2 Forecasts
* Annex H: Monetised Net Benefits Methodology, comprising several subsections detailing the benefit types.
* Annex I: Surface Access CO2 Emissions
* Annex J: Estimate Value of UK Aviation's Climate Change Impact
* Annex K: Aviation's Estimated Share of UK Climate Change Emissions
As you know, there is considerable public skepticism concerning the claim that air travel can increase by 60% over the next 4 decades whilst meeting the UK's climate change obligations. It is the government's claim that these calculated numbers prove otherwise.
Under the Freedom of Information Act 2000, please send me:
* Information about which departmental team or outside consultants is in charge of developing, revising and estimating the values for the input parameters in each of the above models, and whether there is any independent verification of them.
* How frequently the model calculations are rerun with new data and what the procedures exist for either reviewing public policy as a result of changes, or varying the input parameters to keep the output on track.
* What programming languages or software platforms these models run within.
* An actual copy of the software code for the models, including inputs, in a form that someone with access to the same software platforms could rerun them with minimum of effort.
Please advise me of whether you hold this information, and then disclose as much of it as you can.
There is insufficient information in this document to recreate the models and it is not realistic to expect that people must do so in order to independently verify the results given alternative estimates of the input parameters.
For example, according to the footnote 78 in section C.3:
""The Passenger Airport Choice Model is an application of the standard multinomial logit formulation commonly used in this context. The model assumes the proportion of passengers with journey purpose p travelling to/from UK zone i to foreign destination j, that use airport A, PijAp, can be represented by the following very flexible functional form:[78]
...
[78]The form shown is the simplest of those used""
Thank you for your consideration of this request.
Yours faithfully,
Julian Todd
Dear Sir
I am writing to acknowledge receipt of your request for information which has been allocated the reference number P0006237. A response will be issued to you in due course.
Regards,
Department for Transport
Information Rights Unit
D/04, Ashdown House
Sedlescombe Road North
St Leonards on Sea
East Sussex
TN37 7GA
Dear Mr Todd
Please find attached a letter relating to your request for information under the Environmental Information Regulations 2004.
Yours sincerely,
Richard
Dear Richard Clarkson,
Thank you for your 24 March response to my 23 February FOI request for background details on the economic and environmental models used in the DfT's January 2009 "UK Air Passenger Demand and CO2 Forecasts" study.
You advised me that the time limit for responding to my request had been extended for 20 days beyond the statutory limit, and that my request was being handled under the Environmental Information Regulations.
I am writing to ask for some specific details about the progress of my request.
Firstly, I am not sure that all of the listed models cited by the study fall within the definition of "environmental information". Can you list the parts of my request that fall into the different categories?
Secondly, section 10(3) of the FOI Act sets out the circumstances in which the 20 day time limit need not be complied with in respect to qualified exemptions that include a public interest test.
Can you confirm whether the information I requested is on your desk now and is awaiting the application of the public interest test before disclosure, or has the whole process been delayed and the department has not reached this stage yet.
Please could you also inform me which exemptions are being considered in respect to each piece of information, as this may allow me to agree to narrow my request slightly and thus ensure a speedier disclosure.
Yours sincerely,
Julian Todd
Dear Mr Todd
Thank you for your email below. Please see attached letter which responds to the points raised and provides a partial response to your original information request.
Yours sincerely,
Richard
Dear Mr Todd
Please see attached final response to your information request of 23rd February.
Yours sincerely,
Richard
Dear Richard Clarkson,
Thank you for your final reply on 22 April to my request for information about the eleven social, economic and environmental computer models that are used as the basis for supporting the expansion of air-travel policy.
The Department for Transport's opinion is very clear on this issue, which is why any self-generated data such as from these DfT generated models needs to be handled with care.
1. I asked for information about who designs these models, and who maintains them, and whether there is any independent verification.
I have not been given sufficient information to determine whether there is any independent verification of these models or their parameters. I do not know if the "independent peer reviewer" or the "external appraisal expert" are also employed by the DfT, or to what depth they reviewed the modelling.
Please send me further details that you have, if any, that would give me confidence that these models are objective. Owing to 4., there will have been no external assessment of these models that has not been authorized by the DfT.
2. Some of the inputs into these models are the GDP growth forecasts, interest rates, oil price forecasts, and so on, which the Treasury and other international bodies update on a monthly basis.
In many institutions that rely on correct assessments from their economic models it is routine to rerun every time new data becomes available.
You have informed me that these models are run very infrequently owing to the enormous amount of effort involved, and that a new run is done according to a publication schedule that is less than annual.
Can you confirm that there is no procedure for rerunning these models and updating the conclusions when input parameters that are impacted by, say, an unanticipated global recession vary?
3. Your response informed me that the National Air Passenger Allocation Model (NAPAIM) is a highly complex tool written in the C++ programming language which requires specialist skills to operate, understand, and integrate (via another programming language) with other parts of the suite.
I have received no information about what this other programming language is, or the programming languages and software platforms for the other 10 models I listed in my request.
It is manifestly reasonable to expect the DfT to know what are the languages and software platforms they have developed these important and very expensive assets on.
For example, if half of them are written on the Java programming platform, then International Networks Analysis and Support (INAS) managers must know that they need to retain a minimum core of Java trained developers on the staff -- particularly in light of the answer to 4. where it was explained to me that the software is not in an easy to maintain state and with no written step-by-step guides.
Please supply me with the complete answer to my request.
4. If the software code is maintained in such a state that it cannot be re-installed independently and rerun with a minimum of effort, please supply me a copy of the software in the actual state it is.
I have not provided to you my CV, but I can assure you that I probably have enough of a level of programming expertise to read and derive truthful conclusions from the raw form of the code. As you will be backing up the code anyway (to avoid data loss or computer failures), providing me with the code for each, any or all of these data models should incur less cost than drafting reasons for not releasing this data.
Please consider this letter as a request for an internal review.
Yours faithfully,
Julian Todd
Mr Todd
Please see the attached letter in response to your request for a review of
the response to your information request E0006237.
Tom Worsley
TOM WORSLEY
Deputy Director, Network Analysis & Modelling Division (NAM)
RNN, Department for Transport
Zone 4/33, Great Minster House
76 Marsham Street, London SW1P 4DR
Tel: 020-7944 4880
Fax: 020-7944 2160
email: [1][email address]
Tony Bowden left an annotation ()
With respect to the argument that "my review leads me to conclude that release of the software might be in the public interest only if you were to be provided with the support to enable you to carry out well informed model runs", the Decision Notice at http://www.informath.org/apprise/a3900/b... may have some relevance, particularly at paragraph 45:
"QUB also emphasised that even if a copy of the raw data was produced for the complainant, it would be meaningless to the requestor and could not be put to any meaningful purpose. However the Commissioner does not consider this to be a valid consideration when assessing information for possible disclosure under the EIR. There is no requirement for an applicant to demonstrate how they would be able to use any information provided."
Dear Tom Worsley,
Thank you for your comprehensive review of my FOI request for the software code of the air transport models.
I am a little confused by the consistency of some of the answers.
Firstly, if the Department has commissioned a contractor to develop and maintain some software then the code is the property of the Department, unless otherwise specified. There is no reason to believe it is otherwise specified since Scott Wilson Group Plc does not appear to be in the software selling business.
Secondly, 9 out of the 12 models run on Excel VBA. I happen to have a copy of Excel with VBA, so being able to rerun the models would likely require a minimum of effort were I provided copies of the files, since this platform doesn't require any configuration. If they were all in FORTRAN, it would be another matter.
Thirdly, the reasoning about the requirement to provide support for use of this software has been exposed in Paragraph 45 of decision notice FS163282 (29 March 2010):
"QUB also emphasised that even if a copy of the raw data was produced for the complainant, it would be meaningless to the requestor and could not be put to any meaningful purpose. However the Commissioner does not consider this to be a valid consideration when assessing information for possible disclosure under the EIR. There is no requirement for an applicant to demonstrate how they would be able to use any information provided."
Thank you for your time.
Yours sincerely,
Julian Todd
Dear Julian Todd
Thank you for your e-mail of 24th May. I'm replying in order to clear up a possible misunderstanding on your part in respect of computer models which have been commissioned by the Department.
A consultant commissioned to develop a model on behalf of the Department will often use in that model proprietary code developed for some other application by that consultant and for which neither the Department nor any other UK public sector organisation has been the client. Consultants will often develop software which facilitates the data manipulation processes inherent in all transport modelling methods in order to put themselves in a stronger position to bid for modelling work. The use of existing proprietary software in the model reduces the cost of delivery to the Department and, if the software has been tried and tested, increases our confidence in the reliability of the relevant parts of the model. It would not be efficient to develop every application from scratch.
As I explained in my letter which set out the findings of my review, I sought an estimate of the cost of cleaning the code to remove all proprietary coding and taking the code off Scott Wilson plc's computers and writing on to disks for transmission to you. For these activities, the Department would be charged for one and a half days of the consultant's time which is considerably in excess of £600.
I hope that this clears up the confusion that my letter left you with.
Yours sincerely
Tom Worsley
Tony Bowden left an annotation ()
Aside from any other consideration, they've fundamentally misunderstood how costs are to be calculated.
According to the Fees Regulation Guidance ( http://www.ico.gov.uk/upload/documents/l... ):
"An authority can take into account the costs attributable to the time that persons (both the authority’s staff and external contractors) are expected to spend on these activities. Such costs are calculated at £25 per hour per person for all authorities regardless of the actual cost or rate of pay,"
This means that the one and half days that it would for that external consultant fall easily within the limit, and they should therefore supply the information.
Julian Todd left an annotation ()
Complaint to the Information Commissioner has been send in today. The complaint summary is as follows:
"I requested details about and the source code for the 11 economic and environmental models referenced in a recent government aviation report, and was told that these pieces of software were highly complex – for example written in C++ programming language, which requires specialist skills to operate, understand, and integrate via another programming language. The information was considered exempt because it would exceed the cost allowances if they had to train me to use it.
"But according to FS163282, "There is no requirement for an applicant to demonstrate how they would be able to use any information provided."
"Furthermore, the claim about the incomprehensibility of the information was deliberately misleading, because following my complaint I found that 9 of the 11 models were in Excel/VBA which is a standard software platform accessible to anyone who has a copy of Microsoft Office.
"The software code is now considered exempt owing to the cost of cleaning out the proprietary coding that may have been included by the consultants Scott Wilson plc in the software models. However, it is highly unusual for consultants to have any proprietary claim of code which they developed under contract for a customer using their advice and expertise, unless that code is part of their other businesses. There is no evidence that this software which they appear to have written under the pay of the Department is worth anything to them as part of another product. Evidence would be available in the terms of agreement with the contractor, an arrangement to share profits made following investment of expertise by the DfT into the development of the software, or advertising for similar products by this consultancy."
Dear Mr Todd,
Please find attached a letter relating to your complaint made to the ICO
on the 26^th June.
Yours sincerely,
Nathan
Nathan Hill
Assistant Economist
International Networks Analysis & Support Branch 2
020 7944 6608
Dear Nathan Hill,
I look forward to receiving as much as you can disclose of the codes for the 12 economic computer models (at least 9 of which are in Excel VBA, a platform I certainly have access to) that are used as the basis for national environmental decision making.
I am sure you are aware that software Intellectual Property rights which you say belong to the contractor are fully protected under copyright law no matter what form they are published in. They cannot be copied and reused by third parties without prior written permission, therefore their economic interests are fully protected.
I hope that for the software models that you do not disclose you are able to provide satisfactory evidence that the copyright of them is indeed held by the contractor, rather than the client, as is normally the case in industry, and that these products are in fact advertised elsewhere for sale to other clients.
There is little use in me running the models with my own sets of data, because the outputs will be deemed invalid. The purpose of disclosure is to enable public participation in this decision-making, which at the moment can be no deeper than to face the results of these computer models.
I already know that the models are run too in frequently to take account of the timing of, for example, the world-wide recession. There may be other short-comings in these models, only visible through inspection of the code, which a client that has a presumption in favour of aviation growth is liable to overlook.
Yours sincerely,
Julian Todd
[Subject only] RE: Codes for DfT aviation modelling suite and input assumptions
Dear Mr Todd,
Please find attached the codes for the DfT aviation modelling suite
together with its input assumptions. There are 35 files of inputs and VBA
code in Excel spreadsheet and a further 112 files of the NAPALM C++
software code. There is also an accompanying note to briefly introduce the
files provided. These files will be sent via 3 separate emails.
Yours sincerely,
Nathan Hill
Nathan Hill | Aviation Forecasts and Modelling Expert | ISE Group
Analytical Support | 020 7944 6608 | [1][email address]
[Subject only] RE: Codes for DfT aviation modelling suite and input assumptions
Dear Nathan Hill,
Thank you for sending me copies of the DfT's air transport software models on 21 January. In your letter you stated:
"If we adopt a wide interpretation of your request to include, in addition to the aviation models, the code for the software model used to extract user surface transport access to airport costs... [as it is a] proprietary package that has been used numerous times by Scott Wilson for many different applications and clients."
Referring back to the responses to my initial request of 23 February 2010 I note the existence of the National Air Passenger Allocation Model (NAAM) which is written in: "SATURN, MINITRAMP or EMME/2 for skims; then bespoke FORTRAN77 then Excel VBA interfaces"[1]
Is this the model that relates to the user surface transport access to airport costs? I can tell I have not been sent it as there is no FORTRAN77 code in any of the packages that have been emailed to me.
Can you outline what other clients and applications this package has been used for by Scott Wilson? For example, is it used to plan the location of, in particular, airports in other countries, or does it have a use outside this sector?
One final comment: My inquiry was made on the basis of the DfT publication: "UK Air passenger demand and CO2 forecasts 2009". Nowhere does it indicate in those pages that these calculations are not the produced by or owned by the government.
[1] http://www.whatdotheyknow.com/request/29...
Yours sincerely,
Julian Todd
Case Reference Number: FER0319698
DfT Reference: E0006237
Dear Ms Humphries
Please find attached a response to your letter of 24 January. As
requested, I am copying in Mr Todd.
Information Rights Unit
Department for Transport
Dear Peter Parr,
Thank you for explaining the provenance of the User surface access model -- as being derived from Wootten Jeffreys software bought by WS Aktins plc in 1994 and provided a secondary interface and transport model network by Scott Wilson in 1998-1999.
This provides me with the first confirmation of a boundary between the Department of Transport and the private sector aviation/construction industries.
Before I take up the offer of purchasing elements of this software from WS Atkins, can you provide me with a name and description of this product and what computer equipment I will require to run it?
Also, the name "User surface access model" suggests that this is a tool for the aviation industry (for all other sectors "surface" is definitely the default and would not be mentioned in this way).
Can you confirm that Scott Wilson has provided use of this secondary interface and transport model network to more clients than it has the DfT's NAPAlM/Spasm software? If the interface is provided on the same terms as the the rest of the software suite, then it should be disclosed in the same manner.
Thank you for consideration of these questions in helping me get to the bottom of this. As Mr Hill explained in his letter on 14 January 2011,[1] there is an extremely large gap in the DfT's modelling capabilities. By publicising the technical steps and gaps in what the Department is or is not able to do for itself, it cannot but help create an environment for alternative suppliers to come forward.
[1] http://www.whatdotheyknow.com/request/29...
Yours sincerely,
Julian Todd
Dear Mr Todd,
Thanks for your email.
The "FORTRAN77" refers to the interface programs that Scott Wilson developed for other clients outside of the aviation sector. All the FORTRAN 77 programs are part of NAAM, which was not developed for DfT. This is the reason why there was no FORTRAN77 code in any of the files that were disclosed to you. NAAM is the National Airport Accessibility model, not the National Air Passenger Allocation Model as you stated in your email. All code and inputs into the National Air Passenger Allocation model were disclosed to you on the 21st January.
I do not have the answer to your question on the other clients and applications this package has been used for by Scott Wilson. I do not hold any information on Scott Wilson's specific clients and applications; therefore I am unable to answer that query.
I have received your latest email dated 7th February. I will respond to that email in the next few days.
Yours sincerely
Nathan
Dear Mr Todd,
The software you would need to purchase from WS Atkins is the "MINITRAMP" suite of programs. I'm advised by Scott Wilson that it would need to be the version which was prepared by WS Atkins for use by Scott Wilson in 1996. This version of MINITRAMP also requires a DBOS memory extender and to the best of my knowledge this version only operates on PCs running Windows 98/95. DBOS versions were not compatible with Windows2000 onwards.
The name "user surface access model" refers to NAAM which stands National Airport Accessibility Model developed by Scott Wilson c. 1998. This was a version of an all-purpose multi-modal district based transport model or set of models which worked out accessibility from any node or zone in the network (e.g. park and ride sites, stations, airports). I understand that versions of this model were used for transport planning purposes by consultants and was referred to as NAM (National Accessibility Model).
I can confirm that the Scott Wilson MINITRAMP interface software has been supplied to, and developed on behalf of at least one other client and not just for the DfT's aviation modelling suite.
Yours sincerely,
Nathan Hill
T Henderson left an annotation ()
It may be of interest that DfT published their 2011 forecasts this week.
Included was a peer-review of NAPALM, a response to the peer-review of NAPALM and another peer review of the peer review !!
http://www.dft.gov.uk/publications/uk-av...
An extract from the peer review agrees with some of the points made in connection with the original request :
At a higher level, the expert Peer Review has made even clearer than before how immensely complex is the model and how much it is an inheritance from a long history of incremental development. The Peer Review’s “strong plea for more documentation” is addressed in the Response’s paragraph 4.1. However the Peer Review concludes that plea with the comment that “These remarks should also be taken to refer to the program code, which I [i.e. the expert peer reviewer] suspect would benefit from a major overhaul before the details become impenetrable”. The possible need for “root and branch ‘re-engineering’” is accepted in the Response (paragraph 3.19) in the context of the algorithm for achieving capacity restraint (with which most of the code is associated) and I understand this applies more generally to all the code.
I wonder if the original request helped to persuade DfT to shine some light on the model and get an expert to review the software this time rather than rely just on their usual peer reviewer.
T Henderson left an annotation ()
Reading the DfT response to the review of NAPALM, I found that there is a welcome and imminent review of the model documentation about to take place :
It is accepted that the technical documentation of not just NAPALM but also its associated models in the Air Passenger Demand & CO2 Forecasting Appraisal System needs to be refreshed. We therefore propose to include provision for a full update of the technical documentation in the next (and imminent) contract specification.
Summary Response: The Department has commissioned
updated documentation to be prepared later in 2011.
(Section 5.1 of
http://assets.dft.gov.uk/publications/uk...
)
T Henderson left an annotation ()
Related to this request is my request at
http://www.whatdotheyknow.com/request/te...
DfT have provided a draft copy of a version of the "Rules" documenting the operation of the model. There are various warnings about the current state of this issue but the explanations given are generally helpful.
We work to defend the right to FOI for everyone
Help us protect your right to hold public authorities to account. Donate and support our work.
Donate Now
T Henderson left an annotation ()
It may be of interest that some of the spreadsheets involved in the modelling were released in reply to my request
http://www.whatdotheyknow.com/request/he...
I believe that some were in original Excel format showing the cell formulae and others were just copies of the spreadsheet showing the values for one run of the program.
It may also be of interest that the reply to my recent request
http://www.whatdotheyknow.com/request/up...
included a comment to the effect that the old version of the model was unable to deal with the magnitude of the changes imposed by last summer's changes to the carbon price (necessitating a crude preliminary "off-model" analysis). Presumably further developments are in hand to develop the model for the next release of the Passenger demand forecast.