One million Barclays Cycle Hire journeys data
Dear Transport for London,
You have announced today that the first million Barclays Cycle Hire journeys have been completed. Congratulations on reaching this milestone so soon.
Please could you send me a datafile describing those journeys. I would like it as a CSV file that contains a header row followed by one million rows of data.
The data file should include these columns:
Journey ID
Bike ID
Time and date of the start of the journey
Time and date of the end of the journey
Origin docking station ID
Destination docking station ID
The docking station IDs should be consistent with the data served from your web map as used by third-party APIs such as my own:
Otherwise, please also send a datafile containing the docking station IDs and names you are using for this request.
I'm happy for you to upload this file to a public website for me to download as long as you send me the link. It would be unwise to send it as an email attachment.
You may, at your option, compress the file using bz2 or zip. I would prefer the former.
You may split the data into a reasonable number of separate files if it is easier for you to deliver them in that way, provided that they can all be batched together as a single download.
Thank you.
Yours faithfully,
Adrian Short
Our Ref : 1007690467
Date : 11.10.2010
Mr A Short
[FOI #49018 email]
Dear Mr Short
Thank you for your request for information dated 8 October 2010 which was
received by Transport for London on 8 October 2010 concerning Barclays
Cycle Hire. This request will be dealt with under the terms of the
Freedom of Information Act 2000.
We aim to deal with your request promptly and provide you with a response
by 5 November 2010.
In the meantime, if you have any queries or would like to discuss your
request, please do not hesitate to contact me.
Once again, thank you for taking the time to contact us.
Yours sincerely
Patricia Mitchell
Head of Customer Services, Surface Transport
DO NOT DELETE.............................
{ticketno:[1233752]}
DO NOT DELETE.............................
Our Ref: 1007690467/GH
Date: 05.11.2010
Mr A Short
[1][FOI #49018 email]
Dear Mr Short
Thank you for your email received by Transport for London (TfL) on 8
October 2010. You asked for data regarding the first million Barclays
Cycle Hire journeys.
Your request has been considered under the requirements of the Freedom of
Information Act and I can confirm that TfL does hold the information you
require. Please find attached a selection of the data you have requested:
the attachment to this email shows only 100 journeys, as the complete file
is too large to email – however, the full data will be uploaded directly
to the TfL website in csv format and we will let you know as soon as this
has been done.
This data is the first one million cycle journeys recorded from 6:00am on
30 July 2010. Please note the spreadsheet excludes 180 journeys which do
not have an end station and also journeys under three minutes which
started and ended at the same docking station as these would most likely
be mis-hires and not true journeys.
If this is not the information you are looking for, or if you are unable
to access it for some reason, please contact Graham Hurt, Correspondence
Support Manager – London Streets at [2][email address] .
Please see the attached information sheet for details of your right to
appeal as well as information on copyright and what to do if you would
like to re-use any of the information we have disclosed.
Yours sincerely
Samantha Clarke-Jackman
Customer Service Manager – Group Customer Services
DO NOT DELETE.............................
{ticketno:[1233752]}
DO NOT DELETE.............................
Ben Lewis left an annotation ()
Open Data Hack Day sounds fun.. http://www.opendataday.org/wiki/City_Eve...
..where can I find out more?
Is there an offline meet in London? @Hacklabs?
Cheers
Adrian Short left an annotation ()
TfL say that they have a "target date" of 14 December to put this data in their Developers' Area.
https://www.tfl.gov.uk/tfl/businessandpa...
I'm asking that they publish the data sooner (it's already nearly a month late) and don't require registration for me to get an answer to my FOI request.
Andy Cotgreave left an annotation ()
Shame that you need to register. Also, I don't have an IP address (required field for registration). If it's just a CSV, I want to download and save it via the browser - know any way around that?
Dear Transport for London,
Please pass this on to the person who conducts Freedom of Information reviews.
I am writing to request an internal review of Transport for London's handling of my FOI request 'One million Barclays Cycle Hire journeys data'.
Your response to this request is a very long time overdue with no explanation for the delay being offered. To produce the data required would require only a single database query and uploading a compressed datafile to a public web server or delivering it on a disc in the post.
A full history of my FOI request and all correspondence is available on the Internet at this address:
http://www.whatdotheyknow.com/request/on...
Yours faithfully,
Adrian Short
Dear Mr Short
The publication of this information is due to 'go live' on the Developers area of the TfL website later this week and I will let you know as soon as it is available. I am sorry this has taken so long.
I would be grateful if you would let me know whether you still require a review.
Kind regards
Matthew Towey | Information Governance Adviser
Information Access & Compliance Team | Corporate Governance | Transport for London
5th Floor, Windsor House, 42-50 Victoria Street, London SW1H 0TL
T: 020 7126 3063/ auto 63063
Dear Mr Towey,
I do not object to you putting this data in your developers' area but I would like to be able to access it directly without having to register on your website. Please can you send me the data directly or otherwise ensure that this is possible.
Please proceed with your internal review as I can see no reason why something which shouldn't take more than a couple of staff hours has taken (to date) nearly three months.
Yours sincerely,
Adrian Short
Dear Mr Short
The data is now available for download from the Developer's area http://www.tfl.gov.uk/businessandpartner...
I note your preference to be sent this data directly without having to register. This will be considered as part of the internal review, which should take place next week.
Kind regards
Matthew Towey | Information Governance Adviser
Information Access & Compliance Team | Corporate Governance | Transport for London
5th Floor, Windsor House, 42-50 Victoria Street, London SW1H 0TL
T: 020 7126 3063/ auto 63063
Dear Mr Towey,
Please stop wasting my time and money and just send me the data.
The FOIA is very clear that requests should be considered on an applicant- and motive-blind basis. You have no right to refuse to supply information simply because you don't like the person requesting it or might not approve of how they would use it provided that they stay within the law.
Your registration process is designed to filter applicants and therefore there is a possibility (in this case one might suspect a likelihood) that you might refuse my registration and deny me access to this information which you have already conceded is suitable to be supplied to third-parties outside TfL.
Just for a bit of context -- many of the apps that have contributed to the success of the Barclays Cycle Hire scheme are built using code and data that I've supplied free and without strings attached even though Serco and TfL have yet to provide any live data to help people find bikes and free docking points. I find it very hard to square the mayor's rhetoric about opening up London's data with your entirely pointless and counter-productive obstruction of third party developers and data analysts in this case.
I consider that the current status of this request is that you have refused to supply the information requested in clear contravention of the law and the Information Commissioner's guidance:
http://www.ico.gov.uk/upload/documents/l...
Yours sincerely,
Adrian Short
Alex Skene left an annotation ()
This is the link to the zip file with the journey data:
http://www.tfl.gov.uk/tfl/syndication/fe...
Regards
Alex - WhatDoTheyKnow volunteer
M Deutsch left an annotation ()
Oliver O'Brien has presented this data in pretty map form: http://oobrien.com/vis/bikes/flow.php
Adrian Short left an annotation ()
Here's a viz by Matthew Clark at Steer Davies Gleave:
http://www.steerdaviesgleave.com/news-an...
Adrian Short left an annotation ()
This data gets a mention in the Telegraph today:
http://www.telegraph.co.uk/finance/newsb...
Francis Irving left an annotation ()
I'm confused - that BarclaysCycleHireUsageStats.zip that Alex links to doesn't require registration to download.
Aren't they therefore providing the data now?
I don't see that it needs to be sent as an attachment, rather than a link.
Adrian Short left an annotation ()
I certainly don't mind whether the data is delivered as a link rather than an attachment. In fact I'd far prefer it.
But TfL's position here as I understand it is that anyone who wants to use the data must register, accept their terms and conditions and be approved by them.
The fact that they may have lax security which means that someone who has registered can publish the private URL of the data doesn't change its legal position. And a person doing that most probably breaks their agreement with TfL too:
"2.1.3 You shall not make the Transport Data feeds available to any third parties, save with TfL's prior written consent;"
If TfL is happy for me to access the data directly from that URL without accepting their terms then I will consider this matter closed.
Adrian Short left an annotation ()
Here's my view on the current status of this request:
http://adrianshort.co.uk/2011/01/07/tfls...
Adrian Short left an annotation ()
Here's a story about this data on the Guardian's data blog:
http://www.guardian.co.uk/news/datablog/...
Adrian Short left an annotation ()
This is now on the BBC's Mind the Gap transport blog:
http://www.bbc.co.uk/blogs/mindthegap/20...
Adrian Short left an annotation ()
Two more mentions today:
Big Brother Watch:
http://www.bigbrotherwatch.org.uk/home/2...
TechEye.net:
http://www.techeye.net/business/transpor...
Adrian Short left an annotation ()
Two more mentions:
The Register:
http://www.theregister.co.uk/2011/01/11/...
Public Service:
http://www.publicservice.co.uk/news_stor...
Alex Skene left an annotation ()
From Big Brother Watch:
"a Transport for London spokesman said '[...] The reason this information is provided on the [TfL Developer] website is to provide value for money to London taxpayers and passengers. Saving information onto hard discs and sending securely through the post for one person is less cost efficient than making the same data available on the web for a large number of people'."
The data is only 25MB in size (it's a zipped text file), and WDTK can easily handle email attachments this size.
I don't understand why they mention sending out the data on CD via post as they've not proposed this method in their responses to this request.
We would ask authorities to get in touch with the WDTK team to discuss the best way of sending the data if attachments are over 50MB.
http://www.whatdotheyknow.com/help/offic...
M Deutsch left an annotation ()
Presumably they could have complied with the request by uploading the data to the TfL developers' site, and providing you with a direct URL to the data?
After all, there's nothing in the FOIA which requires disclosed data to also be freely published.
Adrian Short left an annotation ()
M Deutch,
You're quite right. As I've said many times, including in my original FOI request email at the top of this page, I'm happy for TfL to upload the data to a web server and send me the link. I don't care whether that link points to somewhere within or outside the developers' area as long as I don't have to enter into any agreement with them to do or not do anything outside my general rights and responsibilities under the law. I just want what every other FOI applicant expects and is due according to the FOI Act.
If TfL *also* wants to make the data available to other developers under restrictive terms that's their right to do so and those developers can decide for themselves whether that arrangement suits them.
27 January 2011
Our ref: IRV-095-1011 Short
Dear Mr Short
The internal review of your Freedom of Information request has been
completed.
You requested a review on the basis that TfL failed to deal with your
request within twenty working days. You have also requested that TfL send
the data to you directly as you do not wish to have to register on the
Developers’ area of the TfL website.
The review considered the way in which you request was handled and noted
that TfL responded to your request on 5 November 2010, within 20 working
days, providing a sample of the information you requested and stating that
the remaining information would be uploaded directly to the TfL website
and we would let you know once this had been done. The remaining
information was uploaded to the Developers area on 05 January 2011 and we
informed you on the same day.
The Panel considered whether TfL’s response of 5 November 2010 was in
accordance with the requirements of the Freedom of Information (FOI) Act.
The Panel agreed that TfL should have either provided the information
requested, in full, or applied one of the statutory exemptions to the
remaining information, in this case section 22 of the FOI Act, which
applies to information intended for future publication. In failing to
apply an exemption, TfL contravened the requirements of section 17 of the
FOI Act. On behalf of TfL please accept my apologies for our failure to
apply this exemption.
In terms of your comments on the way in which TfL has made this
information available it is important to note that the FOI Act applies to
access to information and not re-use. Disclosure of information under the
FOI Act does not automatically give the recipient the right to re-use
information. Accordingly, when responding to FOI requests where we
disclose information subject to copyright we inform individuals that
depending on the use they intend to make of the information they may need
to seek permission from the copyright holder.
The re-use of public sector information is subject to the Re-Use of Public
Sector Information Regulations. We are not obliged under the Regulations
to make public sector information available for re-use, but, if we do
allow re-use we are entitled to impose conditions on the re-use of the
information to ensure it is not used in a manner inconsistent with our
copyright.
We have chosen to make the information available via the Developers’
area of the TfL website. This information is available without charge
subject to our terms and conditions and we do not accept your contention
that there is a possibility that we would refuse your registration based
on who you are or what you would do with the data. We have noted the
comments you have made in relation to section 2.1.2 of our terms of
conditions. This condition is not designed to restrict reasonable
criticism of TfL and should not have that effect. For the avoidance of
doubt, however, we will inform our Group Marketing and Communications
Directorate, who are responsible for these terms and conditions, that this
condition should not be used to take action against individuals and
developers who wish to use the data made available to criticise TfL.
In terms of the current status of your request, the information you have
requested is now available via the TfL website. We consider that your
request is now satisfied as we have provided access to the information. We
will not be providing you with this information directly as this
information is reasonably accessible by other means (section 21 of the FOI
Act refers). We are pleased that members of the public are making use of
the information by registering via the TfL website.
I hope this information is of assistance. If you are dissatisfied with the
outcome of this internal review, you can refer the matter to the
independent authority responsible for enforcing the Freedom of Information
Act at the following address.
Information Commissioner’s Office
Wycliffe House
Water Lane
Wilmslow
Cheshire SK9 5AF
A complaint form is available on the ICO’s website ([1]www.ico.gov.uk).
Yours sincerely
Matthew Towey | Information Governance Adviser
Information Access & Compliance Team | Corporate Governance | Transport
for London
5th Floor, Windsor House, 42-50 Victoria Street, London SW1H 0TL
T: 020 7126 3063/ auto 63063
We work to defend the right to FOI for everyone
Help us protect your right to hold public authorities to account. Donate and support our work.
Donate Now
Andy Cotgreave left an annotation ()
I guess there's no way to chivvy this along? There's an Open Data Hackday on 4th December, and this would be a perfect data set to play with en masse that day.
Fingers crossed...
Andy