Dear Network Rail Limited,

My request for information is about providing metadata for the TD feed. In my specific use case I need sufficient methods to identify individual railway signals that appear in the TD feed in two forms: as a signal berth or as a signalling bit, of which items actually appear in the TD feed depends on the underlying technology of that area. To this date I have been experimenting with the limited amount of information I have gathered from various sources. One example is a method to predict train performance and locate trains on the line. To support developing actual applications, I would need two sets of metadata:

1) Mappings of signal berths provided by C-class messages to railway signals, containing at least this information:
- TD area
- berth identifier
- signal number

2) Mappings of data bits that appear in S-class messages to railway signals, typically held in a SOP file or ECS document, containing at least this information:
- TD area
- address
- data bit
- item description / signal number

There is no need for any additional information beyond this level. Also I would like to limit the search for records to cover only the Scottish region at this time.

This is the minimum level of detail one can work with. To my knowledge this level of detail is limited so that it shouldn't have major safety implications. Obviously, if it has, it doesn't make sense to publish the TD feed in the first place.

The disclosure of this information would allow a developer to monitor actual usage of signal blocks, routes between signals and changes in signal aspects. Further analysis of the interaction between the trains and signal system could provide valuable insight on the use of railway infrastructure. It could also help build tools to predict train performance with greater accuracy. For example, the real-time locations of trains is something that could fall in the public interest. Another example is an app that could tell users when a level crossing nearby is closed (though the level crossing status is available only for a few locations at the moment).

Without this information, I am not able to interpret the data in TD feed. To tell the truth, trying to use the noncontextual TD data is like being two year old again watching written words and, believe me, it's also as fascinating!

I can assure you that if eventually it is decided not to disclose the information - I am speaking hypothetically here - I can step outside the box and take data mining approach for missing metadata. I think I have at least three potentially useful approaches:
- Build an algorithm that compares berth identifiers with signal numbers from Train Planning Network Model (most, though not all, berths are numbered after signal numbers).
- Manually observe signals on the railway. Record the time when the aspect changes. Compare with signalling data. Repeat for each signal (requires a lot of train tickets and some signals are out of my reach).
- Build an algorithm that observes train movements at signal berth level and compares them with changes in signalling data to find respective bits. Then wait for month (or longer) until every signal has been passed by a train. Requires some manual validation of the data.

This would eventually result in almost complete coverage of the metadata I am requesting. But it takes serious amounts of time, tea, and biscuits. Also, my collected metadata could be out-of-date from time to time possibly providing false information. And that is a situation I want to avoid.

However, while I recognise that there is no requirement to create information, I would appreciate if Network Rail took action to build sufficient metadata for its open data. Poor metadata is an important problem with open data that is rarely discussed. In my opinion the TD feed in its present form does not make full use of its potential to all developers.

Yours faithfully,

Juhani Pirttilahti