Difference between revisions of "About the Network Rail feeds"

From Open Rail Data Wiki
Jump to navigation Jump to search
m (Update support email address)
(47 intermediate revisions by 12 users not shown)
Line 1: Line 1:
== What are the feeds? ==
+
== Available Feeds ==
  
Five feeds are available:
+
The following feeds are available from Network Rail:
  
* [[RTPPM]] - Real Time Public Performance Measure.  This shows the performance of trains against the timetable, measured as the percentage of trains arriving at destination on time.  Updated every minute.
+
{| class="wikitable"
* [[Train Movements]] - messaging from the TRUST system, containing reports of train movements past timetabled calling and passing points.
+
! Feed
* [[TD]] - data from the Train Describer system, showing raw data with train movements in more detail than the [[Train Movements]] feed.
+
! Type
* [[VSTP]] - late-notice train schedules which are not available through the [[Schedule]] feed.
+
! Rate
* [[TSR]] - Temporary Speed Restrictions
+
! Details
* [[SCHEDULE]] - extracts of train schedules from [[ITPS]].
+
|-
 +
|[[RTPPM]]
 +
|Real-time
 +
|1 per minute
 +
|Real Time Public Performance Measure.  This shows the performance of trains against the timetable, measured as the percentage of trains arriving at destination on time, and is updated every minute
 +
|-
 +
|[[Train Movements]]
 +
|Real-time
 +
|Up to 600 per minute
 +
|Messaging from the TRUST system, containing reports of train movements past timetabled calling and passing points.  
 +
'''Note:''' Messages are [[NROD Message Batching|batched]] to reduce network overheads.
 +
|-
 +
|[[TD]]
 +
|Real-time
 +
|Up to 1000 per minute
 +
|Berth-level data from the Train Describer system, showing raw data with train movements in more detail than the [[Train Movements]] feed.
 +
'''Note:''' Messages are [[NROD Message Batching|batched]] to reduce network overheads.
 +
|-
 +
|[[VSTP]]
 +
|Real-time
 +
|Low volume
 +
|Late-notice train schedules which are not available through the [[SCHEDULE]] feed
 +
|-
 +
|[[TSR]]
 +
|Low-volume
 +
|<10 per week on Fridays
 +
|Temporary Speed Restriction data as published in the Weekly Operating Notice
 +
|-
 +
|[[SCHEDULE]]  
 +
|Static
 +
|Daily
 +
|Extracts of train schedules from [[ITPS]] in JSON format
 +
|-
 +
|[[Reference_Data|Reference Data]]
 +
|Static
 +
|Infrequent
 +
|Reference data which can be used to help analyse other data feeds
 +
|-
 +
|}
  
 
== How do I get the data? ==
 
== How do I get the data? ==
  
First, sign up for an account at [http://datafeeds.networkrail.co.uk http://datafeeds.networkrail.co.uk].  Log in and select the feeds you wish to use.
+
First, register for an account by visiting [http://datafeeds.networkrail.co.uk http://datafeeds.networkrail.co.uk].  You will receive a confirmation email.  Follow the instructions to log in and change your password.  When your account is active, you can connect to the service.  Your account may be in [[NROD_Account_States|one of three states]] - the system will send you an email when your account is activated and able to access feeds.
  
You will need to write a [http://stomp.github.com/ Stomp] client, [http://stomp.github.com/implementations.html#STOMP_1_1_Clients available for many languages].
+
Once you have logged in to the system, the ''My Feeds'' tab lists all of the data feeds available.  You must select the topics that you wish to access. This can be done by highlighting them in the ''Available'' column and clicking the right arrow icon to move them to the ''Selected'' column. Once you have selected your chosen feeds, click the ''Update Subscription'' button. You will then be able to access those feeds.
  
Schedule Data, cannot be obtained via Stomp, data is obtained by GZ file download from the Amazon S3 Data Buckets, each Zip file consists of a collection of JSON strings.
+
== Accessing the data feeds ==
The data consists of a primary set of data (rather large, can be 1.5GB in size) and a set of daily corrections that should be applied to the base data.
+
 
 +
The real-time feeds are accessed using Stomp - details are on the [[Connecting_with_Stomp#Network_Rail|Network Rail connection details page]].
 +
 
 +
The static feeds are available via authenticated HTTP GET requests.
 +
 
 +
== Good Practice ==
 +
 
 +
Please bear in mind the following points when you use the service - they're here to make sure everyone gets a good service:
 +
 
 +
* '''Sign up with one account''' - it gives an accurate reflection of how many people are actually using the service.
 +
* '''Connect to the service once''' - you can receive multiple feeds in a single connection.  If you need to access the data more than once, you can do so - but be sensible!
 +
* '''Don't leave a failed client running''' - if you can connect but get an authentication or authorization error - '''stop your client''' - don't leave it continually trying.
 +
* '''Make sure your client handles failures''' - if your client fails to connect or is disconnected, use [http://en.wikipedia.org/wiki/Exponential_backoff an exponential backoff] and wait 1s, 2s, 4s, 8s, 16s etc. before trying to reconnect. Service problems sometimes occur, and if everyone tries to reconnect every second, the service will not recover quickly.
 +
* '''Use a durable subscriber''' - if you want data to be queued in case you disconnect, request a [[Durable Subscription|durable subscription]] when you connect.
 +
* '''Use [http://stomp.github.io/stomp-specification-1.1.html#Heart-beating Stomp heartbeats]''' to detect network problems that may otherwise leave your client hanging with no data
  
 
==Examples==
 
==Examples==
Example source code showing how to connect to the data feed and display the messages received in various languages.
 
  
[[PHP Example]]
+
Code examples for various languages are available from the [[Example_Code | Example Code]] page.
 +
 
 +
The [[Advanced Uses|advanced usage]] page contains examples of some advanced applications for the data feeds, including bridging the ActiveMQ feeds to your own messaging server.
 +
 
 +
== Support ==
 +
 
 +
If you are having problems with the feeds:
 +
 
 +
* First, read this wiki - there's a lot of material here that will help you
 +
* Check the [[FAQ]] to see if you have a common issue
 +
* Check [https://twitter.com/open_rail_feeds twitter] to see if an issue has been reported
 +
* If you want to discuss your problem with other people working with the service, the [https://groups.google.com/d/forum/openraildata-talk openraildata-talk] group on Google Groups will be useful
 +
* Finally, if you're still having a problem, email [mailto:dsg_nrod.support@caci.co.uk dsg_nrod.support@caci.co.uk]
 +
 
 +
== Background reading ==
 +
 
 +
It may be useful to understand the background to the sources of some of these data feeds.  These links will help:
 +
 
 +
* [http://www.aecom.com/deployedfiles/Internet/Geographies/Europe/Resources/Rail%20End%20to%20End/rail-end-to-end_stage-3-appendix_20100122.pdf Integrated Passenger Information: Delivering the Rail End to End Journey, Stage 3 - Technical Annex]
 +
* [http://www.networkrail.co.uk/browse%20documents/regulatory%20documents/regulatory%20compliance%20and%20reporting/licence%20documents/catalogue%20of%20railway%20code%20systems.pdf Catalogue of Railway Code Systems]
 +
 
 +
==Releases==
 +
 
 +
For further information on the content of releases please see the [[ReleaseNotes | release notes]].
 +
 
 +
{{Navtable-DataFeeds}}
  
[[C# Example]]
+
[[Category:Network Rail Data Feeds]]

Revision as of 11:10, 2 December 2018

Available Feeds

The following feeds are available from Network Rail:

Feed Type Rate Details
RTPPM Real-time 1 per minute Real Time Public Performance Measure. This shows the performance of trains against the timetable, measured as the percentage of trains arriving at destination on time, and is updated every minute
Train Movements Real-time Up to 600 per minute Messaging from the TRUST system, containing reports of train movements past timetabled calling and passing points.

Note: Messages are batched to reduce network overheads.

TD Real-time Up to 1000 per minute Berth-level data from the Train Describer system, showing raw data with train movements in more detail than the Train Movements feed.

Note: Messages are batched to reduce network overheads.

VSTP Real-time Low volume Late-notice train schedules which are not available through the SCHEDULE feed
TSR Low-volume <10 per week on Fridays Temporary Speed Restriction data as published in the Weekly Operating Notice
SCHEDULE Static Daily Extracts of train schedules from ITPS in JSON format
Reference Data Static Infrequent Reference data which can be used to help analyse other data feeds

How do I get the data?

First, register for an account by visiting http://datafeeds.networkrail.co.uk. You will receive a confirmation email. Follow the instructions to log in and change your password. When your account is active, you can connect to the service. Your account may be in one of three states - the system will send you an email when your account is activated and able to access feeds.

Once you have logged in to the system, the My Feeds tab lists all of the data feeds available. You must select the topics that you wish to access. This can be done by highlighting them in the Available column and clicking the right arrow icon to move them to the Selected column. Once you have selected your chosen feeds, click the Update Subscription button. You will then be able to access those feeds.

Accessing the data feeds

The real-time feeds are accessed using Stomp - details are on the Network Rail connection details page.

The static feeds are available via authenticated HTTP GET requests.

Good Practice

Please bear in mind the following points when you use the service - they're here to make sure everyone gets a good service:

  • Sign up with one account - it gives an accurate reflection of how many people are actually using the service.
  • Connect to the service once - you can receive multiple feeds in a single connection. If you need to access the data more than once, you can do so - but be sensible!
  • Don't leave a failed client running - if you can connect but get an authentication or authorization error - stop your client - don't leave it continually trying.
  • Make sure your client handles failures - if your client fails to connect or is disconnected, use an exponential backoff and wait 1s, 2s, 4s, 8s, 16s etc. before trying to reconnect. Service problems sometimes occur, and if everyone tries to reconnect every second, the service will not recover quickly.
  • Use a durable subscriber - if you want data to be queued in case you disconnect, request a durable subscription when you connect.
  • Use Stomp heartbeats to detect network problems that may otherwise leave your client hanging with no data

Examples

Code examples for various languages are available from the Example Code page.

The advanced usage page contains examples of some advanced applications for the data feeds, including bridging the ActiveMQ feeds to your own messaging server.

Support

If you are having problems with the feeds:

  • First, read this wiki - there's a lot of material here that will help you
  • Check the FAQ to see if you have a common issue
  • Check twitter to see if an issue has been reported
  • If you want to discuss your problem with other people working with the service, the openraildata-talk group on Google Groups will be useful
  • Finally, if you're still having a problem, email dsg_nrod.support@caci.co.uk

Background reading

It may be useful to understand the background to the sources of some of these data feeds. These links will help:

Releases

For further information on the content of releases please see the release notes.


Network Rail Open Data Feeds
Data Feeds About the Feeds Account States Durable Subscriptions Example Code ( PHP / C# / Java / Ruby / Node.js) • Advanced UsesFAQ Release Notes
RTPPM RTPPM Feed
Train Movements Train Movements Feed Train Activation Train Cancellation Train Movement Train Reinstatement Change of Origin Change of Identity Change of Location TSPEED Field Planned Cancellations Cancellation Codes
TD TD Feed C-Class Messages S-Class Messages Train Describers TD Berths
VSTP VSTP Feed
TSR TSR Feed Route Codes
SCHEDULE SCHEDULE Feed Schedule and Location Records Association Records CIF Codes How Scheduling Works Allowances
Reference Data Reference Data Feed TOC Codes CIF Codes Delay Attribution Codes Identifying Locations (STANOX, TIPLOC, NLC and 3-Alpha Codes) STANOX Geographical Areas Train Planning data