DTD

From Open Rail Data Wiki
Revision as of 12:32, 21 February 2019 by PeterHicks (talk | contribs) (Add links to RSP documentation and reformatted text slightly)

Train companies create various types of data associated with the provision of their rail services, including fares, timetable and routeing data. They use this data to provide rail travel information to their customers in a variety of ways, including through approved websites, via mobile phone apps and at their stations through accredited retail systems.

RDG is authorised to make this data available under licence to third parties on behalf of the train companies. Fares, Routeing guide and Timetable data is of a technical nature, and you will require a good level of computer skills in order to be able to process the data.

Fares data consists of product information relating to permanent fares offered by the train operating companies. The data will typically include general fares information such as price, validity, class of travel and restriction information.

Routeing data is created from the information used to specify route validity for certain fares and prescribed customer journey route options.

Timetable data consists of the train service timetable information provided by the train operating companies. The data includes general timetable information such as departure and arrival times for all stations at which a train calls on its journey (e.g. origin, intermediate and terminating).

Accessing Fares, Routeing Guide and Timetable data

To access Fares, Routeing Guide and Timetable data you must have an account on the National Rail Data Portal (NRDP) at http://opendata.nationalrail.co.uk. By creating an account, you can register for a subscription to the Fares, Routeing and Timetable feed.

Important - Please note NRDP accounts expire after extended periods of no use. The unused account expiry period is currently set to 30 days. If you create an account and do not consume any of the feeds during this time your account will be deleted. If your account has been deleted, you will receive a notification email, and you will be able to re-register for a new account.  

Advice on downloading data

Best practice is to check no more than daily for new data.

To determine which file is the latest version, each files header contains the last updated timestamp (response HTTP header, Last-Modified).

NRDP will publish data as received from RDG, on a weekly basis. The data is updated at varying rates.

  • Fares, Routeing Guide and Timetable data - At selected times of the year, RDG will temporarily suspend Fares data publishing. NRDP will continue to make last retrieved file available to NRDP users.
  • Routeing data – weekly
  • Timetable data – Files are rarely updated and Users are advised to poll for updates infrequently

Examples

After selecting the Fares, Routeing Guide and Timetable data subscription type on NRDP, you can receive the files in 2 steps via a rest client by performing the following steps.

  1. Authenticate: Perform HTTP POST call to NRDP via https://opendata.nationalrail.co.uk/authenticate to generate a token.
  2. Download file: Perform HTTP GET call to individual NRDP Feed using the token produced in step 1

Step 1: Generate Token

In this step, using a REST client you perform an HTTP POST call to https://opendata.nationalrail.co.uk/authenticate. This step requires using your email address and password registered on NRDP and will produce a token key. The following example shows the HTTP POST request and response with NRDP:

Example HTTP POST Request to NRDP: 

POST https://opendata.nationalrail.co.uk/authenticate HTTP/1.1
Content-Type: application/x-www-form-urlencoded
Accept: application/json, text/plain, */*
Raw payload: username=user1@gmail.com&password=P@55w0rd1

Note: Raw payload values should contain your NRDP credentials

Corresponding example HTTP Response from NRDP: 

{
  "username": " user1@gmail.com ",
  "roles": {
    "ROLE_USER": true,
    "ROLE_DTD": true
   },
   "token": "user1@gmail.com:1491312310772:56c56baa3e56d35ff0ede4a6aad1bcfb"
}

Each token is valid for 1 hour and you will receive a 401 Unauthorized response when the token has expired.

The following HTTP response example is generated after sending an HTTP request to a NRDP feed with an expired token.

Example HTTP 401 Response Header: Token Expired: 

Date: Thu, 22 Nov 2018 13:39:58 GMT
Server: Apache/2.4.6 (CentOS) OpenSSL/1.0.2k-fips
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Pragma: no-cache
Expires: 0
X-Frame-Options: DENY
Content-Type: application/json;charset=UTF-8
Transfer-Encoding: chunked

Corresponding example JSON Response: Token Expired: 

   {
       "timestamp": "2018-11-21T12:34:33.377+0000",
       "status": 401,
       "error": "Unauthorized",
       "message": "Unauthorized",
       "path": "/api/staticfeeds/2.0/routeing"
   }

 

Step 2: Perform a HTTP GET call for each file

In this step using a REST client, you would perform a HTTP GET call to each Fares, Routeing Guide and Timetable data files using the token key produced in step 1. This will produce the zip file containing fixed format flat text file(s) for the corresponding Fares, Routeing Guide and Timetable data feeds.

The following are the required links to make the respective HTTP GET call:

The following is an example of request/response for Routeing data via NRDP:

Example HTTP GET Request to NRDP feed: 

GET https://opendata.nationalrail.co.uk/api/staticfeeds/2.0/routeing HTTP/1.1
Content-Type: application/json
Accept: */*
X-Auth-Token: user1@gmail.com:1491312310772:56c56baa3e56d35ff0ede4a6aad1bcfb

Note: X-Auth-Token header value should contain the token from the authenticate response

Corresponding example HTTP 200 Response Header: 

Date: Thu, 22 Nov 2018 13:53:00 GMT
Server: Apache/2.4.6 (CentOS) OpenSSL/1.0.2k-fips
Last-Modified: Wed, 21 Nov 2018 05:13:43 GMT
Content-Disposition: attachment; filename="RJRG0541.ZIP"
Accept-Ranges: bytes
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Pragma: no-cache
Expires: 0
X-Frame-Options: DENY
Content-Type: application/zip
Content-Length: 66021837

Example corresponding fixed format response files:

Each feed returns a zip file with numerous fixed format flat text files. An index file is also provided. For Timetable and Fares data this is the DAT file, for Routeing this is the RGI file.

Example Routeing RGI file (Unzipped): 

   /!! Start of file
   /!! Content type:  index
   /!! Sequence:      0541
   /!! Generated:     07/11/2018
   /!! Exporter:      RJIS RjEhrNRG v0.1
   RJRG0541.RGK
   RJRG0541.RGV
   RJRG0541.RGA
   RJRG0541.RGL
   RJRG0541.RGB
   RJRG0541.RGM
   RJRG0541.RGX
   RJRG0541.RGC
   RJRG0541.RGN
   RJRG0541.RGY	
   RJRG0541.RGD
   RJRG0541.RGE
   RJRG0541.RGP
   RJRG0541.RGF
   RJRG0541.RGG
   RJRG0541.RGR
   RJRG0541.RGH
   RJRG0541.RGS
   /!! End of file (18 records) (07/11/2018)

Example portion of Routeing fixed format file RGRJ0541.RGX (Unzipped) 

   /!! Start of file
   /!! Content type:  new stations
   /!! Sequence:      0541
   /!! Generated:     07/11/2018
   /!! Exporter:      RJIS RjEhrNRG v0.1
   AUR,DAG,01011999,31122999
   AYS,AVP,27112008,31122999
   BCV,SVS,27032014,31122999
   BDI,LMR,07042017,31122999
   BGN,LWM,11052005,31122999
   BGN,RIA,11052005,31122999
   BHG,ARM,09092010,31013000
   BKP,CNP,01011999,31122999
   BON,HWI,21012000,21012999
   BTR,BTP,13112000,13112999

Details on how to interpret the fixed format files can be found in specifications listed below.

Support

If you would like more details on the data, see these references

Retrieved from "https://wiki.openraildata.com/index.php?title=DTD&oldid=2463"