SCHEDULE: Difference between revisions

From Open Rail Data Wiki
→‎Data: Deletions packet example
EvelynSnow (talk | contribs)
http://datafeeds.networkrail.co.uk→https://publicdatafeeds.networkrail.co.uk
 
(45 intermediate revisions by 10 users not shown)
Line 1: Line 1:
==Overview==
The SCHEDULE feed is an extract of train schedules from Network Rail's ITPS (Integrated Train Planning System), available in [[JSON File Format|JSON]] and [[CIF File Format|CIF]] format.


The Schedule feed is an extract of train schedules from the Integration Train Planning System.
'''Note:''' If you are just starting out with the service use the JSON files.  The CIF data is more suited to advanced users of the service and requires additional parsing compared to the JSON data.


Schedule Data, cannot be obtained via Stomp, data is obtained by GZ file download from the Amazon S3 Data Buckets, each GZIP file consists of a collection of JSON strings.
= Overview =


The data consists of a primary set of data (rather large, can be 1.5GB in size) and a set of daily corrections that should be applied to the base data.
The schedule data consists of Train Schedules, Train Associations and details of Timing Point Location codes.


The data only contains Passenger Train Information.
A description of the scheduling process is available on the [[HowSchedulingWorks | how scheduling works]] page. This describes how multiple schedules can be used to describe a specific train service.
    Freight services are not included in the schedule; all messages containing FOC codes are filtered out.


== Obtaining the Data ==
= Usage =


Data is downloaded from Amazon S3. Each feed has a Bucket name and a File Name.
Trains are sets of schedules identified by a common ''UID''. A schedule for a specific train service can be uniquely identified by ''UID'', ''Start Date'' and ''STP Indicator''.


Each bucket has one or more files available within it. Normally the FULL_DAILY buckets will contain a single file (toc-full), where as the UPDATE_DAILY buckets will contain 7 files, one for each day.
== Types of file ==


Data is obtained from the Amazon S3 URL
There are two types of files:
    https://datafeeds.networkrail.co.uk/ntrod/CifFileAuthenticate?type=[bucket]&day=[file]
* A full extract, a snapshot of schedule database valid on that day, which can be used to initialise your schedule database
* An update extract is produced each night, which can be applied to the previous update extract to bring your database up-to-date


So for example
Update extracts will specify how the schedule database has changed. It is important that the update files are processed in sequential order. Amendments may be received for schedules that are currently valid, i.e. the schedule has started running but has not finished yet. Changes will replace any previous version of a train schedule. Deletions will not be raised for schedules that have become historic.
    https://datafeeds.networkrail.co.uk/ntrod/CifFileAuthenticate?type=CIF_ALL_FULL_DAILY&day=toc-full


Will give you the Full Schedule for All Regions for Today.
The following formats are available from SCHEDULE Feed:


You will need to be already logged into [https://datafeeds.networkrail.co.uk DataFeeds] in a Web Browser to obtain the data, or if using cURL, HTTP Basic Auth, following HTTP Redirects will login, (using your Email/Password, not your security key)
{| class="wikitable"
! Format
! Type
! Rate
! Details
|-
|rowspan="2"|[[JSON File Format]]
|Full Extract
|Daily
|rowspan="2"|The schedules in JSON format are available for all operators, or for individual operators.
|-
|Update Extract
|Daily
|-
|rowspan="2"|[[CIF File Format]]
|Full Extract
|Weekly
|rowspan="2"|The schedules in CIF format are only available for all operators.  A 'full' extract is available each Friday, and an 'update' extract is available daily.
|-
|Update Extract
|Daily
|-
|}


== Data ==
The JSON data files are generated overnight and will be available from around 0600.  The CIF files are go through a different process and will be available from around 0100.


Like the realtime data feeds, the Schedule data is split down into Train Provider and then from there down into the Full Schedule for that day and the daily updates.
'''Note:''' Late-notice train schedules, which are not available through the SCHEDULE feed, are published via the real-time [[VSTP]] feed.


So if you are building a local schedule database from scratch or are wiping your copy to build a fresh version.
== Downloading ==


* First download and process the Full Daily.
The schedule data is compressed using gzip and is downloaded from Amazon S3 via a private URL which is valid for a few minutes after generation.  Authentication will be required either by logging in through the Data Feeds website, or by sending your username and password as part of the HTTP request. On successful authentication, you will receive a HTTP 302 (Moved Temporarily) redirect to the location of the schedule file. Requests are of the format:
* And then daily grab the Daily update for that day and process that.


Files are normally updated around about Midnight UTC
  https://publicdatafeeds.networkrail.co.uk/ntrod/CifFileAuthenticate?type=type&day=day


A Daily Full file will only contain CREATE transactions, where as a Update can contain CREATE and DELETE transactions.
where ''type'' determines the TOCs contained in the schedule extract and whether the file is a full schedule or an update file, and ''day'' determines which day's update file is downloaded. For example:


Each file contains,
  https://publicdatafeeds.networkrail.co.uk/ntrod/CifFileAuthenticate?type=CIF_ALL_FULL_DAILY&day=toc-full


* a Data/information line,
would download a full schedule file in JSON format for all TOCs. And if the CIF format is what you are after, add a trailing .CIF.gz to get the CIF format.
* a set of Schedule/Train Associations,
* a set of Schedules
* an EOF message
* a blank line


For the Update files. DELETE transactions are listed before CREATE transactions normally.
Values for ''type'' and ''day'' can be one of the following:


Files are New Line Delimited JSON Packets
{| class='wikitable'
!''type''
!''day''
!Description
!Example URL (JSON format)
!Example URL (CIF format)
|-
|CIF_ALL_FULL_DAILY
|toc-full
|Daily all-TOC snapshot
|[https://publicdatafeeds.networkrail.co.uk/ntrod/CifFileAuthenticate?type=CIF_ALL_FULL_DAILY&day=toc-full Daily]
|[https://publicdatafeeds.networkrail.co.uk/ntrod/CifFileAuthenticate?type=CIF_ALL_FULL_DAILY&day=toc-full.CIF.gz Weekly]
|-
|CIF_ALL_UPDATE_DAILY
|toc-update-''DAY''
|Daily all-TOC update
|[https://publicdatafeeds.networkrail.co.uk/ntrod/CifFileAuthenticate?type=CIF_ALL_UPDATE_DAILY&day=toc-update-mon Mon], [https://publicdatafeeds.networkrail.co.uk/ntrod/CifFileAuthenticate?type=CIF_ALL_UPDATE_DAILY&day=toc-update-tue Tue], [https://publicdatafeeds.networkrail.co.uk/ntrod/CifFileAuthenticate?type=CIF_ALL_UPDATE_DAILY&day=toc-update-wed Wed], [https://publicdatafeeds.networkrail.co.uk/ntrod/CifFileAuthenticate?type=CIF_ALL_UPDATE_DAILY&day=toc-update-thu Thu], [https://publicdatafeeds.networkrail.co.uk/ntrod/CifFileAuthenticate?type=CIF_ALL_UPDATE_DAILY&day=toc-update-fri Fri], [https://publicdatafeeds.networkrail.co.uk/ntrod/CifFileAuthenticate?type=CIF_ALL_UPDATE_DAILY&day=toc-update-sat Sat], [https://publicdatafeeds.networkrail.co.uk/ntrod/CifFileAuthenticate?type=CIF_ALL_UPDATE_DAILY&day=toc-update-sun Sun]
|[https://publicdatafeeds.networkrail.co.uk/ntrod/CifFileAuthenticate?type=CIF_ALL_UPDATE_DAILY&day=toc-update-mon.CIF.gz Mon], [https://publicdatafeeds.networkrail.co.uk/ntrod/CifFileAuthenticate?type=CIF_ALL_UPDATE_DAILY&day=toc-update-tue.CIF.gz Tue], [https://publicdatafeeds.networkrail.co.uk/ntrod/CifFileAuthenticate?type=CIF_ALL_UPDATE_DAILY&day=toc-update-wed.CIF.gz Wed], [https://publicdatafeeds.networkrail.co.uk/ntrod/CifFileAuthenticate?type=CIF_ALL_UPDATE_DAILY&day=toc-update-thu.CIF.gz Thu], [https://publicdatafeeds.networkrail.co.uk/ntrod/CifFileAuthenticate?type=CIF_ALL_UPDATE_DAILY&day=toc-update-fri.CIF.gz Fri], [https://publicdatafeeds.networkrail.co.uk/ntrod/CifFileAuthenticate?type=CIF_ALL_UPDATE_DAILY&day=toc-update-sat.CIF.gz Sat], [https://publicdatafeeds.networkrail.co.uk/ntrod/CifFileAuthenticate?type=CIF_ALL_UPDATE_DAILY&day=toc-update-sun.CIF.gz Sun]
|-
|CIF_''XX''_TOC_FULL_DAILY
|toc-full
|Daily snapshot for TOC ''XX''
|[https://publicdatafeeds.networkrail.co.uk/ntrod/CifFileAuthenticate?type=CIF_XX_TOC_FULL_DAILY&day=toc-full Daily]
|Not available
|-
|CIF_''XX''_TOC_UPDATE_DAILY
|toc-update-''DAY''
|Daily update for TOC ''XX''
|[https://publicdatafeeds.networkrail.co.uk/ntrod/CifFileAuthenticate?type=CIF_XX_TOC_UPDATE_DAILY&day=toc-update-mon Mon], [https://publicdatafeeds.networkrail.co.uk/ntrod/CifFileAuthenticate?type=CIF_XX_TOC_UPDATE_DAILY&day=toc-update-Tue Tue], [https://publicdatafeeds.networkrail.co.uk/ntrod/CifFileAuthenticate?type=CIF_XX_TOC_UPDATE_DAILY&day=toc-update-wed Wed], [https://publicdatafeeds.networkrail.co.uk/ntrod/CifFileAuthenticate?type=CIF_XX_TOC_UPDATE_DAILY&day=toc-update-thu Thu], [https://publicdatafeeds.networkrail.co.uk/ntrod/CifFileAuthenticate?type=CIF_XX_TOC_UPDATE_DAILY&day=toc-update-fri Fri], [https://publicdatafeeds.networkrail.co.uk/ntrod/CifFileAuthenticate?type=CIF_XX_TOC_UPDATE_DAILY&day=toc-update-sat Sat], [https://publicdatafeeds.networkrail.co.uk/ntrod/CifFileAuthenticate?type=CIF_XX_TOC_UPDATE_DAILY&day=toc-update-sun Sun]
|Not available
|-
|CIF_FREIGHT_FULL_DAILY
|toc-full
|Daily FREIGHT snapshot
|[https://publicdatafeeds.networkrail.co.uk/ntrod/CifFileAuthenticate?type=CIF_FREIGHT_FULL_DAILY&day=toc-full Daily]
|Not available
|-
|CIF_FREIGHT_UPDATE_DAILY
|toc-update-''DAY''
|Daily update for FREIGHT
|[https://publicdatafeeds.networkrail.co.uk/ntrod/CifFileAuthenticate?type=CIF_FREIGHT_UPDATE_DAILY&day=toc-update-mon Mon], [https://publicdatafeeds.networkrail.co.uk/ntrod/CifFileAuthenticate?type=CIF_FREIGHT_UPDATE_DAILY&day=toc-update-tue Tue], [https://publicdatafeeds.networkrail.co.uk/ntrod/CifFileAuthenticate?type=CIF_FREIGHT_UPDATE_DAILY&day=toc-update-wed Wed], [https://publicdatafeeds.networkrail.co.uk/ntrod/CifFileAuthenticate?type=CIF_FREIGHT_UPDATE_DAILY&day=toc-update-thu Thu], [https://publicdatafeeds.networkrail.co.uk/ntrod/CifFileAuthenticate?type=CIF_FREIGHT_UPDATE_DAILY&day=toc-update-fri Fri], [https://publicdatafeeds.networkrail.co.uk/ntrod/CifFileAuthenticate?type=CIF_FREIGHT_UPDATE_DAILY&day=toc-update-sat Sat], [https://publicdatafeeds.networkrail.co.uk/ntrod/CifFileAuthenticate?type=CIF_FREIGHT_UPDATE_DAILY&day=toc-update-sun Sun]
|Not available
|-
|}


=== Examples ===
''DAY'' should be replaced with the shortened version of the name of the previous day.  For example, on Monday, Sunday's ('sun') update should be requested.


==== Header ====
A list of [[TOC Codes]] is available.  Note that the TOC code used in the URL is not the ATOC code as seen in schedule files, but the business code as used in Network Rail systems.
    "JsonTimetableV1":
        "classification":"public",
        "timestamp":1343952450,
        "owner":"Network Rail",
        "Sender":
            "organisation":"Rockshore",
            "application":"NTROD",
            "component":"SCHEDULE",
        "Metadata":
            "type":"full",
            "sequence":0
Example from CIF_ALL_FULL_DAILY


Gives the Last Update time of the File as a UNIX TIMESTAMP, in this example, Friday 3rd August 2012 01:07:30 +0100. All data should be send from the Rockshore Organisation.
=== Downloading via curl ===


==== Association ====
The schedule file can be downloaded from the command line using the ''curl'' utility (included with linux, versions available for other operating systems) with the following syntax:


===== Create =====
  curl -L -u 'username:password' -o file.gz 'https://publicdatafeeds.networkrail.co.uk/ntrod/CifFileAuthenticate?type=CIF_ALL_FULL_DAILY&day=toc-full'
    "JsonAssociationV1":
        transaction_type":"Create",
        "main_train_uid":"C05307",
        "assoc_train_uid":"C05351",
        "assoc_start_date":"2011-12-11T00:00:00Z",
        "assoc_end_date":"2012-09-09T00:00:00Z",
        "assoc_days":"0000001",
        "category":"NP",
        "date_indicator":"S",
        "location":"HTRWTM4",
        "base_location_suffix":null,
        "assoc_location_suffix":null,
        "diagram_type":"T",
        "CIF_stp_indicator":"P"
===== Delete =====
    "JsonAssociationV1":
        "transaction_type":"Delete",
        "main_train_uid":"W36743",
        "assoc_train_uid":"W37173",
        "assoc_start_date":"2012-08-03T00:00:00Z",
        "location":"STPANCI",
        "base_location_suffix":null,
        "diagram_type":"T",
        "cif_stp_indicator":null


* The transaction type indicates, if this is a new Entry to create or old Entry to delete.
(Thanks to [[User:Jules Self|Jules Self]] for figuring this syntax out.)
* Location is a [[Identifying_Stations|TIPLOC]] Reference
* assoc_days represent if the Association is valid on the relevant day (MTWTFSS)


=== Schedule ===
=== Downloading via C# ===


===== Create =====
Below is sample code written in C# that will download files.
    "JsonScheduleV1":
        "CIF_bank_holiday_running":null,
        "CIF_stp_indicator":"P",
        "CIF_train_uid":"C24056",
        "applicable_timetable":"Y",
        "atoc_code":"GW",
        "new_schedule_segment":
            "traction_class":"",
            "uic_code":""
        "schedule_days_runs":"0000010",
        "schedule_end_date":"2012-12-08",
        "schedule_segment":
            "signalling_id":"1A35",
            "CIF_train_category":"XX",
            "CIF_headcode":"1234",
            "CIF_course_indicator":1,
            "CIF_train_service_code":"25397003",
            "CIF_business_sector":"??",
            "CIF_power_type":"HST",
            "CIF_timing_load":null,
            "CIF_speed":"125",
            "CIF_operating_characteristics":null,
            "CIF_train_class":"B",
            "CIF_sleepers":null,
            "CIF_reservations":"S",
            "CIF_connection_indicator":null,
            "CIF_catering_code":"C",
            "CIF_service_branding":"",
            "schedule_location": <snip>
        "schedule_start_date":"2011-12-17",
        "train_status":"P",
        "transaction_type":"Create"


* atoc_code can be looked up on [[TOC_Codes]] (this example is missing)
There is also a last modified check method that can be used to see if a new file has been uploaded using the file header - without downloading the full file. This is useful if you quickly want to check every hour if a file is new compared to any you may have.
* Schedule Segment - signalling_id can be used to follow the train on the [[TD]] Feed
* Schedule Segment - CIF Power Type is what is a reference to what is pulling the Train
* Schedule Segment - CIF Speed is the top Speed of the Train
* Schedule Segment - CIF Sleepers is the service a Sleeper Service


===== Delete =====
[https://github.com/Legolash2o/OpenRailDataScheduleDownload C# Sample]
    "JsonAssociationV1":
        "transaction_type":"Delete",
        "main_train_uid":"F53453",
        "assoc_train_uid":"F53699",
        "assoc_start_date":"2012-07-30T00:00:00Z",
        "location":"SHBRYNS",
        "base_location_suffix":null,
        "diagram_type":"T",
        "cif_stp_indicator":null


====== Schedule Location ======
== Validity ==


Times are given in hhmm format, example 2005 for 5 minutes past 8 PM. (I'm not sure why some times are followed with a H)
Associations and schedule validities are between a start date and an end date, and on particular days of the week. They each have a Short Term Planning (STP) indicator field as follows:


Schedule Locations consists of none or more locations
* '''C''' - Planned cancellation: the schedule does not apply on this date, and the train will not run.  Typically seen on public holidays when an alternate schedule applies, or on Christmas Day.
* '''N''' - STP schedule: similar to a permanent schedule, but planned through the Short Term Planning process and not capable of being overlaid
* '''O''' - Overlay schedule: an alteration to a permanent schedule
* '''P''' - Permanent schedule: a schedule planned through the Long Term Planning process


The Path key can describe the route the train is expected to take into a station. For example Leeds Central when approaching from the South as Paths A-F
Permanent ('P') schedules can be overlaid by another schedule with the same UID - either a Variation ('O') or Cancellation Variation ('C').  For any particular day, of all the schedules for that UID valid on that day, the 'C' or 'O' schedule is the one which applies. Conveniently, it also means that the lowest alphabetical STP indicator wins - 'C' and 'O' are both lower in the alphabet than 'P'.


Platform Data is not always provided.
This process allows a different schedule to be valid on particular days, or the service to not be valid on that day.  For example, a schedule may be valid Monday - Friday each day of the year, but have a Cancellation Variation on Christmas Day and Boxing Day only.


Schedule Locations come in three types
Cancellation variations are usually within the validity period of the base (permanent) schedule, however it is possible to have a cancellation variation which has a 'valid from' date which is within one version of a schedule and a 'valid to' date within another version of a schedule.  In this case, the variation applies to both.  An example of this is schedule C33552 at the start of 2015, where a cancellation variation applied to two separate permanent schedules.


* LO - Train Origin
{{Navtable-DataFeeds}}
* LI - Stopping/Passing/Timing Point
* LT - Train Terminus


Stopping points contain both the Arrival and Departure time for the Train
[[Category:Schedule Data]]
 
As the data is an array, the Index of the Array can be used to help determine Stop Order.
 
======Train Origin======
    "location_type":"LO",
    "record_identity":"LO",
    "tiploc_code":"ABRDEEN",
    "tiploc_instance":null,
    "departure":"1350",
    "public_departure":"1350",
    "platform":"7",
    "line":null,
    "engineering_allowance":null,
    "pathing_allowance":null,
    "performance_allowance":null
 
As a departure only reference only the departure keys are present
 
======Stopping Point======
    "location_type":"LI",
    "record_identity":"LI",
    "tiploc_code":"FNPK",
    "tiploc_instance":null,
    "arrival":"0657",
    "departure":"0658",
    "pass":null,
    "public_arrival":"0657",
    "public_departure":"0658",
    "platform":"3",
    "line":null,
    "path":null,
    "engineering_allowance":null,
    "pathing_allowance":null,
    "performance_allowance":null
 
This train arrives and ~1 minute later, is expected to leave.
 
======Passing Point======
Some stopping points are just Passing Points, used for routing trains over points for specific paths or lines.
    "location_type":"LI",
    "record_identity":"LI",
    "tiploc_code":"KNGXBEL",
    "tiploc_instance":null,
    "arrival":null,
    "departure":null,
    "pass":"2052H",
    "public_arrival":null,
    "public_departure":null,
    "platform":null,
    "line":"FL2",
    "path":null,
    "engineering_allowance":null,
    "pathing_allowance":null,
    "performance_allowance":null
 
======Train Terminus======
    "location_type":"LT",
    "record_identity":"LT",
    "tiploc_code":"KNGX",
    "tiploc_instance":null,
    "arrival":"2054H",
    "public_arrival":"2100",
    "platform":"4",
    "path":null
 
As this is an arrival entry, only arrival keys are present.
 
=== EOF ===
    "EOF":true
 
Just a handy note to say you reached the end of the file, in case you obtained a broken download.
 
=== Further Information ===
 
The first ~4% of the Full Daily file contains schedule associations, this links multiple Train UID's with a Primary Train UID, which can be looked up in the Schedules.
 
Due to the Size of the Full Daily, (1.5GB when gunzip'ed) it can take some time (about 2 hours) to import the data from the file.
 
A given Schedule entry, contains information about the schedule, including its Start and End dates, and can then contain one or more schedule stops, which describe the calling points for a train on its schedule.
These calling points will have a official arrival/departure time and a Public arrival/departure time. When displaying data to the end user, its probably best to use the Public versions.

Latest revision as of 11:48, 25 February 2023

The SCHEDULE feed is an extract of train schedules from Network Rail's ITPS (Integrated Train Planning System), available in JSON and CIF format.

Note: If you are just starting out with the service use the JSON files. The CIF data is more suited to advanced users of the service and requires additional parsing compared to the JSON data.

Overview

The schedule data consists of Train Schedules, Train Associations and details of Timing Point Location codes.

A description of the scheduling process is available on the how scheduling works page. This describes how multiple schedules can be used to describe a specific train service.

Usage

Trains are sets of schedules identified by a common UID. A schedule for a specific train service can be uniquely identified by UID, Start Date and STP Indicator.

Types of file

There are two types of files:

  • A full extract, a snapshot of schedule database valid on that day, which can be used to initialise your schedule database
  • An update extract is produced each night, which can be applied to the previous update extract to bring your database up-to-date

Update extracts will specify how the schedule database has changed. It is important that the update files are processed in sequential order. Amendments may be received for schedules that are currently valid, i.e. the schedule has started running but has not finished yet. Changes will replace any previous version of a train schedule. Deletions will not be raised for schedules that have become historic.

The following formats are available from SCHEDULE Feed:

Format Type Rate Details
JSON File Format Full Extract Daily The schedules in JSON format are available for all operators, or for individual operators.
Update Extract Daily
CIF File Format Full Extract Weekly The schedules in CIF format are only available for all operators. A 'full' extract is available each Friday, and an 'update' extract is available daily.
Update Extract Daily

The JSON data files are generated overnight and will be available from around 0600. The CIF files are go through a different process and will be available from around 0100.

Note: Late-notice train schedules, which are not available through the SCHEDULE feed, are published via the real-time VSTP feed.

Downloading

The schedule data is compressed using gzip and is downloaded from Amazon S3 via a private URL which is valid for a few minutes after generation. Authentication will be required either by logging in through the Data Feeds website, or by sending your username and password as part of the HTTP request. On successful authentication, you will receive a HTTP 302 (Moved Temporarily) redirect to the location of the schedule file. Requests are of the format:

 https://publicdatafeeds.networkrail.co.uk/ntrod/CifFileAuthenticate?type=type&day=day

where type determines the TOCs contained in the schedule extract and whether the file is a full schedule or an update file, and day determines which day's update file is downloaded. For example:

 https://publicdatafeeds.networkrail.co.uk/ntrod/CifFileAuthenticate?type=CIF_ALL_FULL_DAILY&day=toc-full

would download a full schedule file in JSON format for all TOCs. And if the CIF format is what you are after, add a trailing .CIF.gz to get the CIF format.

Values for type and day can be one of the following:

type day Description Example URL (JSON format) Example URL (CIF format)
CIF_ALL_FULL_DAILY toc-full Daily all-TOC snapshot Daily Weekly
CIF_ALL_UPDATE_DAILY toc-update-DAY Daily all-TOC update Mon, Tue, Wed, Thu, Fri, Sat, Sun Mon, Tue, Wed, Thu, Fri, Sat, Sun
CIF_XX_TOC_FULL_DAILY toc-full Daily snapshot for TOC XX Daily Not available
CIF_XX_TOC_UPDATE_DAILY toc-update-DAY Daily update for TOC XX Mon, Tue, Wed, Thu, Fri, Sat, Sun Not available
CIF_FREIGHT_FULL_DAILY toc-full Daily FREIGHT snapshot Daily Not available
CIF_FREIGHT_UPDATE_DAILY toc-update-DAY Daily update for FREIGHT Mon, Tue, Wed, Thu, Fri, Sat, Sun Not available

DAY should be replaced with the shortened version of the name of the previous day. For example, on Monday, Sunday's ('sun') update should be requested.

A list of TOC Codes is available. Note that the TOC code used in the URL is not the ATOC code as seen in schedule files, but the business code as used in Network Rail systems.

Downloading via curl

The schedule file can be downloaded from the command line using the curl utility (included with linux, versions available for other operating systems) with the following syntax:

 curl -L -u 'username:password' -o file.gz 'https://publicdatafeeds.networkrail.co.uk/ntrod/CifFileAuthenticate?type=CIF_ALL_FULL_DAILY&day=toc-full'

(Thanks to Jules Self for figuring this syntax out.)

Downloading via C#

Below is sample code written in C# that will download files.

There is also a last modified check method that can be used to see if a new file has been uploaded using the file header - without downloading the full file. This is useful if you quickly want to check every hour if a file is new compared to any you may have.

C# Sample

Validity

Associations and schedule validities are between a start date and an end date, and on particular days of the week. They each have a Short Term Planning (STP) indicator field as follows:

  • C - Planned cancellation: the schedule does not apply on this date, and the train will not run. Typically seen on public holidays when an alternate schedule applies, or on Christmas Day.
  • N - STP schedule: similar to a permanent schedule, but planned through the Short Term Planning process and not capable of being overlaid
  • O - Overlay schedule: an alteration to a permanent schedule
  • P - Permanent schedule: a schedule planned through the Long Term Planning process

Permanent ('P') schedules can be overlaid by another schedule with the same UID - either a Variation ('O') or Cancellation Variation ('C'). For any particular day, of all the schedules for that UID valid on that day, the 'C' or 'O' schedule is the one which applies. Conveniently, it also means that the lowest alphabetical STP indicator wins - 'C' and 'O' are both lower in the alphabet than 'P'.

This process allows a different schedule to be valid on particular days, or the service to not be valid on that day. For example, a schedule may be valid Monday - Friday each day of the year, but have a Cancellation Variation on Christmas Day and Boxing Day only.

Cancellation variations are usually within the validity period of the base (permanent) schedule, however it is possible to have a cancellation variation which has a 'valid from' date which is within one version of a schedule and a 'valid to' date within another version of a schedule. In this case, the variation applies to both. An example of this is schedule C33552 at the start of 2015, where a cancellation variation applied to two separate permanent schedules.


Network Rail Open Data Feeds
Data Feeds About the Feeds Account States Durable Subscriptions Example Code ( PHP / C# / Java / Ruby / Node.js) • Advanced UsesFAQ Release Notes
RTPPM RTPPM Feed
Train Movements Train Movements Feed Train Activation Train Cancellation Train Movement Train Reinstatement Change of Origin Change of Identity Change of Location TSPEED Field Planned Cancellations Cancellation Codes
TD TD Feed C-Class Messages S-Class Messages Train Describers TD Berths
VSTP VSTP Feed
TSR TSR Feed Route Codes
SCHEDULE SCHEDULE Feed TIPLOC Records Schedule and Location Records Association Records CIF Codes How Scheduling Works Allowances
Reference Data Reference Data Feed TOC Codes CIF Codes Delay Attribution Codes Identifying Locations (STANOX, TIPLOC, NLC and 3-Alpha Codes) STANOX Geographical Areas Train Planning data (BPLAN)