KnowledgeBase: Difference between revisions

From Open Rail Data Wiki
← Older edit
EvelynSnow (talk | contribs)
205 edits
Reword internal KB URLs section (internal. isn't the only source of them)
 
(22 intermediate revisions by 9 users not shown)
Line 1: Line 1:
== About ==
= About =


KnowledgeBase is RDG’s Content Management System providing customer friendly information on the railways and associated products.  
Knowledgebase is RDG’s Content Management System providing customer friendly information on the railways and associated products. It contains a wealth of static and real-time information about traveling by train on the GB rail network, such as information about station facilities, service disruption, and engineering work.


KnowledgeBase data is available on National Rail Data Portal through 2 subscription types:
Data is available on the National Rail Data Portal through two subscription types:
* Real Time Data Feeds – for new, modified and deleted Incidents
 
* On Demand Data Feeds – for requesting the latest information on the following
* Real Time incidents - for new, modified and deleted Incidents
** Ticket Restrictions
* Knowledgebase API - for requesting the latest information on the following static data
** Companies
** Companies (TOCs)
** Incidents
** Incidents (Service Disruptions and Engineering Works)
** National Service Indicator
** National Service Indicator (NSI)
** Promotions (Public)
** Promotions (Public)
** Stations
** Stations
** Ticket Restrictions
** Ticket Types
** Ticket Types


== Accessing the KnowledgeBase Feeds ==
= Accessing the Knowledgebase Feeds =
 
To access the RDG Knowledgebase feeds, you must have an account on the National Rail Data Portal (NRDP) at http://opendata.nationalrail.co.uk. By creating an account, you can register for a subscription to the Knowledgebase API and the Knowledgebase Real Time Feeds.


To access the RDG KnowledgeBase feeds, you must have an account on the National Rail Data Portal (NRDP) at http://datafeeds.nationalrail.co.uk. After logging in, you will see 3 subscription types: Historical Service Performance (HSP), Real Time Data Feeds and On Demand Data Feeds. By selecting the checkbox for each of these subscription types, the respective sections become visible on the My Feeds page.  
To receive Knowledgebase Real Time Incidents, select Knowledgebase (KB) Real Time Incidents under the Knowlegebase (KB) subscriptions. The connection information required to start receiving Real Time Incidents is located on the My Feeds page.


The previous Darwin Data Feed on the NRDP Account page has been replaced by the Real Time Data Feed which once selected include details about the Darwin push-port feeds and the KnowledgeBase – Real Time Incident feed in the My Feeds page.
To receive Knowldegebase static data, select Knowledgebase (KB) API under the Knowledgebase (KB) subscriptions. The My Feeds page will then display details about accessing the latest versions of Ticket Restrictions, Companies, Incidents, National Service Indicator, Promotions (Public), Stations and Ticket Types files from Knowledgebase.


To receive KnowledgeBase Real Time Incidents, select Real Time Data Feeds under Subscription Type in the Edit User Details page and then select the Update button to save this change. Go to the My Feeds page and select the KnowledgeBase data type – Real Time Incidents option. You will need the [http://nrodwiki.rockshore.net/index.php/Darwin:Push_Port#Accessing_real-time_data/ Real Time Information /STOMP details] to start receiving real time incident messages via a STOMP client.
'''Important''' - Please note NRDP accounts expire after an extended period of inactivity. The unused account expiry period is currently set to 30 days. If you create an account and do not consume any of the feeds during this time your account will be deleted. If your account has been deleted, you will receive a notification email, and you will be able to re-register for a new account.


The On Demand Data Feeds section on the My Feeds page contains details about accessing the latest versions of Ticket Restrictions, Companies, Incidents, National Service Indicator, Promotions (Public), Stations and Ticket Types files from KnowledgeBase. To receive these KnowledgeBase files, select the On Demand Data Feeds checkbox under the Subscription type in the Edit User Details page and then select the update button to save this change.
= Internal KB URLs =
Some endpoints are currently accessible without authentication on subdomains reserved for internal use only. Internal KB was originally firewalled, this was removed in March of 2020, but it may return to being firewalled without warning. If a URL you're using starts with anything other than ''opendata.nationalrail.co.uk'', you're probably in the wrong place!


== RDG KnowledgeBase APIs ==
= RDG Knowledgebase APIs =


=== On Demand Data Feeds ===
== Knowledgebase API ==
After selecting the On Demand checkbox subscription type on NRDP, you can receive the KnowlegeBase xml files in 2 steps via a rest client.


:: Step 1: Perform HTTP POST call to NRDP via https://datafeeds.nationalrail.co.uk/authenticate to generate a token.
After selecting the Knowledgebase (KB) API checkbox subscription type on NRDP, you can receive the Knowledgebase xml files in two steps via a REST client.
:: Step 2: Perform HTTP GET call to individual NRDP Knowledgebase Feeds using the token produced in step 1
==== Step 1: Generate Token ====
In this step, using a REST client you perform an HTTP POST call to https://datafeeds.nationalrail.co.uk/authenticate. This step requires using your email address and password registered on NRDP and will produce a token key. The following example shows the HTTP POST request and response with NRDP:
: Example HTTP POST Request to NRDP:
  <code><pre> POST https://datafeeds.nationalrail.co.uk/authenticate HTTP/1.1
  Content-Type: application/x-www-form-urlencoded
  Accept: application/json, text/plain, */*
  Raw payload: username=user1@gmail.com&password=P@55w0rd1
  //Add your registered NRDP email address and NRDP password in the “Raw payload” section above </pre></code>
: Corresponding example HTTP Response from NRDP:
<code><pre> {
"username": " user1@gmail.com ",
"roles": {
"Role(id=6ab28d46-0e4a-11e7-9339-060a84373309, name='ROLE_KB_USER')": true,
"Role(id=47d36dc4-16b9-11e6-9746-060a84373309, name='ROLE_HSP_USER')": true,
"Role(id=4e660138-bb39-11e4-9da1-060a84373309, name='ROLE_D3_USER')": true
},
"token": " user1@gmail.com:1491312310772:56c56baa3e56d35ff0ede4a6aad1bcfb"
} </pre></code>


Each token is valid for 1 hour and you will receive a 401 unauthorized response when the token has expired. The following http response example is generated after sending an http request to a NRDP knowledgebase feed with an expired token.
=== Generate Authentication Token ===
: Example HTTP 401 Response Header: Token Expired:
See the [[National Rail Data Portal#Authentication|Authentication]] section on the [[National Rail Data Portal|National Rail Data Portal]] article
<code><pre> Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Content-Language: en
Content-Length: 1081
Content-Type: text/html;charset=utf-8
Date: Wed, 05 Apr 2017 09:52:11 GMT
Expires: 0
Pragma: no-cache
Server: Apache-Coyote/1.1
X-Content-Type-Options: nosniff
X-Frame-Options: DENY
X-Xss-Protection: 1; mode=block </pre></code>
: Corresponding example HTML Response file: Token Expired:
<code><pre> <html>
<head>
<title>Apache Tomcat/7.0.56 - Error report</title>
<style>
<!--H1 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:22px;} H2 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-
size:16px;} H3 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:14px;} BODY {font-family:Tahoma,Arial,sans-serif;color:black;background-
color:white;} B {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;} P {font-family:Tahoma,Arial,sans-serif;background:white;color:black;font-size:12px;}A
{color : black;}A.name {color : black;}HR {color : #525D76;}-->
</style> </head>
<body>
<h1>HTTP Status 401 - Unauthorized: Authentication token was either missing or invalid.</h1>
<HR size="1" noshade="noshade">
<p>
<b>type</b> Status report</p>
<p>
<b>message</b> <u>Unauthorized: Authentication token was either missing or invalid.</u>
</p>
<p>
<b>description</b> <u>This request requires HTTP authentication.</u>
</p>
<HR size="1" noshade="noshade">
<h3>Apache Tomcat/7.0.56</h3>
</body>
</html> </pre></code>


==== Step 2: Perform a HTTP GET call for each KnowledgeBase file ====
==== Make a call for each file ====
In this step using a REST client, you would perform a HTTP GET call to each Knowledgebase file using the token key produced in step 1. This will produce the xml file for the corresponding KnowledgeBase feeds.


The following are the required links to make the respective HTTP GET call:  
Once you have a valid authentication token, you can make an HTTP GET request.  Supply the authentication token in a header named `X-Auth-Token`:
* Ticket Restrictions: https://datafeeds.nationalrail.co.uk/api/staticfeeds/4.0/ticket-restrictions  
 
* Companies: https://datafeeds.nationalrail.co.uk/api/staticfeeds/4.0/tocs
  GET https://opendata.nationalrail.co.uk/api/staticfeeds/4.0/ticket-restrictions
* Incidents: https://datafeeds.nationalrail.co.uk/api/staticfeeds/5.0/incidents
 
* National Service Indicator: https://datafeeds.nationalrail.co.uk/api/staticfeeds/4.0/serviceIndicators
    X-Auth-Token: jane@doe.com:1491312310772:56c56baa3e56d35ff0ede4a6aad1bcfb
* Promotions (Public): https://datafeeds.nationalrail.co.uk/api/staticfeeds/4.0/promotions-publics
 
* Stations: https://datafeeds.nationalrail.co.uk/api/staticfeeds/4.0/stations
A successful response will result in the requested file:
* Ticket Types: https://datafeeds.nationalrail.co.uk/api/staticfeeds/4.0/ticket-types
 
The header of the HTTP response will also include the date when the xml file was last modified using this format: Last-Modified: DAY, DD MMM YYYY HH:MM:SS GMT
    <?xml version="1.0" encoding="utf-8"?>
The following is an example of request/response with Ticket Restrictions Knowledgebase Feed via NRDP:
    <TicketRestrictions
: Example HTTP GET Request to NRDP knowledgebase feed:
    ...
<code><pre>GET https://datafeeds.nationalrail.co.uk/api/staticfeeds/4.0/ticket-restrictions HTTP/1.1
 
Content-Type: application/json
If authorization failed, the response will be an HTTP 401 Unauthorized:
Accept: */*
 
X-Auth-Token: NRDP_email_address:1491312310772:56c56baa3e56d35ff0ede4a6aad1bcfb
    {
//Enter your NRDP email address and the Token Key used in step 1 above </pre></code>
        "timestamp": "2019-02-13T11:50:16.048+0000",
: Corresponding example HTTP 200 Response Header:
        "status": 401,
<code><pre> Cache-Control: no-cache, no-store, max-age=0, must-revalidate
        "error": "Unauthorized",
Content-Length: 1194601
        "message": "Unauthorized",
Content-Type: text/plain;charset=ISO-8859-1
        "path": "/api/staticfeeds/4.0/ticket-restrictions"
Date: Wed, 05 Apr 2017 08:29:16 GMT
    }
Expires: 0
 
Last-Modified: Wed, 05 Apr 2017 08:16:39 GMT
If you are not subscribed to the Knowledgebase API,  the response will be an HTTP 403 Forbidden:
Pragma: no-cache
 
Server: Apache-Coyote/1.1
    {
X-Content-Type-Options: nosniff
        "timestamp": "2019-02-13T11:51:29.992+0000",
X-Frame-Options: DENY
        "status": 403,
X-Xss-Protection: 1; mode=block
        "error": "Forbidden",
</pre></code>
        "message": "Forbidden",
: Corresponding XML Response file:
        "path": "/api/staticfeeds/4.0/ticket-restrictions"
<code><pre> <?xml version="1.0" encoding="utf-8"?>
    }
<TicketRestrictions xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"  
 
xmlns:com="http://nationalrail.co.uk/xml/common" xsi:schemaLocation="http://internal.nationalrail.co.uk/xml/XsdSchemas/External/Version4.0/nre-ticket-restriction-v4-0.xsd"  
=== Stations ===
xmlns="http://nationalrail.co.uk/xml/ticketrestriction">
 
  <TicketRestriction>
In the stations data, London St Pancras and Ashford International both have have ''two'' entries each, intended to correspond to domestic services and facilities (STP and AFK), and those dedicated to international facilities (SPX and ASI). These second entries are not maintained, and these CRS codes, derived from underlying Network Rail data, are not recognised by LDB, or in the Darwin push port reference data, which mantains the National Rail convention of one CRS code per station (here STP and AFK). These entries bear confusingly similar names to the more canonical ones, and may be sorted before them, something to consider if you're allowing a user to search stations.
    <Name><![CDATA[S2]]></Name><LinkToDetailPage><![CDATA[http://int.web.kb.awsnre.co.uk/times_fares/ticket_types/145184.aspx]]></LinkToDetailPage>
    <RestrictionCode>S2</RestrictionCode>
    <TicketRestrictionIdentifier>52CD8939787B42789817C8E83A38F74E</TicketRestrictionIdentifier>
    <ApplicableDays><![CDATA[<p>TEST SM 19/01</p>]]></ApplicableDays>
    <OutwardDirection>Outward Travel</OutwardDirection>
    <ReturnDirection>Return Travel</ReturnDirection>
    <RestrictionsType>2</RestrictionsType>
    <Restrictions>
      <Restriction>
        <DetailsOutward><![CDATA[<p>SM 2 </p>]]></DetailsOutward>
        <DetailsReturn><![CDATA[<p>SM TEST 2 </p>]]></DetailsReturn>
      </Restriction>
    </Restrictions>
  </TicketRestriction>
</pre></code>


== Real Time Incident Feed ==
== Real Time Incident Feed ==
After selecting the Real Time Data Feed checkbox subscription type on account page & Real Time Incidents checkbox on My Feeds page, you can receive the KnowlegeBase xml file via a STOMP client as follows:
The real-time incidents feed details the creation, modification, and deletion of Knowledgebase incidents in real-time. The underlying information is the same as is available through the HTTP API (which can be used as a snapshot, if necessary), but its real-time nature is useful for tracking state changes more reliably, efficiently, and quickly. Messages from the real-time incidents feed can be received using either STOMP or Openwire. After selecting the Knowledgebase real-time incidents checkbox on the ''edit account'' page, connection information (host, credentials, and topic name) will now be shown in the relevant section on your ''My Feeds'' page.
: Using the STOMP client, connect to https://datafeeds.nationalrail.co.uk/ with the login credential shown on My Feeds/ Real Time Feed Information using your stomp script:
 
:: Username: D3user
Please note:
:: Password: D3password
* If using STOMP, your client must support STOMP v1.2.
:: Queue name: This is the unique value for your account
* Knowledgebase message bodies are not compressed, unlike the Darwin push port.
Upon successfully connecting to ActiveMQ via the STOMP client, you will start receiving each real time incident message, with its respective incident number and other message information. The header of each message will contain one of the following incident change statuses:
* The opening XML tag is malformed, and should be ''PtIncident'' instead of ''uk.co.nationalrail.xml.incident.PtIncidentStructure''. Simple string substitution can resolve this if it presents a problem using the schema. This issue does ''not'' affect the incidents file retrievable over HTTP.
: '''New''': This status represents new incident message information
* Topic advisory messages are disabled on the KB Real Time Incidents messaging host, so you must also connect with this facility disabled if using Openwire (e.g. by setting watchTopicAdvisories=false if using NMS)
: '''Modified''': This status represents modifications to existing incidents message information
* The feed is relatively low volume with typically only a few messages per hour and often gaps of several hours overnight; if monitoring message volumes, or testing, you should take this into account.
: '''Removed''': This status represents deleted message information
 
=== Example Real Time Incidents ===
=== Headers ===
The following are examples of Real Time Incidents which include new, modified and deleted incident messages:
Each frame from the real-time incidents feed contains the following headers:
<code><pre> C:\>ruby Darwin-pre.rb
 
################################################################
* ''INCIDENT_MESSAGE_STATUS'': either ''NEW'' (a new incident message), ''MODIFIED'' (a change to existing incident information), or ''REMOVED'' (incident deletion)
##########Stomp consumer for National Rail Data Portal##########
* ''KNOWLEDGEBASE_DATA_TYPE'', set to ''INCIDENT_MESSAGE''.
################################################################
* ''INCIDENT_ID'': the ID of the KB incident
Connected to ActiveMQ/5.14.1 server
 
-----------------------------------------------------------------
=== Example Real Time Incidents Bodies ===
##Example New Incident messages##
The following are examples of real-time incidents messages which include ''new'', ''modified'' and ''deleted'' incidents.
-----------------------------------------------------------------
 
Header for message is :
==== New Incident message ====
{
<pre>
"content-length"=>"857",
"expires"=>"0",
"KNOWLEDGEBASE_DATA_TYPE"=>"INCIDENT_MESSAGE",
"INCIDENT_ID"=>"214453059CDC4017A0E32F3A6ECA0D45",
"destination"=>"/queue/D345d5d738-7713-4b04-aa62-ac02bcd391b3",
"subscription"=>"cfde902ac319642fcbb0e97e42a4e54a8b009624",
"priority"=>"0", "INCIDENT_MESSAGE_STATUS"=>"NEW",
"breadcrumbId"=>"ID-EC2D3-PRE-DB-50643-1493047216416-0-5633",
"message-id"=>"ID:EC2D3-PRE-DB-47730-1493047193600-2:1:3:1:214111",
"persistent"=>"true",
"timestamp"=>"1493048298934"
}
Decompressed Message is:
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<uk.co.nationalrail.xml.incident.PtIncidentStructure xmlns:ns2="http://nationalrail.co.uk/xml/common" xmlns:ns3="http://nationalrail.co.uk/xml/incident">
<uk.co.nationalrail.xml.incident.PtIncidentStructure xmlns:ns2="http://nationalrail.co.uk/xml/common" xmlns:ns3="http://nationalrail.co.uk/xml/incident">
Line 204: Line 123:
<ns3:ClearedIncident>false</ns3:ClearedIncident>
<ns3:ClearedIncident>false</ns3:ClearedIncident>
<ns3:IncidentPriority>2</ns3:IncidentPriority></uk.co.nationalrail.xml.incident.PtIncidentStructure>
<ns3:IncidentPriority>2</ns3:IncidentPriority></uk.co.nationalrail.xml.incident.PtIncidentStructure>
----------------------------------------------------------------
...
##Example Modified Incident Message##
</pre>
----------------------------------------------------------------
 
Header for message is :
==== Modified Incident Message ====
{
 
"content-length"=>"1369",
<pre>
"expires"=>"0",
"KNOWLEDGEBAS E_DATA_TYPE"=>"INCIDENT_MESSAGE",
"INCIDENT_ID"=>"8B3B4C1F358A45A186F6F95984F52F4F",
"destination"=>"/queue/D345d5d738-7713-4b04-aa62-ac02bcd391b3",
"subscription"=>"cfde902ac319642fcbb0e97e42a4e54a8b009624",
"priority"=>"0",
"INCIDENT_MESS AGE_STATUS"=>"MODIFIED",
"breadcrumbId"=>"ID-EC2D3-PRE-DB-42611-1493050980100-0- 1259",
"message-id"=>"ID:EC2D3-PRE-DB-60889-1493050957804-2:1:4:1:84412",
"persistent"=>"true",
"timestamp"=>"1493051706498"
}
Decompressed Message is:
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<uk.co.nationalrail.xml.incident.PtIncidentStructure xmlns:ns2="http://nationalrail.co.uk/xml/common" xmlns:ns3="http://nationalrail.co.uk/xml/incident">
<uk.co.nationalrail.xml.incident.PtIncidentStructure xmlns:ns2="http://nationalrail.co.uk/xml/common" xmlns:ns3="http://nationalrail.co.uk/xml/incident">
Line 241: Line 147:
<p><strong>Twitter:</strong><br />If you would like to follow this incident on Twitter, please use
<p><strong>Twitter:</strong><br />If you would like to follow this incident on Twitter, please use
  <a href="https://twitter.com/hashtag/CrystalPalace?f=tweets">#</a><span><a href="https://twitter.com/hashtag/CrystalPalace?f=tweets">CrystalPalace</a> </span></p>
  <a href="https://twitter.com/hashtag/CrystalPalace?f=tweets">#</a><span><a href="https://twitter.com/hashtag/CrystalPalace?f=tweets">CrystalPalace</a> </span></p>
  <p><strong>Check before you travel:</strong><br />Please use the National Rail Enquiries real-time  
  <p><strong>Check before you travel:</strong><br />Please use the National Rail Enquiries real-time
  <a href="http://ojp.nationalrail.co.uk/service/planjourney/search">Journey Planner</a> to check your journey before you travel.</p>
  <a href="http://ojp.nationalrail.co.uk/service/planjourney/search">Journey Planner</a> to check your journey before you travel.</p>
  <p><strong>Compensation:</strong><br />You may be entitled to <a href="http://www.nationalrail.co.uk/times_fares/ticket_types/121354.aspx">
  <p><strong>Compensation:</strong><br />You may be entitled to <a href="http://www.nationalrail.co.uk/times_fares/ticket_types/121354.aspx">
  compensation</a> if you experience a delay in completing your journey today.
  compensation</a> if you experience a delay in completing your journey today.
  Please keep your train ticket and make a note of your journey, as both will be required to support any claim.</p>
  Please keep your train ticket and make a note of your journey, as both will be required to support any claim.</p>
  <p><strong>Feedback:</strong><br />We want to make information better - tell us how! Fill out this online  
  <p><strong>Feedback:</strong><br />We want to make information better - tell us how! Fill out this online
  <a href="http://survey3.accent-mr.com/D4/(S(dtxglkj3ksyum52j0j01qpio))/2908m.aspx?urn=opensurveyNRwebsite">Disruption Survey.</a></p></ns3:Description>
  <a href="http://survey3.accent-mr.com/D4/(S(dtxglkj3ksyum52j0j01qpio))/2908m.aspx?urn=opensurveyNRwebsite">Disruption Survey.</a></p></ns3:Description>
<ns3:InfoLinks><ns3:InfoLink><ns3:Uri>http://www.nationalrail.co.uk/service_disruptions/162083.aspx</ns3:Uri>
<ns3:InfoLinks><ns3:InfoLink><ns3:Uri>http://www.nationalrail.co.uk/service_disruptions/162083.aspx</ns3:Uri>
Line 257: Line 163:
<ns3:ClearedIncident>false</ns3:ClearedIncident>
<ns3:ClearedIncident>false</ns3:ClearedIncident>
<ns3:IncidentPriority>2</ns3:IncidentPriority></uk.co.nationalrail.xml.incident.PtIncidentStructure>
<ns3:IncidentPriority>2</ns3:IncidentPriority></uk.co.nationalrail.xml.incident.PtIncidentStructure>
----------------------------------------------------------------
...
##Example Removed Incident Message##
</pre>
----------------------------------------------------------------
==== Removed Incident Message ====
Header for message is :
Removed messages contain empty bodies.
{
<pre></pre>
"content-length"=>"20",
 
"expires"=>"0",
= Support =
"KNOWLEDGEBASE_DATA_TYPE"=>"INCIDENT_MESSAGE",
KnowledgeBase documentation and schemas:
"INCIDENT_ID"=>"05A3E842D53C4AAEA2EFAA8533EA84BE",
* [https://www.rspaccreditation.org/publicDocumentation.php#RSPS5xxx RSPS5050]
"destination"=>"/queue/D345d5d738-7713-4b04-aa62-ac02bcd391b3",
"subscription"=>"cfde902ac319642fcbb0e97e42a4e54a8b009624",
"priority"=>"0",
"INCIDENT_MESSAGE_STATUS"=>"REMOVED",
"breadcrumbId"=>"ID-EC2D3-PRE-DB-50643-1493047216416-0-4165",
"message-id"=>"ID:EC2D3-PRE-DB-47730-1493047193600-2:1:4:1:164882",
"persistent"=>"true",
"timestamp"=>"1493048178800"
}
Decompressed Message is: (empty body)
</pre></code>


=== An Example STOMP script: Darwin-pre.rb ===
{{Navtable-NreDataFeeds}}
The STOMP client script below can be used as a template for receiving the real time incident messages on a STOMP client.
<code><pre>require "stomp"
require "zlib"
require "stringio"
class NrPollerPoller
  # Initialize the poller
  def initialize
    @hostname = 'datafeeds.nationalrail.co.uk'
    @username = 'd3user'
    @password = 'd3password'
    puts "########################################################"
    puts "#####Stomp consumer for National Rail Data Portal ######"
    puts "########################################################"
  end
# Connect to the service and process messages
  def run
    port = 61613
    queue = "/queue/NRDP_QUEUE"
    client = Stomp::Client.new( "d3user", "d3password",  "datafeeds.nationalrail.co.uk", port )
    # Check we have connected successfully
    raise "Connection failed" unless client.open?
    raise "Connect error: #{client.connection_frame().body}" if client.connection_frame().command == Stomp::CMD_ERROR
    puts "Connected to #{client.connection_frame().headers['server']} server "
    # Subscribe to my user's queue and process messages
    client.subscribe queue, { :ack => :client } do | msg |
        client.acknowledge msg # tell the server the message was handled and to dispose of it
        # Now unzip the string
        gz = Zlib::GzipReader.new( StringIO.new( msg.body ) )
        xml = gz.read
        puts "Decompressed Message is: #{xml}"
    end
    client.join
    # We will probably never end up here
    client.close
    puts "Client close complete"
  end
end
e = NrPollerPoller.new
e.run
</pre></code>
=== Real Time Incident Schema ===
This section contains the Real Time Incident XSD.
<code><pre><xs:schema attributeFormDefault="unqualified" elementFormDefault="qualified" targetNamespace="http://www.govtalk.gov.uk/metadata/egms" xmlns:xs="http://www.w3.org/2001/XMLSchema">
  <xs:element name="Metadata">
    <xs:complexType>
      <xs:sequence>
        <xs:element name="Contributor" maxOccurs="unbounded" minOccurs="0">
          <xs:complexType>
            <xs:simpleContent>
              <xs:extension base="xs:string">
                <xs:attribute type="xs:string" name="Role" use="optional"/>
                <xs:attribute type="xs:string" name="Href" use="optional"/>
              </xs:extension>
            </xs:simpleContent>
          </xs:complexType>
        </xs:element>
        <xs:element type="xs:string" name="Creator"/>
        <xs:element type="xs:date" name="Created"/>
        <xs:element type="xs:date" name="Modified"/>
        <xs:element type="xs:string" name="Description"/>
        <xs:element name="Format">
          <xs:complexType>
            <xs:simpleContent>
              <xs:extension base="xs:string">
                <xs:attribute type="xs:anyURI" name="Scheme"/>
              </xs:extension>
            </xs:simpleContent>
          </xs:complexType>
        </xs:element>
        <xs:element name="Identifier">
          <xs:complexType>
            <xs:simpleContent>
              <xs:extension base="xs:string">
                <xs:attribute type="xs:anyURI" name="Href"/>
              </xs:extension>
            </xs:simpleContent>
          </xs:complexType>
        </xs:element>
        <xs:element name="Language">
          <xs:complexType>
            <xs:simpleContent>
              <xs:extension base="xs:string">
                <xs:attribute type="xs:string" name="codeListID"/>
              </xs:extension>
            </xs:simpleContent>
          </xs:complexType>
        </xs:element>
<xs:element name="Publisher">
          <xs:complexType>
            <xs:simpleContent>
              <xs:extension base="xs:string">
                <xs:attribute type="xs:string" name="Href"/>
              </xs:extension>
            </xs:simpleContent>
          </xs:complexType>
        </xs:element>
        <xs:element type="xs:anyURI" name="ConformsTo"/>
        <xs:element type="xs:string" name="HasPart"/>
        <xs:element name="ProvidesDefinitionOf" maxOccurs="unbounded" minOccurs="0">
          <xs:complexType>
            <xs:simpleContent>
              <xs:extension base="xs:string">
                <xs:attribute type="xs:anyURI" name="Namespace" use="optional"/>
              </xs:extension>
            </xs:simpleContent>
          </xs:complexType>
        </xs:element>
        <xs:element name="Copyright">
          <xs:complexType>
            <xs:simpleContent>
              <xs:extension base="xs:string">
                <xs:attribute type="xs:anyURI" name="href"/>
              </xs:extension>
            </xs:simpleContent>
          </xs:complexType>
        </xs:element>
        <xs:element type="xs:string" name="ProtectiveMarking"/>
        <xs:element type="xs:string" name="Status"/>
        <xs:element type="xs:string" name="StatusDescription"/>
        <xs:element name="Category">
          <xs:complexType>
            <xs:simpleContent>
              <xs:extension base="xs:string">
                <xs:attribute type="xs:string" name="codeListID"/>
              </xs:extension>
            </xs:simpleContent>
          </xs:complexType>
        </xs:element>
        <xs:element type="xs:float" name="Version"/>
        <xs:element type="xs:string" name="Project"/>
        <xs:element type="xs:string" name="Title"/>
        <xs:element type="xs:string" name="Type"/>
      </xs:sequence>
    </xs:complexType>
  </xs:element>
</xs:schema>
</pre></code>
=== Differentiating Darwin and Incident Messages ===
If you have selected to receive both Real Time Incident messages and Darwin messages on the My Feeds page, then you will receive both message types on the Real Time Feed. To differentiate the messages, you can use values contained in the header of each message. The Darwin message header will always contain the FilterHeaderLevel field and the Incident message header will always contain "KNOWLEDGEBASE_DATA_TYPE"=>"INCIDENT_MESSAGE”. 
: The following example contains a Real Time Incident message along with a Darwin message:
<code><pre> C:\>ruby Darwin-pre.rb
################################################################
##########Stomp consumer for National Rail Data Portal##########
################################################################
Connected to ActiveMQ/5.14.1 server
-----------------------------------------------------------------
##Example for Darwin messages##
-----------------------------------------------------------------
Header for message is :
{
"content-length"=>"332",
"expires"=>"1493126390151",
"destination"=>"/queue/D376b51b89-ff57-409c-89ff-788a2399062b",
"subscription"=>"24d97d83ea2e55261e6fc7c5b491d22ac2b",
"priority"=>"0",
"CacheHeaderFilterLevels"=>"201704256780438,",
"FilterHeaderLevel"=>"TRAINSTATUS,",
"message-id"=>"ID:EC2D3-PRE-DB-35387-1493114307456-21:1831042",
"persistent"=>"true",
"timestamp"=>"1493126090154"
}
Decompressed Message is:
<?xml version="1.0" encoding="UTF-8"?>
<Pport xmlns="http://www.thalesgroup.com/rtti/PushPort/v12" xmlns:ns3="http://www.thalesgroup.com/rtti/PushPort/
asts/v2" ts="2017-04-25T14:14:50.6845894+01:00" version="STG-12.0"><uR updateOrigin="TD">
<TS rid="201704256780438" ssd="2017-04-25" uid="C80438">
<ns3:Location pta="14:13" ptd=6" tpl="ELYY" wta="14:12:30" wtd="14:16:30">
<ns3:arr at="14:14" src="TD"/>
<ns3:dep et="14:16" src="Darwin"/>
<ns3:plat cisPlatsup="true" platsup="true">3B</ns3:plat>
</ns3:Locat/TS></uR></Pport>


-----------------------------------------------------------------
[[Category:National Rail Enquiries Data Feeds]]
##Example for New Incident messages##
-----------------------------------------------------------------
Header for message is :  
{
"content-length"=>"857",
"expires"=>"0",
"KNOWLEDGEBASE_DATA_TYPE"=>"INCIDENT_MESSAGE",
"INCIDENT_ID"=>"214453059CDC4017A0E32F3A6ECA0D45",
"destination"=>"/queue/D345d5d738-7713-4b04-aa62-ac02bcd391b3",
"subscription"=>"cfde902ac319642fcbb0e97e42a4e54a8b009624",
"priority"=>"0",
"INCIDENT_MESSAGE_STATUS"=>"NEW",
"breadcrumbId"=>"ID-EC2D3-PRE-DB-50643-1493047216416-0-5633",
"message-id"=>"ID:EC2D3-PRE-DB-47730-1493047193600-2:1:3:1:214111",
"persistent"=>"true",
"timestamp"=>"1493048298934"
}
Decompressed Message is:
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<uk.co.nationalrail.xml.incident.PtIncidentStructure xmlns:ns2="http://nationalrail.co.uk/xml/common" xmlns:ns3="http://nationalrail.co.uk/xml/incident">
<ns3:CreationTime>2017-03-21T16:17:00.000Z</ns3:CreationTime>
<ns3:ChangeHistory>
<ns2:ChangedBy>SPimperton</ns2:ChangedBy>
<ns2:LastChangedDate>2017-03-26T07:01:00.000+01:00</ns2:LastChangedDate></ns3:ChangeHistory>
<ns3:ParticipantRef>73913</ns3:ParticipantRef>
<ns3:IncidentNumber>214453059CDC4017A0E32F3A6ECA0D45</ns3:IncidentNumber>
<ns3:Version>20170326070125</ns3:Version>
<ns3:Source><ns3:TwitterHashtag>#SouthernStrike</ns3:TwitterHashtag></ns3:Source>
<ns3:ValidityPeriod><ns2:StartTime>2017-03-21T16:17:00.000Z</ns2:StartTime></ns3:ValidityPeriod>
<ns3:Planned>false</ns3:Planned>
<ns3:Summary>Possible industrial action to affect Southern services on Saturday 8 April</ns3:Summary>
<ns3:Description>
<p>The RMT Union have announced they are planning to take industrial action on Saturday 8 April.</p>
<p>It is not yet known how Southern services will be affected, if this industrial action goes ahead.</p>
<p><strong>Twitter:</strong><br />
If you would like to follow this incident on Twitter, please use <a href="https://twitter.com/hashtag/SouthernStrike?f=tweets">#SouthernStrike</a></p>
</ns3:Description>
<ns3:InfoLinks><ns3:InfoLink><ns3:Uri>http://www.nationalrail.co.uk/service_disruptions/160174.aspx</ns3:Uri><ns3:Label>nationalrail.co.uk</ns3:Label></ns3:InfoLink></ns3:InfoLinks>
<ns3:Affects><ns3:Operators><ns3:AffectedOperator><ns3:OperatorRef>SN</ns3:OperatorRef><ns3:OperatorName>Southern</ns3:OperatorName></ns3:AffectedOperator></ns3:Operators>
<ns3:RoutesAffected><p>Various Southern routes</p></ns3:RoutesAffected></ns3:Affects>
<ns3:ClearedIncident>false</ns3:ClearedIncident>
<ns3:IncidentPriority>2</ns3:IncidentPriority></uk.co.nationalrail.xml.incident.PtIncidentStructure>
</pre></code>

Latest revision as of 16:25, 24 December 2023

About

Knowledgebase is RDG’s Content Management System providing customer friendly information on the railways and associated products. It contains a wealth of static and real-time information about traveling by train on the GB rail network, such as information about station facilities, service disruption, and engineering work.

Data is available on the National Rail Data Portal through two subscription types:

  • Real Time incidents - for new, modified and deleted Incidents
  • Knowledgebase API - for requesting the latest information on the following static data
    • Companies (TOCs)
    • Incidents (Service Disruptions and Engineering Works)
    • National Service Indicator (NSI)
    • Promotions (Public)
    • Stations
    • Ticket Restrictions
    • Ticket Types

Accessing the Knowledgebase Feeds

To access the RDG Knowledgebase feeds, you must have an account on the National Rail Data Portal (NRDP) at http://opendata.nationalrail.co.uk. By creating an account, you can register for a subscription to the Knowledgebase API and the Knowledgebase Real Time Feeds.

To receive Knowledgebase Real Time Incidents, select Knowledgebase (KB) Real Time Incidents under the Knowlegebase (KB) subscriptions. The connection information required to start receiving Real Time Incidents is located on the My Feeds page.

To receive Knowldegebase static data, select Knowledgebase (KB) API under the Knowledgebase (KB) subscriptions. The My Feeds page will then display details about accessing the latest versions of Ticket Restrictions, Companies, Incidents, National Service Indicator, Promotions (Public), Stations and Ticket Types files from Knowledgebase.

Important - Please note NRDP accounts expire after an extended period of inactivity. The unused account expiry period is currently set to 30 days. If you create an account and do not consume any of the feeds during this time your account will be deleted. If your account has been deleted, you will receive a notification email, and you will be able to re-register for a new account.

Internal KB URLs

Some endpoints are currently accessible without authentication on subdomains reserved for internal use only. Internal KB was originally firewalled, this was removed in March of 2020, but it may return to being firewalled without warning. If a URL you're using starts with anything other than opendata.nationalrail.co.uk, you're probably in the wrong place!

RDG Knowledgebase APIs

Knowledgebase API

After selecting the Knowledgebase (KB) API checkbox subscription type on NRDP, you can receive the Knowledgebase xml files in two steps via a REST client.

Generate Authentication Token

See the Authentication section on the National Rail Data Portal article

Make a call for each file

Once you have a valid authentication token, you can make an HTTP GET request. Supply the authentication token in a header named `X-Auth-Token`: 

  GET https://opendata.nationalrail.co.uk/api/staticfeeds/4.0/ticket-restrictions

   X-Auth-Token: jane@doe.com:1491312310772:56c56baa3e56d35ff0ede4a6aad1bcfb

A successful response will result in the requested file: 

   <?xml version="1.0" encoding="utf-8"?>
   <TicketRestrictions
   ...

If authorization failed, the response will be an HTTP 401 Unauthorized: 

   {
       "timestamp": "2019-02-13T11:50:16.048+0000",
       "status": 401,
       "error": "Unauthorized",
       "message": "Unauthorized",
       "path": "/api/staticfeeds/4.0/ticket-restrictions"
   }

If you are not subscribed to the Knowledgebase API, the response will be an HTTP 403 Forbidden: 

   {
       "timestamp": "2019-02-13T11:51:29.992+0000",
       "status": 403,
       "error": "Forbidden",
       "message": "Forbidden",
       "path": "/api/staticfeeds/4.0/ticket-restrictions"
   }

Stations

In the stations data, London St Pancras and Ashford International both have have two entries each, intended to correspond to domestic services and facilities (STP and AFK), and those dedicated to international facilities (SPX and ASI). These second entries are not maintained, and these CRS codes, derived from underlying Network Rail data, are not recognised by LDB, or in the Darwin push port reference data, which mantains the National Rail convention of one CRS code per station (here STP and AFK). These entries bear confusingly similar names to the more canonical ones, and may be sorted before them, something to consider if you're allowing a user to search stations.

Real Time Incident Feed

The real-time incidents feed details the creation, modification, and deletion of Knowledgebase incidents in real-time. The underlying information is the same as is available through the HTTP API (which can be used as a snapshot, if necessary), but its real-time nature is useful for tracking state changes more reliably, efficiently, and quickly. Messages from the real-time incidents feed can be received using either STOMP or Openwire. After selecting the Knowledgebase real-time incidents checkbox on the edit account page, connection information (host, credentials, and topic name) will now be shown in the relevant section on your My Feeds page.

Please note:

  • If using STOMP, your client must support STOMP v1.2.
  • Knowledgebase message bodies are not compressed, unlike the Darwin push port.
  • The opening XML tag is malformed, and should be PtIncident instead of uk.co.nationalrail.xml.incident.PtIncidentStructure. Simple string substitution can resolve this if it presents a problem using the schema. This issue does not affect the incidents file retrievable over HTTP.
  • Topic advisory messages are disabled on the KB Real Time Incidents messaging host, so you must also connect with this facility disabled if using Openwire (e.g. by setting watchTopicAdvisories=false if using NMS)
  • The feed is relatively low volume with typically only a few messages per hour and often gaps of several hours overnight; if monitoring message volumes, or testing, you should take this into account.

Headers

Each frame from the real-time incidents feed contains the following headers:

  • INCIDENT_MESSAGE_STATUS: either NEW (a new incident message), MODIFIED (a change to existing incident information), or REMOVED (incident deletion)
  • KNOWLEDGEBASE_DATA_TYPE, set to INCIDENT_MESSAGE.
  • INCIDENT_ID: the ID of the KB incident

Example Real Time Incidents Bodies

The following are examples of real-time incidents messages which include new, modified and deleted incidents.

New Incident message

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<uk.co.nationalrail.xml.incident.PtIncidentStructure xmlns:ns2="http://nationalrail.co.uk/xml/common" xmlns:ns3="http://nationalrail.co.uk/xml/incident">
<ns3:CreationTime>2017-03-21T16:17:00.000Z</ns3:CreationTime>
<ns3:ChangeHistory><ns2:ChangedBy>SPimperton</ns2:ChangedBy>
<ns2:LastChangedDate>2017-03-26T07:01:00.000+01:00</ns2:LastChangedDate></ns3:ChangeHistory>
<ns3:ParticipantRef>73913</ns3:ParticipantRef>
<ns3:IncidentNumber>214453059CDC4017A0E32F3A6ECA0D45</ns3:IncidentNumber>
<ns3:Version>20170326070125</ns3:Version>
<ns3:Source><ns3:TwitterHashtag>#SouthernStrike</ns3:TwitterHashtag></ns3:Source>
<ns3:ValidityPeriod><ns2:StartTime>2017-03-21T16:17:00.000Z</ns2:StartTime></ns3:ValidityPeriod>
<ns3:Planned>false</ns3:Planned>
<ns3:Summary>Possible industrial action to affect Southern services on Saturday 8 April</ns3:Summary>
<ns3:Description>
<p>The RMT Union have announced they are planning to take industrial action on Saturday 8 April.</p>
<p>It is not yet known how Southern services will be affected, if this industrial action goes ahead.</p>
<p><strong>Twitter:</strong><br />
If you would like to follow this incident on Twitter, please use <a href="https://twitter.com/hashtag/SouthernStrike?f=tweets">#SouthernStrike</a></p>
</ns3:Description>
<ns3:InfoLinks>
<ns3:InfoLink><ns3:Uri>http://www.nationalrail.co.uk/service_disruptions/160174.aspx</ns3:Uri>
<ns3:Label>nationalrail.co.uk</ns3:Label></ns3:InfoLink></ns3:InfoLinks>
<ns3:Affects><ns3:Operators><ns3:AffectedOperator><ns3:OperatorRef>SN</ns3:OperatorRef>
<ns3:OperatorName>Southern</ns3:OperatorName></ns3:AffectedOperator></ns3:Operators>
<ns3:RoutesAffected><p>Various Southern routes</p></ns3:RoutesAffected></ns3:Affects>
<ns3:ClearedIncident>false</ns3:ClearedIncident>
<ns3:IncidentPriority>2</ns3:IncidentPriority></uk.co.nationalrail.xml.incident.PtIncidentStructure>
...

Modified Incident Message

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<uk.co.nationalrail.xml.incident.PtIncidentStructure xmlns:ns2="http://nationalrail.co.uk/xml/common" xmlns:ns3="http://nationalrail.co.uk/xml/incident">
<ns3:CreationTime>2017-04-24T17:22:00.000+01:00</ns3:CreationTime>
<ns3:ChangeHistory><ns2:ChangedBy>bives</ns2:ChangedBy>
<ns2:LastChangedDate>2017-04-24T17:32:00.000+01:00</ns2:LastChangedDate>
</ns3:ChangeHistory><ns3:IncidentNumber>8B3B4C1F358A45A186F6F95984F52F4F</ns3:IncidentNumber>
<ns3:Version>20170424173219</ns3:Version>
<ns3:Source><ns3:TwitterHashtag>#CrystalPalace</ns3:TwitterHashtag></ns3:Source>
<ns3:ValidityPeriod><ns2:StartTime>2017-04-24T17:22:00.000+01:00</ns2:StartTime></ns3:ValidityPeriod>
<ns3:Planned>false</ns3:Planned><ns3:Summary>Delays in theCrystal Palace area expected until 20:15</ns3:Summary>
<ns3:Description>
<p>Due <span>to trespassers on the railway at Crystal Palace trains have to run at reduced speed on all lines.
<br /><br /><strong>Impact:</strong>
<br />Train services running through this station may be cancelled or delayed by up to20 minutes. Disruption is expected until 20:15 24/04.<br />
<br /><strong>Customer Advice:</strong>
<br />Owing to a passenger forcing the doors open and exiting the train onto the trackin the Crystal Palace area trains are obliged to run at a reduced speed.<br/></span></p>
<p><strong>Twitter:</strong><br />If you would like to follow this incident on Twitter, please use
 <a href="https://twitter.com/hashtag/CrystalPalace?f=tweets">#</a><span><a href="https://twitter.com/hashtag/CrystalPalace?f=tweets">CrystalPalace</a> </span></p>
 <p><strong>Check before you travel:</strong><br />Please use the National Rail Enquiries real-time
 <a href="http://ojp.nationalrail.co.uk/service/planjourney/search">Journey Planner</a> to check your journey before you travel.</p>
 <p><strong>Compensation:</strong><br />You may be entitled to <a href="http://www.nationalrail.co.uk/times_fares/ticket_types/121354.aspx">
 compensation</a> if you experience a delay in completing your journey today.
 Please keep your train ticket and make a note of your journey, as both will be required to support any claim.</p>
 <p><strong>Feedback:</strong><br />We want to make information better - tell us how! Fill out this online
 <a href="http://survey3.accent-mr.com/D4/(S(dtxglkj3ksyum52j0j01qpio))/2908m.aspx?urn=opensurveyNRwebsite">Disruption Survey.</a></p></ns3:Description>
<ns3:InfoLinks><ns3:InfoLink><ns3:Uri>http://www.nationalrail.co.uk/service_disruptions/162083.aspx</ns3:Uri>
<ns3:Label>nationalrail.co.uk</ns3:Label></ns3:InfoLink>
<ns3:InfoLink><ns3:Uri>http://www.nationalrail.co.uk/static/documents/maps/Crystal2404.pdf</ns3:Uri>
<ns3:Label>Additional Maps</ns3:Label></ns3:InfoLink></ns3:InfoLinks>
<ns3:Affects><ns3:Operators><ns3:AffectedOperator><ns3:OperatorRef>SN</ns3:OperatorRef>
<ns3:OperatorName>Southern</ns3:OperatorName></ns3:AffectedOperator></ns3:Operators>
<ns3:RoutesAffected><p>Routes through Crystal Palace</p></ns3:RoutesAffected></ns3:Affects>
<ns3:ClearedIncident>false</ns3:ClearedIncident>
<ns3:IncidentPriority>2</ns3:IncidentPriority></uk.co.nationalrail.xml.incident.PtIncidentStructure>
...

Removed Incident Message

Removed messages contain empty bodies. 



Support


KnowledgeBase documentation and schemas:












National Rail Enquiries Data Feeds





Data Feeds

 About the Feeds •  Darwin Webservice (Public) •   Darwin Webservice (Staff) •  Historical Service Performance  •  Push Port  • KnowledgeBase • DTD • Locations (PoC) • Real Time Journey Planner





LDB API

 About 





LDB-SV API

 About 





HSP

 About 





DTD

 About  •  Fares •  Timetable





Push Port

 About  •  XML Schemas  •  Schedules  •  Associations  •  Train Status  •  Station Messages  •  Alarms •  Train Order •  Train Alerts •  Formations  •  Formation loading







Retrieved from "https://wiki.openraildata.com/index.php?title=KnowledgeBase&oldid=3331"