Difference between revisions of "KnowledgeBase"

From Open Rail Data Wiki
Jump to: navigation, search
(Example Real Time Incidents)
(An Example STOMP script: Darwin-pre.rb)
Line 228: Line 228:
  
 
=== An Example STOMP script: Darwin-pre.rb ===
 
=== An Example STOMP script: Darwin-pre.rb ===
 +
The STOMP client script below can be used as a template for receiving the real time incident messages on a STOMP client.
 +
<code><pre>require "stomp"
 +
require "zlib"
 +
require "stringio"
 +
class NrPollerPoller
 +
  # Initialize the poller
 +
  def initialize
 +
    @hostname = 'datafeeds.nationalrail.co.uk'
 +
    @username = 'd3user'
 +
    @password = 'd3password'
 +
    puts "########################################################"
 +
    puts "#####Stomp consumer for National Rail Data Portal ######"
 +
    puts "########################################################"
 +
  end
 +
# Connect to the service and process messages
 +
  def run
 +
    port = 61613
 +
    queue = "/queue/NRDP_QUEUE"
 +
    client = Stomp::Client.new( "d3user", "d3password",  "datafeeds.nationalrail.co.uk", port )
 +
    # Check we have connected successfully
 +
    raise "Connection failed" unless client.open?
 +
    raise "Connect error: #{client.connection_frame().body}" if client.connection_frame().command == Stomp::CMD_ERROR
 +
    puts "Connected to #{client.connection_frame().headers['server']} server "
 +
    # Subscribe to my user's queue and process messages
 +
    client.subscribe queue, { :ack => :client } do | msg |
 +
        client.acknowledge msg # tell the server the message was handled and to dispose of it
 +
        # Now unzip the string
 +
        gz = Zlib::GzipReader.new( StringIO.new( msg.body ) )
 +
        xml = gz.read
 +
        puts "Decompressed Message is: #{xml}"
 +
    end
 +
    client.join
 +
    # We will probably never end up here
 +
    client.close
 +
    puts "Client close complete"
 +
  end
 +
end
 +
e = NrPollerPoller.new
 +
e.run
 +
</pre></code>

Revision as of 12:54, 28 April 2017

About

KnowledgeBase is RDG’s Content Management System providing customer friendly information on the railways and associated products. KnowledgeBase data is available on National Rail Data Portal through 2 subscription types

  • Real Time Data Feeds – for new, modified and deleted Incidents
  • On Demand Data Feeds – for requesting the latest information on the following
    • Ticket Restrictions
    • Companies
    • Incidents
    • National Service Indicator
    • Promotions (Public)
    • Stations
    • Ticket Types

Accessing the KnowledgeBase Feeds

To access the RDG KnowledgeBase feeds, you must have an account on the National Rail Data Portal (NRDP) at http://datafeeds.nationalrail.co.uk. After logging in, you will see 3 subscription types: Historical Service Performance (HSP), Real Time Data Feeds and On Demand Data Feeds. By selecting the checkbox for each of these subscription types, the respective sections become visible on the My Feeds page.

The previous Darwin Data Feed on the NRDP Account page has been replaced by the Real Time Data Feed which once selected include details about the Darwin push-port feeds and the KnowledgeBase – Real Time Incident feed in the My Feeds page.

To receive KnowledgeBase Real Time Incidents, select Real Time Data Feeds under Subscription Type in the Edit User Details page and then select the Update button to save this change. Go to the My Feeds page and select the KnowledgeBase data type – Real Time Incidents option. You will need the Real Time Information /STOMP details to start receiving real time incident messages via a STOMP client.

The On Demand Data Feeds section on the My Feeds page contains details about accessing the latest versions of Ticket Restrictions, Companies, Incidents, National Service Indicator, Promotions (Public), Stations and Ticket Types files from KnowledgeBase. To receive these KnowledgeBase files, select the On Demand Data Feeds checkbox under the Subscription type in the Edit User Details page and then select the update button to save this change.

RDG KnowledgeBase APIs

On Demand Data Feeds

After selecting the On Demand checkbox subscription type on NRDP, you can receive the KnowlegeBase xml files in 2 steps via a rest client.

Step 1: Generate Token

In this step, using a REST client you perform an HTTP POST call to https://datafeeds.nationalrail.co.uk/authenticate. This step requires using your email address and password registered on NRDP and will produce a token key. The following example shows the HTTP POST request and response with NRDP:

Example HTTP POST Request to NRDP:
 POST https://datafeeds.nationalrail.co.uk/authenticate HTTP/1.1
   Content-Type: application/x-www-form-urlencoded
   Accept: application/json, text/plain, */*
   Raw payload: username=user1@gmail.com&password=P@55w0rd1
   //Add your registered NRDP email address and NRDP password in the “Raw payload” section above 
Corresponding example HTTP Response from NRDP:
 {
 "username": " user1@gmail.com ",
 "roles": {
 "Role(id=6ab28d46-0e4a-11e7-9339-060a84373309, name='ROLE_KB_USER')": true,
 "Role(id=47d36dc4-16b9-11e6-9746-060a84373309, name='ROLE_HSP_USER')": true,
 "Role(id=4e660138-bb39-11e4-9da1-060a84373309, name='ROLE_D3_USER')": true
 },
 "token": " user1@gmail.com:1491312310772:56c56baa3e56d35ff0ede4a6aad1bcfb"
 } 

Each token is valid for 1 hour and you will receive a 401 unauthorized response when the token has expired. The following http response example is generated after sending an http request to a NRDP knowledgebase feed with an expired token.

Example HTTP 401 Response Header: Token Expired:
 Cache-Control: no-cache, no-store, max-age=0, must-revalidate
 Content-Language: en
 Content-Length: 1081
 Content-Type: text/html;charset=utf-8
 Date: Wed, 05 Apr 2017 09:52:11 GMT
 Expires: 0
 Pragma: no-cache
 Server: Apache-Coyote/1.1
 X-Content-Type-Options: nosniff
 X-Frame-Options: DENY
 X-Xss-Protection: 1; mode=block 
Corresponding example HTML Response file: Token Expired:
 <html>
 <head>
 <title>Apache Tomcat/7.0.56 - Error report</title>
 <style>
 <!--H1 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:22px;} H2 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-
 size:16px;} H3 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:14px;} BODY {font-family:Tahoma,Arial,sans-serif;color:black;background-
 color:white;} B {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;} P {font-family:Tahoma,Arial,sans-serif;background:white;color:black;font-size:12px;}A 
 {color : black;}A.name {color : black;}HR {color : #525D76;}-->
 </style> </head>
 <body>
 <h1>HTTP Status 401 - Unauthorized: Authentication token was either missing or invalid.</h1>
 <HR size="1" noshade="noshade">
 <p>
 <b>type</b> Status report</p>
 <p>
 <b>message</b> <u>Unauthorized: Authentication token was either missing or invalid.</u>
 </p>
 <p>
 <b>description</b> <u>This request requires HTTP authentication.</u>
 </p>
 <HR size="1" noshade="noshade">
 <h3>Apache Tomcat/7.0.56</h3>
 </body>
 </html> 

Step 2: Perform a HTTP GET call for each KnowledgeBase file

In this step using a REST client, you would perform a HTTP GET call to each Knowledgebase file using the token key produced in step 1. This will produce the xml file for the corresponding KnowledgeBase feeds. The following are the required links to make the respective HTTP GET call:

The header of the HTTP response will also include the date when the xml file was last modified using this format: Last-Modified: DAY, DD MMM YYYY HH:MM:SS GMT The following is an example of request/response with Ticket Restrictions Knowledgebase Feed via NRDP:

Example HTTP GET Request to NRDP knowledgebase feed:
GET https://datafeeds.nationalrail.co.uk/api/staticfeeds/4.0/ticket-restrictions HTTP/1.1
Content-Type: application/json
Accept: */*
X-Auth-Token: NRDP_email_address:1491312310772:56c56baa3e56d35ff0ede4a6aad1bcfb
//Enter your NRDP email address and the Token Key used in step 1 above 
Corresponding example HTTP 200 Response Header:
 Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Content-Length: 1194601
Content-Type: text/plain;charset=ISO-8859-1
Date: Wed, 05 Apr 2017 08:29:16 GMT
Expires: 0
Last-Modified: Wed, 05 Apr 2017 08:16:39 GMT
Pragma: no-cache
Server: Apache-Coyote/1.1
X-Content-Type-Options: nosniff
X-Frame-Options: DENY
X-Xss-Protection: 1; mode=block
Corresponding XML Response file:
 <?xml version="1.0" encoding="utf-8"?>
<TicketRestrictions xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
xmlns:com="http://nationalrail.co.uk/xml/common" xsi:schemaLocation="http://internal.nationalrail.co.uk/xml/XsdSchemas/External/Version4.0/nre-ticket-restriction-v4-0.xsd" 
 xmlns="http://nationalrail.co.uk/xml/ticketrestriction">
  <TicketRestriction>
    <Name><![CDATA[S2]]></Name><LinkToDetailPage><![CDATA[http://int.web.kb.awsnre.co.uk/times_fares/ticket_types/145184.aspx]]></LinkToDetailPage>
    <RestrictionCode>S2</RestrictionCode>
    <TicketRestrictionIdentifier>52CD8939787B42789817C8E83A38F74E</TicketRestrictionIdentifier>
    <ApplicableDays><![CDATA[<p>TEST SM 19/01</p>]]></ApplicableDays>
    <OutwardDirection>Outward Travel</OutwardDirection>
    <ReturnDirection>Return Travel</ReturnDirection>
    <RestrictionsType>2</RestrictionsType>
    <Restrictions>
      <Restriction>
        <DetailsOutward><![CDATA[<p>SM 2 </p>]]></DetailsOutward>
        <DetailsReturn><![CDATA[<p>SM TEST 2 </p>]]></DetailsReturn>
      </Restriction>
    </Restrictions>
  </TicketRestriction> 

Real Time Incident Feed

After selecting the Real Time Data Feed checkbox subscription type on account page & Real Time Incidents checkbox on My Feeds page, you can receive the KnowlegeBase xml file via a STOMP client as follows:

Using the STOMP client, connect to https://datafeeds.nationalrail.co.uk/ with the login credential shown on My Feeds/ Real Time Feed Information using your stomp script:
Username: D3user
Password: D3password
Queue name: This is the unique value for your account

Upon successfully connecting to ActiveMQ via the STOMP client, you will start receiving each real time incident message, with its respective incident number and other message information. The header of each message will contain one of the following incident change statuses:

New: This status represents new incident message information
Modified: This status represents modifications to existing incidents message information
Deleted: This status represents deleted message information

Example Real Time Incidents

The following are examples of Real Time Incidents which include new, modified and deleted incident messages:

 C:\>ruby Darwin-pre.rb
 ################################################################
 ##########Stomp consumer for National Rail Data Portal##########
 ################################################################
 Connected to ActiveMQ/5.14.1 server
 -----------------------------------------------------------------
 ##Example New Incident messages##
 -----------------------------------------------------------------
 Header for message is :
 {
 "content-length"=>"857", 
 "expires"=>"0",
 "KNOWLEDGEBASE_DATA_TYPE"=>"INCIDENT_MESSAGE", 
 "INCIDENT_ID"=>"214453059CDC4017A0E32F3A6ECA0D45", 
 "destination"=>"/queue/D345d5d738-7713-4b04-aa62-ac02bcd391b3",
 "subscription"=>"cfde902ac319642fcbb0e97e42a4e54a8b009624",
 "priority"=>"0", "INCIDENT_MESSAGE_STATUS"=>"NEW",
 "breadcrumbId"=>"ID-EC2D3-PRE-DB-50643-1493047216416-0-5633",
 "message-id"=>"ID:EC2D3-PRE-DB-47730-1493047193600-2:1:3:1:214111",
 "persistent"=>"true",
 "timestamp"=>"1493048298934"
 }
Decompressed Message is: 
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<uk.co.nationalrail.xml.incident.PtIncidentStructure xmlns:ns2="http://nationalrail.co.uk/xml/common" xmlns:ns3="http://nationalrail.co.uk/xml/incident">
<ns3:CreationTime>2017-03-21T16:17:00.000Z</ns3:CreationTime>
<ns3:ChangeHistory><ns2:ChangedBy>SPimperton</ns2:ChangedBy>
<ns2:LastChangedDate>2017-03-26T07:01:00.000+01:00</ns2:LastChangedDate></ns3:ChangeHistory>
<ns3:ParticipantRef>73913</ns3:ParticipantRef>
<ns3:IncidentNumber>214453059CDC4017A0E32F3A6ECA0D45</ns3:IncidentNumber>
<ns3:Version>20170326070125</ns3:Version>
<ns3:Source><ns3:TwitterHashtag>#SouthernStrike</ns3:TwitterHashtag></ns3:Source>
<ns3:ValidityPeriod><ns2:StartTime>2017-03-21T16:17:00.000Z</ns2:StartTime></ns3:ValidityPeriod>
<ns3:Planned>false</ns3:Planned>
<ns3:Summary>Possible industrial action to affect Southern services on Saturday 8 April</ns3:Summary>
<ns3:Description>
<p>The RMT Union have announced they are planning to take industrial action on Saturday 8 April.</p>
<p>It is not yet known how Southern services will be affected, if this industrial action goes ahead.</p>
<p><strong>Twitter:</strong><br />
If you would like to follow this incident on Twitter, please use <a href="https://twitter.com/hashtag/SouthernStrike?f=tweets">#SouthernStrike</a></p>
</ns3:Description>
<ns3:InfoLinks>
<ns3:InfoLink><ns3:Uri>http://www.nationalrail.co.uk/service_disruptions/160174.aspx</ns3:Uri>
<ns3:Label>nationalrail.co.uk</ns3:Label></ns3:InfoLink></ns3:InfoLinks>
<ns3:Affects><ns3:Operators><ns3:AffectedOperator><ns3:OperatorRef>SN</ns3:OperatorRef>
<ns3:OperatorName>Southern</ns3:OperatorName></ns3:AffectedOperator></ns3:Operators>
<ns3:RoutesAffected><p>Various Southern routes</p></ns3:RoutesAffected></ns3:Affects>
<ns3:ClearedIncident>false</ns3:ClearedIncident>
<ns3:IncidentPriority>2</ns3:IncidentPriority></uk.co.nationalrail.xml.incident.PtIncidentStructure>
----------------------------------------------------------------
##Example Modified Incident Message##
----------------------------------------------------------------
Header for message is : {"content-length"=>"1369", "expires"=>"0", "KNOWLEDGEBAS E_DATA_TYPE"=>"INCIDENT_MESSAGE", "INCIDENT_ID"=>"8B3B4C1F358A45A186F6F95984F52F4F", "destination"=>"/queue/D345d5d738-7713-4b04-aa62-ac02bcd391b3", "subscription"=>"cfde902ac319642fcbb0e97e42a4e54a8b009624", "priority"=>"0", "INCIDENT_MESS AGE_STATUS"=>"MODIFIED", "breadcrumbId"=>"ID-EC2D3-PRE-DB-42611-1493050980100-0- 1259", "message-id"=>"ID:EC2D3-PRE-DB-60889-1493050957804-2:1:4:1:84412", "persistent"=>"true", "timestamp"=>"1493051706498"}
Decompressed Message is: <?xml version="1.0" encoding="UTF-8" standalone="yes"?><uk.co.nationalrail.xml.incident.PtIncidentStructure xmlns:ns2="http://nationalrail.co.uk/xml/common" xmlns:ns3="http://nationalrail.co.uk/xml/incident"><ns3:CreationTime>2017-04-24T17:22:00.000+01:00</ns3:CreationTime><ns3:ChangeHistory><ns2:ChangedBy>bives</ns2:ChangedBy><ns2:LastChangedDate>2017-04-24T17:32:00.000+01:00</ns2:LastChangedDate></ns3:ChangeHistory><ns3:IncidentNumber>8B3B4C1F358A45A186F6F95984F52F4F</ns3:IncidentNumber><ns3:Version>20170424173219</ns3:Version><ns3:Source><ns3:TwitterHashtag>#CrystalPalace</ns3:TwitterHashtag></ns3:Source><ns3:ValidityPeriod><ns2:StartTime>2017-04-24T17:22:00.000+01:00</ns2:StartTime></ns3:ValidityPeriod><ns3:Planned>false</ns3:Planned><ns3:Summary>Delays in theCrystal Palace area expected until 20:15</ns3:Summary><ns3:Description><p>Due <span>to trespassers on the railway at Crystal Palace trains have to run at reduced speed on all lines.<br /><br /><strong>Impact:</strong><br />Train services running through this station may be cancelled or delayed by up to20 minutes. Disruption is expected until 20:15 24/04.<br /><br /><strong>Customer Advice:</strong><br />Owing to a passenger forcing the doors open and exiting the train onto the trackin the Crystal Palace area trains are obliged to run at a reduced speed.<br/></span></p><p><strong>Twitter:</strong><br />If you would like to follow this incident on Twitter, please use <a href="https://twitter.com/hashtag/CrystalPalace?f=tweets">#</a><span><ahref="https://twitter.com/hashtag/CrystalPalace?f=tweets">CrystalPalace</a> </span></p><p><strong>Check before you travel:</strong><br />Please use the National Rail Enquiries real-time <a href="http://ojp.nationalrail.co.uk/service/planjourney/search">Journey Planner</a> to check your journey before you travel.</p><p><strong>Compensation:</strong><br />You may be entitled to <a href="http://www.nationalrail.co.uk/times_fares/ticket_types/121354.aspx"> compensation</a> if you experience a delay in completing your journey today. Please keep your train ticket and make a note of your journey, as both will be required to support any claim.</p><p><strong>Feedback:</strong><br />We want to make information better - tell us how! Fill out this online <a href="http://survey3.accent-mr.com/D4/(S(dtxglkj3ksyum52j0j01qpio))/2908m.aspx?urn=opensurveyNRwebsite">Disruption Survey.</a></p></ns3:Description><ns3:InfoLinks><ns3:InfoLink><ns3:Uri>http://www.nationalrail.co.uk/service_disruptions/162083.aspx</ns3:Uri><ns3:Label>nationalrail.co.uk</ns3:Label></ns3:InfoLink><ns3:InfoLink><ns3:Uri>http://www.nationalrail.co.uk/static/documents/maps/Crystal2404.pdf</ns3:Uri><ns3:Label>Additional Maps</ns3:Label></ns3:InfoLink></ns3:InfoLinks><ns3:Affects><ns3:Operators><ns3:AffectedOperator><ns3:OperatorRef>SN</ns3:OperatorRef><ns3:OperatorName>Southern</ns3:OperatorName></ns3:AffectedOperator></ns3:Operators><ns3:RoutesAffected><p>Routes through Crystal Palace</p></ns3:RoutesAffected></ns3:Affects><ns3:ClearedIncident>false</ns3:ClearedIncident><ns3:IncidentPriority>2</ns3:IncidentPriority></uk.co.nationalrail.xml.incident.PtIncidentStructure>
----------------------------------------------------------------
##Example Deleted Incident Message##
----------------------------------------------------------------
Header for message is : 
{
"content-length"=>"20",
 "expires"=>"0",
 "KNOWLEDGEBASE_DATA_TYPE"=>"INCIDENT_MESSAGE",
 "INCIDENT_ID"=>"05A3E842D53C4AAEA2EFAA8533EA84BE",
 "destination"=>"/queue/D345d5d738-7713-4b04-aa62-ac02bcd391b3",
 "subscription"=>"cfde902ac319642fcbb0e97e42a4e54a8b009624",
 "priority"=>"0",
 "INCIDENT_MESSAGE_STATUS"=>"REMOVED",
 "breadcrumbId"=>"ID-EC2D3-PRE-DB-50643-1493047216416-0-4165",
 "message-id"=>"ID:EC2D3-PRE-DB-47730-1493047193600-2:1:4:1:164882",
 "persistent"=>"true",
 "timestamp"=>"1493048178800"
 }
Decompressed Message is: (empty body)

An Example STOMP script: Darwin-pre.rb

The STOMP client script below can be used as a template for receiving the real time incident messages on a STOMP client.

require "stomp"
require "zlib"
require "stringio"
class NrPollerPoller
  # Initialize the poller
  def initialize
    @hostname = 'datafeeds.nationalrail.co.uk'
    @username = 'd3user'
    @password = 'd3password'
    puts "########################################################"
    puts "#####Stomp consumer for National Rail Data Portal ######"
    puts "########################################################"
  end
# Connect to the service and process messages
  def run
     port = 61613
     queue = "/queue/NRDP_QUEUE"
     client = Stomp::Client.new( "d3user", "d3password",  "datafeeds.nationalrail.co.uk", port )
     # Check we have connected successfully
     raise "Connection failed" unless client.open?
     raise "Connect error: #{client.connection_frame().body}" if client.connection_frame().command == Stomp::CMD_ERROR
     puts "Connected to #{client.connection_frame().headers['server']} server "
     # Subscribe to my user's queue and process messages
     client.subscribe queue, { :ack => :client } do | msg |
        client.acknowledge msg # tell the server the message was handled and to dispose of it
         # Now unzip the string
         gz = Zlib::GzipReader.new( StringIO.new( msg.body ) )
         xml = gz.read
         puts "Decompressed Message is: #{xml}"
     end
     client.join
     # We will probably never end up here
     client.close
     puts "Client close complete"
  end
end
e = NrPollerPoller.new
e.run