= TRAFFIC
Adam Evyčędo
clockworkOrange, 2024-02-09
:sectnums:
:toc:
== Abstract
TRAFFIC (Transport Related Augmented File Format Improved & Corrected) is a data format used to represent public transport timetables and related information.
Apart from defining format for data at rest, TRAFFIC describes an API used to exchange information between providers and clients.
TRAFFIC is inspired by https://gtfs.org[GTFS], but it’s more concise and not human-readable; data can be converted from GTFS format to TRAFFIC, and TRAFFIC is compatible with GTFS Realtime.
== Comments
Comments are solicited and should be addressed to the mailing list at traffic@git.apiote.xyz.
== Copyright Notice
Copyright (c) Adam Evyčędo
This work is licensed under the Creative Commons Attribution-ShareAlike 4.0 International License. To view a copy of this license, visit http://creativecommons.org/licenses/by-sa/4.0/ or send a letter to Creative Commons, PO Box 1866, Mountain View, CA 94042, USA.
== Requirements Language
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in https://www.rfc-editor.org/rfc/bcp14.html[BCP 14] https://www.rfc-editor.org/rfc/rfc2119.html[RFC2119] https://www.rfc-editor.org/rfc/rfc8174.html[RFC8174] when, and only when, they appear in all capitals, as shown here.
== Introduction
The primary goal set before authors of TRAFFIC is ease of use—for both, users of clients, and programmers implementing the spec.
This encompasses practicality and simplicity; TRAFFIC was created to be used even on less powerful devices.
That’s why TRAFFIC employs multiple indices, and—with the use of modern data structures—timetables use space more efficiently.
TRAFFIC is not meant to replace GTFS—indeed, it works in tandem with GTFS Realtime—and does not fully implement all edge cases covered by GTFS.
With time, when more and more schedules will be used by TRAFFIC servers, the specification will cover more and more real-world cases, but the goal of making passengers’ lives easier will always be first.
=== Difference from GTFS
The most visible difference between TRAFFIC and GTFS is data structure used by the standards.
Where GTFS uses zipped csv files and unnecessarily repeats much information, TRAFFIC encodes data in BARE structures, which together are packed into an xz compressed tar archive.
In TRAFFIC, there are few ID fields and file offsets are used to locate a specific record.
Enums are reordered so that when they specify an unknown value, it’s located at index zero, so that it’s a zero value of an enum.
== Versioning
TRAFFIC versions are named and not numbered.
Each version name starts with next letter of English alphabet and versions are named after metro lines.
Data format is not backwards compatible between versions.
API is backwards compatible; single version of API defines one or more versions of responses.
Clients MUST include `accept` headers with versions of responses they understand and server MUST respond with the most recent version that both the client asked for and the server can produce, or send an error response if agreement is not possible.
As an exception, version 0 is reserved for development and MUST be sent by server if requested, irrelevant of the highest version the server can produce.
== Names
* Timetable is a collection of information valid for specific time in one feed.
* Line is what GTFS names a Route
* Stop is a specific point where a vehicle stops to pick up or drop passengers. It may be further subdivided into platforms. Stop's name SHOULD contain the shortest ID distinguishing it from other stops of the same node.
* Node is an abstract collection of stops. For stops to be part of the same node, they must have the same `stop_name` in `stops.txt` file of GTFS feed.
* Stop code is a stable identifier, one that doesn't change between timetable changes. It may, for example, be used in 2D codes at stops, that link to a website with realtime departures for this stop.
== Data format specification
TRAFFIC file consists of multiple files defined below, packed into an xz compressed tar archive.
Each TRAFFIC file name must end with two dates in format `YYYYMMDD` which denote first and last day of the timetable validity, followed by `.tar.xz` or `.txz` extension.
=== Files definitions
Each file (aside from Lua scripts) consists of concatenated https://baremessages.org/[BARE] encoded structures, one after another.
At the top level, files do not contain lists of structures—there is no length at offset 0—because the length is not know in advance.
Should need arise to read the whole file, it must be read one structure at a time till EOF.
Usually, it’s enough to read one structure by skipping to a given offset and unmarshalling a single structure.
Schema for structures is defined in `traffic.bare` file.
==== agencies.bare
Consists of one or more `Agency` structures and defines firms which operate vehicles.
`timezone` is a name of IANA time zone for given agency.
`language` is a IETF BCP 47 language code.
==== calendar.bare
Consists of one or more `Schedule` structures and defines when a trip operates.
`weekdays` is a bitmap. 0th and 7th bit denote Sunday—both MUST be set for Sunday schedule; 1st through 6th bits denote respectively Monday through Saturday.
`startDate` and `endDate` MUST be in format `YYYYMMDD`.
==== feed_info.bare
Usually, consists of one `FeedInfo` structure, but MAY be empty.
It contains info about the particular timetable.
`language` is a IETF BCP 47 language code.
==== ix_lines.bare
Index of lines from line name to offset in `lines.bare` file.
Consists of multiple `NameOffset` structures.
`name` is latinised—according to feed rules—line name without non-alphanumeric characters in lower-case.
==== ix_stop_codes.bare
Index of stops from stop code to offset in `stops.bare` file.
Consists of multiple `CodeIndex`-es.
==== ix_stop_names.bare
Index of stops from stop name to offset in `stops.bare` file.
Consists of multiple `NameOffset` structures.
`name` is latinised—according to feed rules—line name without non-alphanumeric characters in lower-case.
==== ix_trips.bare
Index of trips from trip ID to offset in `trips.bare` file.
Consists of multiple `NameOffset` structures.
`name` is unchanged trip ID.
==== lines.bare
Consists of one or more `Line` structures and defines lines.
`nextNodes` in `LineGraph` define edges of the graph — from map key to each of its value for ``nextNodes`.
`nextNodes` value of `-1` means that key is a final terminus for some or all trips.
==== stops.bare
Consists of one or more `Stop` structures and defines stops.
Depending on the feed, it MAY contain unused stops, ones without any departures.
`zone` MAY be an empty string if there is just one zone without a name.
`timezone` is a name of IANA time zone for given stop.
==== trips.bare
Consists of one or more `Trip` structures and defines trips for lines.
Currently, `shapeID` is not used.
==== vehicles.bare
Consists of one or more `Vehicle` structures and defines vehicles on trips as defined by realtime information.
==== updates.lua
_This file is optional._
Contains single Lua function `getUpdates(tripID, sequence, stopID, stopCode)`.
`tripID`, `stopCode`, and `stopID` are strings, `sequence` is a number coresponding to stop sequence in trip from static timetable.
The function returns in case of success a json-encoded `LuaUpdates` and an empty string; an empty string and a json-encoded `LuaError` in case of errors.
LuaUpdates.Updates MAY contain an entry with key `''` which will be ignored.
=== Lua scripts
In Lua scripts used for accessing custom APIs following modules are available:
* http
** `get(url: string, options: {query: string, cookies: table, headers: table, timeout: string, auth: {user: string, pass: string}}): {body: string, body_size:number, headers: table, cookies: table, status_code: number} or (nil, error: string)`
* json
** `decode(j: string): (decoded_value: lvalue, error: string)`
** `encode(v: lvalue): (encoded_json: string, error: string)`
TODO http and json is avalable with functions …
== API specification
API is defined as https://spec.openapis.org/oas/v3.1.0.html[OpenAPI spec v3.1.0] in file `api/api.yml` and data structure is defined as https://baremessages.org/[BARE schema] in file `api/api.bare`.
The spec is extended in following ways:
* added types
[cols="1,1"]
|===
|type |description
|data
|unstructured binary (octet-stream) data
|===
* added formats to types which correspond to primitive types used in BARE encoding
[cols="1,1"]
|===
|format |BARE type
|bare-uint8
|u8
|bare-uint16
|u16
|bare-uint
|uint
|bare-int
|int
|bare-int8
|i8
|bare-float64
|f64
|bare-float32
|f32
|bare-bool
|bool
|===
* added generic formats to types
[cols="1,1"]
|===
|format |description
|uint
|unsigned integer
|===
* added keys `keys` and `items` to type `object`. Such object defines a map.
=== API specification customisation
==== API info
Fields under `.info` must be customised according to the entity responsible for the server.
==== API location
API specification file MUST be available at `/.well-known/traffic-api` and define API location in field `.servers.[$i].url`. First server MUST be a production server to be used by clients.
==== API security
API specification file MUST be available at `/.well-known/traffic-api` and define API security in field `.security`:
* if the only security schema is `{}`, the server is available without restrictions,
* if the only security schema is `api_key: []`, the server is private and all requests must include a valid key in `x-traffic-key` header,
* if both security schemas are present, the server is rate limited for requests without the `x-traffic-key` header.
The keys under `.security` key MUST NOT be empty.
=== Transport layer
All requests to the API MUST be performed via HTTP and they SHOULD be secured using TLS.
Apart from the specification response described in subsection 8.1.2, responses to all requests are encoded using https://baremessages.org[BARE].
[appendix]
== Version name
Clockwork Orange is an unofficial tourist name of Glasgow Subway (Fo-thalamh Ghlaschu).
It consists of Outer and Inner Circle lines going the same route clockwise and couterclockwise respectively.
The trains travel between 15 stations for 10.5 kilometers.
Batong (北京地铁八通线) is one of Beijing Subway lines.
This line extends Line 1.
It has 15 stations, is over 23 km long but runs only about 4 km underground.
It will take you to the Universal Beijing Resort.
AREX (Airport Railroad Express, 인천국제공항철도) is one of Seoul metro lines.
It has 14 stations, is over 63 km long and was opened in 2007.
It will take you from Seoul Station to Incheon Airport and back.