protocol.md

Tenthbit Protocol

About

The tenthbit protocol is a JSON-based communications protocol that offers the following advantages over IRC:

• Required SSL for client connections and intra-server linking, ensuring that a valid network doesn't put bare data on the wire
• Optional GZip compression when supported and beneficial, e.g. between server nodes
• Federated, decentralized authentication between independant servers, and possibly more later on
• A baked-in account system and access-control list (ACL) for channels, distributed across nodes for reliability
• Support for meshed links between server nodes, reducing impact of a node failure, provided by unique identifiers attached to every packet

Communications

Communications on a server are organized in to topics, much like channels on an IRC network. Topics are identified by UUIDs and by names; thus, topic names can be changed at any point. Each topic stores an ACL which determines who is authorized to complete certain actions. In addition, topics have a list of key/value pairs storing metadata such as the topic's name, a description, message format, and really anything else.

Message formats

A message format dictates how a client should encode and parse formatting into a topic's messages. Official formats are:

• plain: plain utf-8 text. No formatting should be done. Clients may use a variable-width font when displaying text.
• fixed: fixed-width utf-8 text. Like plain, except should be shown with a fixed-width font.
• markdown: utf-8 text, formatted using markdown. This will probably be default and should probably be used in discussion channels.
• html: utf-8 html markup, using a subset of html tags [to be clarified]
• latex: A message containing valid LaTeX.
• irc: ansi text like what would be used with an IRC server. Should be shown in a fixed-width font, and may contain mIRC-style formatting and color codes, which clients may or may not honor (but must at least strip if offering this format).
• binary: raw binary octets. Note that this is not particularly efficient, as the binary string will be encoded in JSON.

Third-parties may invent custom formats. When doing so, starting with an above format name and appending a colon and then a proprietary name allows nonsupporting clients to use the mentioned base format to at least somewhat understand what's going on; for example, fixed:ascii-art. Otherwise, clients should assume plain as the base format.

Access control list

TODO

Federation

At the moment, only the account system is federated (communications may be federated later). Essentially, an account that exists on one server (the account's "home" server) can be used to authenticate against other servers (servers which are "foreign" to that account). Once authentication succeeds, foreign servers are not required to fully authorize the nonlocal user as a local user.

Accounts are specified using an email-style notation; a user danopia on a server located at 10b.it would be globally addressable by [email protected]. However, other users accessing the 10b.it server may refer to the same user with simply danopia. Bare usernames act the same as usernames with the current server's name appended.

Foreign authentication flow

A ticket-based system is used to prevent credentials from flowing through potentially untrusted foreign servers. An alphanumeric ticket is generated by an account's home server via an established connection as per the user's request at any time, and must expire at some point in the future. [TODO: Pair a ticket with a certain remote server, and only let it work when presented by that certain server.]

A valid ticket may be presented to a foreign server during the authentication flow, along with the ticket's home server. For example, [email protected] received the ticket OHMY. danopia may connect to somewhere.else and request authentication with {ticket: "OHMY", server: "10b.it"}. somewhere.else SHOULD connect to 10b.it (if not already connected) and relay the ticket. 10b.it must reply with the corresponding username (in this case, danopia) if the ticket was valid, or an error otherwise. If the ticket existed, 10b.it MUST invalidate it so that further redemption attempts with the same ticket will fail.

Wire protocol

The 10bit protocol is JSON-based, and transported via a TLS connection to TCP port 10817. Every packet is serialized into a JSON representation, which is then sent over the SSL socket, terminated with a UNIX newline ("\n"). After the other party receives the packet and decrypts it, it can parse the JSON into whatever internal representation it likes most.

The SSL transport is basic for now, though SSL keys may later be used to authorize clients to their account. When available, the protocol should be advertised by the server as 10bit (via NPN or ALPN). A gzip transport may be enabled between SSL and the JSON by negotiating a 10bit-gzip protocol through NPN/ALPN. A server MAY choose to accept the gzip request when available and allowed.

Servers may also take advantage of the NPN/ALPN mechanism by offering a websocket transport. This allows for a clean webchat implementation. Consider that it's up to the server to deny webchats running on unknown domains, if desired. Websockets are a message-based protocol, unlike TCP's stream-based, so JSON should not be newline-terminated.

JSON payload

Most packets have a similar base structure, consisting of:

• id: ID; key containing a unique ID for the action which caused the packet. Useful for mesh linking and some context perks.
• ts: timestamp; UNIX timestamp for when the packet was first received/processed through a server node, in milliseconds.
• rm: room; Unique ID (hex string) identifying the topic that this payload is in reference to. Not present if not applicable.
• op: operation; Common ops are act, auth, join, leave, and meta.
• sr: source; origin of the action. May be a username (danopia), server (@10b.it), or federated user ([email protected]).
• ex: extra; op-specific data, in an object. Data in ex that can not be ignored is the same for all instances of a particular op. The data in ex MUST be transmitted as it was recieved by the server - that is, it should be considered to be immutable. ex MAY contain data other than what is set out in this document, but it will always be safe to ignore it and simply pass it on to the clients if acting as a server, or simply not handle if acting as a client.

Operations

• welcome: Sent by the server to initiate the connection. Contains limited server metadata, and more importantly, an array of supported authentication method names (array of strings) in the auth extra. Two additional extras are required: server and software; respectively, the hostname of the server, and the name and version of the software running it (e.g. "10bitd.js/0.1"). Both are strings.
• auth: Any authentication-related packet that isn't an error. More details TODO. Official methods may include password, ticket, anonymous, twostep, and ssl.
• meta, what it says on the tin. Indicates that the payload contains metadata about an object (the ID of that object is stored in the target extra), which is stored in the data extra. Includes a type extra to indicate whether the metadata is on a server, client, or topic.
• meta-get, again, straightforward. Indicates a request for metadata about an object, the ID of that object is stored in the target extra. Includes a type extra like sendmeta
• error: Attempts to convey some sort of protocol failure. May be followed by a dropped connection. TODO
• join: Sent by a client to request to join a room, and sent by a server to convey that a user has been added to a room's userlist. ex may contain arbitrary data. Acknowledgement is in the form of an error opcode, or a response join with an isack extra boolean set to true.
• leave: Like join, except that the user has been removed.
• disconnect: Similar to IRC's QUIT, disconnect can be sent by a client to request a clean disconnection from the server, or sent by a server to relay that a client has been disconnected (cleanly or not). A valid packet to disconnect from the server is {"op":"disconnect"}.
• find: Searches for objects (users, topics) by metadata. #TODO
• act: Room activity. Extras are mostly arbitrary, but a few key ones are listed below. Acknowledgement is in the form of an error opcode, or a response join with an isack extra boolean set to true.
  • A textual message is usually included in the message string extra.
  • A context string extra may be set to a previous packet's id field, defining a relationship between the two messages.
  • When context is set, an isrevision boolean extra can be true, asking clients to replace the older activity with the current one. Revisions should not fundamentally change the activity structure, but this is not illegal yet.
  • When the activity includes a message, setting isaction to true means the the message is in the third person and should be prefixed with the sender's name when shown. See also: IRC's /me command.
  • istyping and hastext boolean extras can be used to convey if a user is currently editing a message or has an unsent message entered. These are often sent without other extras.

Operations and their common ex fields

Operation	Common Fields
welcome	server (string), software (string), auth ([string])
auth	isack (boolean), method (string)
meta	target (string), data ({string:string}), type (string) (Tentative. TODO)
meta-get	target (string), type (string) (Tentative. TODO)
error	errnum (int), errmsg (string) (Tentative. TODO)
join	isack (boolean)
leave	isack (boolean)
find	TODO
act	isack (boolean), message (string)

Example Socket Converstions

Handshake

<-> ssl handshake, negotiation for protocol 10bit/0.1
<-- server sends op=welcome, id="3", ts=234297552342, ex={server: "10b.it", software: "10bit reference server/0.0.1", now: 1373552037052, auth: ["password", "ticket", "anonymous"]}
--> client sends op=auth, ex={method: "password", username: "danopia", password: "hellosecret"}
<-- op=auth, id="6", ts=234298352352, ex={method: "password", username: "danopia", isack: true}
<-- op=meta, id="7", ts=234298352353, sr="@10b.it", ex={...} # includes own metadata, like favorite topics and fullname
<-- op=join, id="8", ts=234298352364, sr="danopia", rm="deadbeef" # sent to everyone in room, since autojoined
<-- op=meta, id="9", ts=234298352366, sr="@10b.it", rm="deadbeef", ex={...} # topic metadata, also includes self in nicklist

Conversation

--> op=act, rm="deadbeef", ex={message: "message goes here"}
<-- op=act, id="59", rm="deadbeef", ts=234298362352, sr="lonestarr", ex={message: "message goes here", isack: true}
<-- op=act, id="63", rm="deadbeef", ts=234298366252, sr="bender",    ex={message: "response goes here"}