MARC

MARC is the Magic Anonet Resource Claims System. This document describes MARC version 2. See an older revision of this document for the previous, now deprecated version of the protocol.

Design requirements

• Distributes resource claims to all nodes in a network
• Unattended automatic synchronisation should be possible, but people should not be able to alter each others' data
• Support for real-time and offline synchronisation
• Flexible but reliable representation of complex structures (nested arrays, associative arrays, binary strings)
• Should be simple to implement, audit and use
• Conflicts should be prevented, and should not affect other resources or updates

Features

• Authenticated updates: only the first to claim a resource can update it afterwards (ownership based on first claim)
• Updates are cryptographically signed using the ed25519 public-key signature system
• Can use any transport that can transfer an arbitrary block of binary data for synchronization (eg HTTP, e-mail, USB stick, UDPMSG3, git, rsync, newsgroups)
• Can use any storage mechanism that can store blocks of binary data (SQL, flat files)
• Requires to store only the last known version of each resource (but can store older versions if desired)
• A node is allowed not to store unwanted data, or to ignore unwanted updates

How?

• Resources are identified by a label, a 0-255 byte binary string
• IP addresses, domain names and AS numbers are encoded into a label, all sharing the same namespace
• Resource details are represented in a complex structure (nested arrays, associative arrays, binary strings), which are encoded (binary format) into a binary string, the resource value
• Each resource has a serial number, a higher serial number represents a more recent version, and should normally replace an older version in the database
• Each resource has a status, which tells whether the resource is taken or has been released (and can be taken by someone else)
• Each time a resource is changed a message is generated, which contains the label, value, status and serial; this message is signed and the cryptographic public key is added to form the update message
• An update message always contains all resource details

Procedure for importing an update message

1. if a resource with the given label exists:
   1. if the current serial is higher than or equal to the update serial, ignore the update
   2. if the current status is not set to released and the key in the update is different from the current key, ignore the update
2. if the update message is not signed with the public key specified in the update message, ignore the update (this can be done earlier, but that would generally decrease performance)
3. optional: if the update is too big (per node configuration), ignore the update
4. optional: if the label is not acceptable (e.g. does not represent a valid IP network, AS number or domain name; node specific policy), ignore the update
5. optional: if the resource value is not acceptable (missing fields, unexpected formatting; node specific policy), ignore the update
6. optional: import the resource value into existing data structures (e.g. configuration file, resdb, SQL database)
7. store update message in the local database, identified/indexed by its label

Layout of an update message

• Version number (1 byte, value 2)
• Public key (32 bytes)
• Signature (64 bytes)
• Resource data (m bytes)
  • Status (1 byte, value 0=deleted, 1=claimed, 2=transfer, 3=released)
  • Serial (4 bytes, 32 bit big-endian integer)
  • Label length (1 byte)
  • Label (binary string)
  • Number of extensions (1 byte)
    • List of extensions: (n blocks)
      • Extension identifier (1 byte, value 1=transfer-to-key)
      • Extension data length (2 bytes, 16 bit big-endian integer)
      • Extension data
  • Encoded value (see Complex structure encoding, this block covers the remainder of the update message)

If Status is 2 (transfer) and an extension with identifier 1 (transfer-to-key) is defined, the specified public transfer-to-key is allowed to update the resource in addition to the owning key. If no transfer-to-key extension is specified, any key can be used to update the resource. Rules for checking the serial number and verification of the signature are not changed.

Complex structure encoding

Complex structures are encoded into a binary string as follows:

• NULL (no data)
  • Data type, 1 byte, value 0
• String (binary or text string, may contain a textual representation of a number)
  • Data type, 1 byte, value 1
  • String data
• List (ordered array with no otherwise relevant indices)
  • Data type, 1 byte, value 2
  • For each item in the collection:
    • Encoded item size in bytes (4 byte, 32 bit big-endian integer)
    • The item is recursively encoded using this scheme
• Dictionary (array with relevant indices, associative array or list of key-value pairs)
  • Data type, 1 byte, value 3
  • For each item in the dictionary:
    • Key or index string length in bytes, 1 bytes
    • Key or index string
    • Encoded item size in bytes (4 byte, 32 bit big-endian integer)
    • The item is recursively encoded using this scheme

Notes

• Resources in the deleted status may be removed from the database after some time
• For efficient client-server request-response synchronisation, the server should store the local update timestamp for each resource. The client can store the highest timestamp it has received from the server, and ask for resources newer than the previous timestamp.

Interactive HTTP synchronisation

This is currently the preferred method. The server runs a CGI script, for which the client makes either a GET, POST or PUT request.

The query string for the request contains at least one parameter, 'version'. This section describes protocol version 3. If the client wishes to receive/pull updates from the server, the query string should include a parameter 'get', with a value of either 0 (to retrieve all records) or the highest previous timestamp (see below).

If the client also wishes to push updates to the server, either the POST or PUT method should be used. For a POST request, the updates to push should be encoded according to the application/x-www-form-urlencoded MIME type, with field names of update[], and set the HTTP Content-Type header accordingly. For a PUT request, all updates are sent as one binary block of data, where each update is prefixed with its length in bytes encoded as a 32 bit big-endian integer.

The server should respond with the HTTP Content-Type header set to application/octet-stream. The response data looks as follows:

• Protocol version, 1 byte, value 3
• Encoded information, encoded using the Complex structure encoding, may contain a dictionary with the following keys:
  • 'imported': number of imported updates
  • 'exported': number of exported updates
  • 'maxtimestamp': a string containing the current server time minus some time window to prevent race conditions, the format is defined by the server and and should be handled as an opaque string by the client
• For each exported update (read until all data has been consumed, ignoring the provided number of exported frames):
  • Local timestamp, 4 byte, 32 bit big-endian integer
  • Update length, 4 byte, 32 bit big-endian integer
  • Update data (see Layout of an update message)

The client can use either the maxtimestamp or the highest Local timestamp of all received updates and provide it as the 'get' parameter during the next update. It is recommended to use the maxtimestamp if available. The server should accept a maxtimestamp or a Local timestamp in the 'get' parameter, and should format the maxtimestamp so that it can distinguish it from a Local timestamp if their interpretations are different.

Anonet resource encoding

The different resource types are encoded as follows:

• IPv4 network
  • label (6 bytes):
    • Resource type, 1 byte, value 1
    • Network address, 4 bytes, network byte order
    • Prefix length, 1 byte, value 0-32
  • value: dictionary
    • 'owner': optional, string, free text, name of owner
    • 'descr': optional, string, free text, description
    • 'as': optional, numeric string, as number announcing this network
    • 'dns': optional, collection, see the Domain resource type, defines the reverse DNS zone for the IP network
• IPv6 network
  • label (18 bytes):
    • Resource type, 1 byte, value 2
    • Network address, 16 bytes
    • Prefix length, 1 byte, value 0-128
  • value: dictionary
    • 'owner': optional, string, free text, name of owner
    • 'descr': optional, string, free text, description
    • 'as': optional, numeric string, as number announcing this network
    • 'dns': optional, collection, see the Domain resource type, defines the reverse DNS zone for the IP network
• AS number
  • label (5 bytes):
    • Resource type, 1 byte, value 3
    • AS number, 4 bytes, 32 bit big-endian integer
  • value: dictionary
    • 'owner': optional, string, free text, name of owner
    • 'descr': optional, string, free text, description
    • 'speed': optional, numeric string, speed of this node in Mbit/sec
    • 'hasipv6': optional, NULL, existence indicates that the node has IPv6 connectivity
• Domain
  • label (n+1 bytes):
    • Resource type, 1 byte, value 4
    • Domain name, n bytes, all lower case, FQDN format, no terminating dot
  • value: dictionary
    • 'owner': optional, string, free text, name of owner
    • 'descr': optional, string, free text, description
    • 'dns': optional, collection containing dictionary items
      • 'label': optional, string, part before the domain name, no terminating dot (do not confuse with the resource label!)
      • 'type': required, string, DNS record type (A, AAAA, NS, MX, CNAME, PTR, TXT), bind format, upper case
      • 'data': required, string, DNS record data, bind format
      • 'ttl': optional, numeric string, number of seconds this record may be cached