MARC

MARC is the Magic Anonet Resource Claims System.

Design requirements

• Distributes resource claims to all nodes in a network
• Unattended automatic synchronisation should be possible, but people should not be able to alter each others' data
• Support for real-time and offline synchronisation
• Flexible but reliable representation of complex structures (nested arrays, associative arrays, binary strings)
• Should be simple to implement, audit and use
• Conflicts should be uncommon, and should not affect other resources or updates

Features

• Authenticated updates: only the first to claim a resource can update it afterwards (ownership based on first claim)
• Updates are cryptographically signed using the edwards25519sha512batch primitive provided by the NaCl library
• Can use any transport that can transfer an arbitrary block of binary data for synchronization (eg HTTP, e-mail, USB stick, UDPMSG3, git, rsync, newsgroups)
• Can use any storage mechanism that can store blocks of binary data (SQL, flat files)
• Requires to store only the last known version of each resource (but can store older versions if desired)
• A node is allowed not to store unwanted data, or to ignore unwanted updates

How?

• Resources are identified by a label, a 0-255 byte binary string
• IP addresses, domain names and AS numbers are encoded into a label, all sharing the same namespace
• Resource details are represented in a complex structure (nested arrays, associative arrays, binary strings), which are encoded (binary format) into a binary string, the resource value
• Each resource has a serial number, a higher serial number represents a more recent version, and should normally replace an older version in the database
• Each resource has a status, which tells whether the resource is taken or has been released (and can be taken by someone else)
• Each time a resource is changed, an message is generated, which contains the label, value, status and serial; this message is signed and the cryptographic public key is added to form the update message
• An update message always contains all resource details

Procedure for importing an update message

1. if a resource with the given label exists:
   1. if the current serial is higher than or equal to the update serial, ignore the update
   2. if the current status is not set to released and the key in the update is different from the current key, ignore the update
2. if the update message is not signed with the public key specified in the update message, ignore the update (this can be done earlier, but that would generally decrease performance)
3. optional: if the update is too big (per node configuration), ignore the update
4. optional: if the label is not acceptable (e.g. does not represent a valid IP network, AS number or domain name; node specific policy), ignore the update
5. optional: if the resource value is not acceptable (missing fields, unexpected formatting; node specific policy), ignore the update
6. optional: import the resource value into existing data structures (e.g. configuration file, resdb, SQL database)
7. store update message in the local database, identified/indexed by its label

Layout of an update message

• Version number (1 byte, value 1)
• Public key (32 bytes)
• Signed data (m+64 bytes):
  • First half of signature (32 bytes)
  • Resource data (m bytes)
    • Status (1 byte, value 0=deleted, 1=claimed, 2=transfer, 3=released)
    • Serial (4 byte, 32 bit big-endian integer)
    • Label length (1 byte)
    • Label (binary string)
    • Encoded value length (4 byte, 32 bit big-endian integer)
    • Encoded value (see Complex structure encoding)
    • Possibly other data (reserved for future use)
      • If status is 2: public key to transfer resource to, 32 bytes
  • Second half of signature (32 bytes)

Complex structure encoding

Complex structures are encoded into a binary string as follows:

• NULL (no data)
  • Data type, 1 byte, value 0
• String (binary or text string, may contain a textual representation of a number)
  • Data type, 1 byte, value 1
  • String length in bytes, 4 bytes, 32 bit big-endian integer
  • String data
• Collection (ordered array with no otherwise relevant indices)
  • Data type, 1 byte, value 2
  • Number of items in collection, 4 bytes, 32 bit big-endian integer
  • For each item in the collection:
    • The item is recursively encoded using this scheme
• Dictionary (array with relevant indices or associative array)
  • Data type, 1 byte, value 3
  • Number of items in dictionary, 4 bytes, 32 bit big-endian integer
  • For each item in the dictionary:
    • Key or index string length in bytes, 4 bytes, 32 bit big-endian integer
    • Key or index string
    • The item is recursively encoded using this scheme

Notes

• Resources in the released status may be removed from the database after some time
• For efficient client-server request-response synchronisation, the server should store the local update timestamp for each resource. The client can store the highest timestamp it has received from the server, and ask for resources newer than the previous timestamp.

Interactive HTTP synchronisation

This is currently the preferred method. The server runs a CGI script, for which the client makes either a GET, POST or PUT request.

The query string for the request contains at least one parameter, 'version'. This section describes protocol version 2. If the client wishes to receive/pull updates from the server, the query string should include a parameter 'get', with a value of either 0 (to retrieve all records) or the highest previous timestamp (see Notes).

If the client also wishes to push updates to the server, either the POST or PUT method should be used. For a POST request, the updates to push should be encoded according to the application/x-www-form-urlencoded MIME type, with field names of update[], and set the HTTP Content-Type header accordingly. For a PUT request, all updates are sent as one binary block of data, where each update is prefixed with its length in bytes encoded as a 32 bit big-endian integer.

The server should respond with the HTTP Content-Type header set to application/octet-stream. The response data looks as follows:

• Protocol version, 1 byte, value 2
• Encoded information length, 4 bytes, 32 bit big-endian integer
• Encoded information, encoded using the Complex structure encoding, may contain a dictionary with the following keys:
  • 'imported': number of imported updates
  • 'exported': number of exported updates
  • 'maxtimestamp': highest local timestamp in exported updates, or NULL if none were exported, can be an arbitrary string that is only meaningful to the server
• For each exported update (read until all data has been consumed, do not use the above number of exported frames):
  • Local timestamp, 4 byte, 32 bit big-endian integer
  • Update length, 4 byte, 32 bit big-endian integer
  • Update data (see Layout of an update message)

Anonet resource encoding

The different resource types are encoded as follows:

• IPv4 network
  • label (6 bytes):
    • Resource type, 1 byte, value 1
    • Network address, 4 bytes, ordered as in the textual representation
    • Prefix length, 1 byte, value 0-32
  • value: dictionary
    • 'owner': optional, string, free text, name of owner
    • 'descr': optional, string, free text, description
    • 'as': optional, numeric string, as number announcing this network
    • 'dns': optional, collection, see the Domain resource type
• IPv6 network
  • label (18 bytes):
    • Resource type, 1 byte, value 2
    • Network address, 16 bytes
    • Prefix length, 1 byte, value 0-128
  • value: dictionary
    • 'owner': optional, string, free text, name of owner
    • 'descr': optional, string, free text, description
    • 'as': optional, numeric string, as number announcing this network
    • 'dns': optional, collection, see the Domain resource type
• AS number
  • label (5 bytes):
    • Resource type, 1 byte, value 3
    • AS number, 4 bytes, 32 bit big-endian integer
  • value: dictionary
    • 'owner': optional, string, free text, name of owner
    • 'descr': optional, string, free text, description
    • 'speed': optional, numeric string, speed of this node in Mbit/sec
    • 'hasipv6': optional, NULL, existence indicates that the node has IPv6 connectivity
• Domain
  • label (n+1 bytes):
    • Resource type, 1 byte, value 4
    • Domain name, n bytes, all lower case, FQDN format, no terminating dot
  • value: dictionary
    • 'owner': optional, string, free text, name of owner
    • 'descr': optional, string, free text, description
    • 'dns': optional, collection containing dictionary items
      • 'label': optional, string, part before the domain name, no terminating dot (do not confuse with the resource label!)
      • 'type': required, string, DNS record type (A, AAAA, NS, MX), bind format
      • 'data': required, string, DNS record data, bind format
      • 'ttl': optional, numeric string, number of seconds this record may be cached