This document specifies version 1 of the Auspex protocol, a mail protocol for Urbit ships. Every Auspex message is signed by the networking key of the ship that wrote it. A reply or a forward carries the signed messages on the path from the thread root to the message being answered, so that any recipient can verify every message on that path independently, including messages written by ships it has never communicated with. This document defines the message format, the message identifier and signing digest, the verification procedure and its three verdicts, the rules for merging and bounding received chains, content-addressed attachments, the delivery transport, and a capability discovery mechanism. It also defines the conformance test vectors that a second implementation is expected to reproduce.
This document is a protocol proposal published by its author. It is not an Internet-Draft or an RFC. It is not a product of the IETF or of any other standards body, and no such body has reviewed or endorsed it. The document follows the conventions of the RFC series because those conventions are familiar to implementers.
The protocol described here is implemented and deployed. The unsigned structure defined in Section 4.1 is frozen. Other parts of this document may be revised. Comments are solicited and should be filed as issues at https://github.com/nisfeb/auspex.
Auspex is mail between Urbit ships. It differs from chat protocols on the same network in one respect that shapes the whole design. An Auspex message is evidence that can be handed to a third party. A message is signed by its author's ship, and the signature is carried with the message wherever the message goes. A recipient of a forwarded conversation verifies each message in it against the key of the ship that wrote it. The recipient does not need to trust the forwarder.
Three properties follow from this goal and are developed in the rest of the document.
This document is written for an implementer who has an Urbit ship and access to its key store, and who does not have the reference implementation's runtime. The reference implementation is a Grubbery nexus [GRUBBERY]. Nothing about Grubbery is normative except the single transport defined in Section 8.
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here.
@p. Galaxies, stars, and planets are ships whose public keys are registered in Azimuth.
.^ expression.
jammugshax, shas, shaf, shamshax is SHA-256. (shas sal ruz) is a salted SHA-256, computed as SHA-256 over the bitwise exclusive-or of the salt sal and (shax ruz). (shaf sal ruz) folds the 256-bit result of (shas sal ruz) to 128 bits by exclusive-or of its two halves. (sham yux) is a 128-bit hash of an arbitrary noun. For a cell it is (shaf %sham (jam yux)). For an atom it is (shaf %mash yux).
unsigned structure together with a signature over its digest. See Section 4.
prev fields.
%verified, %unverified, or %forged.
Data structures and reference procedures are given in Hoon. A name with a leading +, such as +merge, names a procedure ("arm") of the reference library [AUSPEX-LIB]. A name with a leading $, such as $unsigned, names a type. A name with a leading %, such as %forged, is a constant term.
The atom types used in this document are as follows.
| Type | Meaning |
|---|---|
@p |
ship address |
@ud |
unsigned decimal integer |
@t |
UTF-8 text, stored least significant byte first |
@tas |
a term, which is lowercase ASCII text with hyphens |
@da |
absolute date, 128 bits |
@dr |
time interval |
@ux, @uv, @uw |
unsigned integer printed in base 16, base 32, or base 64 |
The symbol ~ is null and is the atom 0. A (unit x) is either ~ or the cell [~ x]. A (list x) is either ~ or the cell [item rest]. A (set x) is the ordered tree described in Section 4.3.
Hoon prints decimal numbers with a dot between each group of three digits, so that 100.000 in a Hoon excerpt is one hundred thousand. Prose and tables in this document use plain decimal numbers. Hoon excerpts are reproduced exactly.
The reference library [AUSPEX-LIB] is the authoritative definition of the message format and of every procedure quoted in this document. Where this document and the library disagree, the library is correct and the disagreement is a defect in this document.
An author composes a message by building a nine-field structure called unsigned. The message identifier is a hash of that structure. The author signs a domain-separated digest of the same hash with its networking key and attaches the signature. The result is a message.
A message that answers or forwards another message names its parent in the prev field. A thread is therefore a tree. When a ship sends a message, it sends a chain. The chain contains the new message and every stored copy of each message on the path from the thread root to the new message's parent. It contains nothing from sibling branches.
A sender first reads the recipient's published capabilities over remote scry. If the two ships share a protocol version and the chain fits the recipient's published limits, the sender pokes the chain to the recipient.
A receiver applies size limits to the incoming chain and refuses the chain whole if any limit is exceeded. It then retrieves one public key for each distinct signer named in the chain, verifies every message, and records one verdict per copy. It determines which thread the chain belongs to from the content of the chain. It merges the chain into its stored thread, discards excess copies in a defined order, and stores the result. Forged messages are stored and labelled. They are never dropped.
Attachment bytes do not travel in a chain. A message carries a descriptor with a content hash for each attachment. A ship that wants the bytes retrieves them by hash over remote scry, from any ship that holds them, and checks them against the hash.
Read state, labels, filters, drafts, and the verdicts themselves are local to each ship. They are never signed and never transmitted.
The protocol version is the name of the wire mark. The mark %auspex-chain is version 1.
The payload contains no version field, and an implementation MUST NOT add one. A version field would be covered by the message identifier and by the signature. Changing its value would invalidate every message signed before the change.
A breaking change is published as a new mark name. An implementation that changes the format MUST publish the new format under a different mark, for example %auspex-chain-2. It MUST leave the meaning of %auspex-chain exactly as defined in this document.
A change is breaking if it alters either of the following.
$unsigned, meaning the number, order, or type of its nine fields.$unsigned.Both are breaking for the same reason. The message identifier is (sham unsigned), and the signature covers (shaf %auspex (sham unsigned)). The hash runs over the noun itself, so the order of the cells is significant. A message signed against a different shape has a different identifier and a signature that verifies against nothing.
Stored messages cannot be migrated to a new shape. A rewritten message carries a signature that no longer matches its contents, and every peer would read it as %forged. Presenting genuine mail as forgery is a worse outcome than refusing it.
The following changes are not breaking, because they affect nothing that a signature covers.
A receiver MUST NOT accept a poke that carries a mark it does not implement, and it MUST NOT coerce the payload of such a poke into %auspex-chain. A noun that arrives under an unknown mark is a message for a protocol that the receiver does not speak. It is not a malformed chain. An implementation that guesses at such a payload can turn a valid message of a later version into an apparent forgery under version 1.
A receiver SHOULD nack such a poke where its transport allows. On the reference transport a poke of an unknown mark is never acknowledged at all. See Section 8.4. The discovery mechanism in Section 9 exists so that a sender does not issue such a poke.
unsigned StructureThe unsigned structure contains every field that a signature covers.
+$ unsigned
$: from=ship
life=@ud
to=(set ship)
subj=@t
body=@t
body-mime=@t
sent=@da
prev=(unit msg-id)
attachments=(list attachment)
==As a noun, unsigned is a right-nested nine-tuple.
[from [life [to [subj [body [body-mime [sent [prev attachments]]]]]]]]The structure has no header, no version, and no padding. The message identifier is computed over exactly this noun. The order of the nine fields is therefore part of the protocol. An implementation that builds the cell in any other order produces a different identifier and a signature that cannot be verified.
| # | Field | Type | Noun shape | Meaning |
|---|---|---|---|---|
| 1 | from |
@p |
atom | The ship that signed the message. |
| 2 | life |
@ud |
atom | The key revision under which the signature was made. |
| 3 | to |
(set ship) |
ordered tree, or ~ when empty |
The visible recipients. |
| 4 | subj |
@t |
atom | The subject. |
| 5 | body |
@t |
atom | The message body. |
| 6 | body-mime |
@t |
atom | The media type of body. The empty value means text/plain. |
| 7 | sent |
@da |
atom | The time of sending, by the author's clock. |
| 8 | prev |
(unit msg-id) |
~, or [~ id] |
The identifier of the parent message, or ~ for a thread root. |
| 9 | attachments |
(list attachment) |
~, or [item rest] |
Attachment descriptors. Never attachment bytes. |
This structure is frozen. No field may be added, removed, reordered, or retyped within version 1.
from[from life]. It MUST NOT retrieve the key of any other ship. A message whose from names a ship that did not sign it is a forgery, and the mismatch is what verification detects.
life[ship life] and never by ship alone. The value is under the control of an attacker. A modified life names a key that the verifier does not hold, and the verdict in that case is %unverified and never %forged. See Section 5.4.
tosubjbodybody-mimebody. The empty value means text/plain. The field is signed because the rendering instruction is part of the message. An intermediary must not be able to change whether a body is presented as plain text or as some other type. The value MUST NOT exceed 128 bytes and MUST NOT contain control bytes. See Section 4.4 for the rules that govern its use.
sentsent. A receiver MUST NOT reject a message because its sent value lies in the past or in the future.
prev~ marks a thread root. The field names a message and not a thread. That is sufficient, because a message identifier is a hash of the full contents of a message. It identifies exactly one message and therefore exactly one thread.
attachmentsunsigned, so the signature and the message identifier both cover it. Replacing an attached file invalidates the signature. A message MUST NOT carry more than 16 descriptors.
The byte length of a text field is measured as (met 3 field).
to FieldThis section is normative.
The to field is a (set ship), and the message identifier hashes the tree that represents the set. The identifier of a message with more than one recipient cannot be reproduced from a list of ships alone. It depends on the shape of the tree. An implementation that builds the tree differently computes a different identifier and produces a signature that no other implementation can verify.
A (set ship) has the following type in [HOON].
+$ nlr $@(~ [n=ship l=nlr r=nlr])The empty set is the atom 0. A node is the right-nested triple [n [l r]], where n is the item, l is the left subtree, and r is the right subtree.
The tree is a treap. It is ordered in two ways at once.
mor. The expression (mor a b) compares (mug (mug a)) with (mug (mug b)) and compares the atoms directly if the two values are equal. The item at every node precedes the items of both of its children under mor.gor. The expression (gor a b) compares (mug a) with (mug b) and falls back to dor, the direct ordering of nouns, if the two values are equal. Every item in l precedes n under gor, and n precedes every item in r.An implementation MUST build the tree exactly as the +put:in procedure of [HOON] does. That procedure inserts each ship in turn and rotates the tree to restore the heap property. The procedures +sy and +gas:in use the same insertion. The result is canonical. A set built from the same ships in any order of insertion is the same noun. Two ships whose users entered the same recipients in different orders therefore compute the same message identifier.
An implementation MUST NOT substitute a sorted list, a map, or a tree with any other balancing rule.
As a worked example, consider the set that contains the ships ~zod, ~nec, and ~bud.
| Ship | Address as an atom | mug of the address |
|---|---|---|
~zod |
0 | 2046756072 |
~nec |
1 | 1901865568 |
~bud |
2 | 1904972904 |
The resulting noun is as follows.
[0 [2 [1 0 0] 0] 0]In this noun ~zod is the root. ~bud is the left child of ~zod. ~nec is the left child of ~bud. Every right subtree is empty. The jam of this noun is 0x1.54e2.c859, which prints as 0w5.kUIxp in base 64. The jam of the empty set is 0w2.
The test case named three-recipients in [AUSPEX-VECTORS] fixes this encoding. Its to field holds these three ships. Every other signed case in the vectors names a single recipient and therefore tests nothing about the shape of the tree.
Three kinds of field are supplied by a possible attacker and arrive already signed. They are body-mime, the name of each attachment, and the mime of each attachment.
A signature shows that the author chose a value. It shows nothing about whether the value is safe to use. A recipient cannot repair a signed field without destroying the signature that makes the message evidence. The only point at which such a value can be refused is the receiving boundary, before the message is stored.
A receiver MUST refuse a chain in which any body-mime, attachment name, or attachment mime exceeds its length limit or contains a byte with a value of 0x1f or lower, or the byte 0x7f. The reference predicate is +text-ok.
++ text-ok
|= [t=@t m=@ud]
^- ?
?& (lte (met 3 t) m)
%+ levy (trip t)
|=(c=@tD &((gth c 0x1f) !=(c 0x7f)))
==The mime value is a candidate for an HTTP Content-Type header, where a carriage return or a line feed permits header injection. The name value is a candidate for a file name.
A renderer MUST NOT act on body-mime directly. It MUST compare the value with a fixed list of permitted types and MUST treat any other value as text/plain. It MUST NOT copy the value into a header of any kind. A renderer reports body-mime to the user. It does not obey it. The reference client renders every body as plain text and informs the user when a message requested another type.
An implementation MUST sanitize an attachment name before using it in a Content-Disposition header or in a file system path. This requirement is additional to the boundary check above.
The sent field is hostile in a different way. It is always well formed, and it may still be false. It is safe to display and unsafe as the basis of any decision.
++ id
|= u=unsigned
^- msg-id
(sham u)The message identifier has type @uv. It is (sham unsigned), the hash of the whole nine-tuple, with no salt and no wrapping. The identifier covers exactly what the signature covers. Two ships that hold the same message compute the same identifier without coordination.
An implementation MUST NOT compute a message identifier in any other way. An implementation MUST NOT include the signature in the computation. Two copies of one message that differ only in signature share an identifier. Section 6.4 depends on this property.
++ digest
|= u=unsigned
^- @
(shaf %auspex (sham u))The digest is the value that is signed. It is (shaf %auspex (sham unsigned)). The term %auspex is a domain separation tag, passed as the salt of shaf.
An implementation MUST compute the digest with shaf and the salt %auspex. An implementation MUST NOT sign an unsalted sham, a message body, or any other preimage. The reason is given in Section 13.1.
An earlier deployment of this software was named urmail and used the salt %urmail. A message salted with %urmail is not a version 1 Auspex message.
+$ msg [=unsigned sig=@ux]
+$ chain (list msg)A message is the cell [unsigned sig]. The signature is outside unsigned. It is covered neither by the message identifier nor by itself. This is what allows two signed copies of one message to exist and to be distinguished from each other.
To sign a message, an author performs the following steps.
.^(@ud %j /=life=/(scot %p our)).$unsigned, with from set to the author's own ship and life set to the value read in step 1.$unsigned as defined in Section 4.6..^(ring %j /=vein=/(scot %ud life)). Jael answers this scry only when the requester is the ship itself.The reference signing procedure is as follows.
++ sign-with
|= [=ring msg=@]
^- @ux
(sigh:as:(nol:nu:cric:crypto ring) msg)The signature MUST be made with the key of the ship named in from, at the life named in life. A signature made with any other key is a forgery, and a verifier will detect it.
To verify one message, a verifier performs the following steps.
unsigned.[from life] as described in Section 5.3.%unverified and stop.%verified if the check succeeds and %forged if it fails.The reference verification procedure is as follows.
++ verify-with
|= [=pass sig=@ux msg=@]
^- ?
(safe:as:(com:nu:cric:crypto pass) sig msg)Verification applies to a message and never to a chain. A single chain routinely yields several verdicts. A forwarded chain whose forwarder is known to the verifier and whose original author is not yields %verified for one message and %unverified for another.
An implementation MUST NOT compute a single verdict for a chain. It MUST NOT allow the verdict of one message to influence the verdict of another. It MUST NOT present a verdict for a thread as a whole. A thread that contains one unverified message is not an unverified thread.
The public key for a pair [who life] is read from Jael with the %puby scry.
.^((unit [crypto-suite=@ud =pass]) %j /=puby=/(scot %p who)/(scot %ud life))The %puby scry returns ~ for a ship that is absent from the local Azimuth snapshot. It does not block. An implementation SHOULD use %puby. An implementation SHOULD NOT use the %deed scry, which blocks and therefore stalls the process that issues it.
The current life of a ship is read with the %life scry.
.^(@ud %j /=life=/(scot %p who))The following transcript was recorded on a live planet on 8 September 2026.
> .^(@ud %j /=life=/~zod)
6
> .^((unit [@ud @]) %j /=puby=/~zod/6)
[~ [1 2.224.943.983…]]Crypto suite 1 is the suite named %b in the kernel. Fake ships derive their keys under the same suite. One verification procedure therefore serves both live ships and fake ships.
Jael answers a scry only when the date in the request is exactly the current time of the ship. A request at any other date blocks. An implementation MUST issue Jael scries with the current time. It MUST NOT issue one from a context that has no current time. It MUST NOT store a Jael answer under a date and treat it as current later.
An implementation SHOULD perform one key retrieval for each distinct pair [ship life] that appears in a chain, and not one for each message. Section 6.7 bounds the number of such pairs.
+$ verdict ?(%verified %unverified %forged)| Verdict | Condition |
|---|---|
%verified |
A key was found for [from life], and the signature is valid under that key. |
%forged |
A key was found for [from life], and the signature is not valid under that key. |
%unverified |
No key was found for [from life]. |
The reference procedure is as follows.
=/ k (~(get by keys) [from.unsigned.m life.unsigned.m])
?~ k %unverified
?~ u.k %unverified
?: (verify-with u.u.k sig.m (digest unsigned.m))
%verified
%forgedA missing key MUST NOT produce the verdict %forged. The absence of a key is not a finding about a signature. A verifier cannot distinguish an honest ship whose key it never retrieved from a modified life field that names a key it does not hold. A verdict of forgery against a ship whose key was never available is a false accusation, and the three-verdict scheme exists to prevent it.
A receiver MUST store a %forged message and MUST present it as forged. A forged message is evidence. A receiver MUST NOT drop, hide, or filter a %forged message. A filter that could suppress a message would allow an attacker who learns a user's filter rules to conceal the evidence of a forgery. A receiver SHOULD exclude %forged messages from unread counts and from the default inbox ordering. A receiver MUST return %forged messages from search, labelled as forged.
A verdict is local state. It records how one ship read one signature against its own Azimuth snapshot at one moment. A verdict is never transmitted. See Section 10.
Under version 1, a third party cannot verify a message from a moon or from a comet. This is a property of the key infrastructure and not of the message format. Jael releases a moon's key only to the moon's parent. It resolves a comet's key only when the comet asks about itself. Mail from a moon therefore verifies on its parent and on no other ship. On every other ship, this entire class of sender receives the verdict %unverified.
This is the reason that %unverified ranks above %forged when copies are discarded (Section 6.5). Without that ranking, junk copies could displace the only genuine copy of a moon's message.
Both limitations can be removed without a change to the format. A comet's address is derived from its key, so a comet's signature can be checked with no retrieval at all. A moon's signature can be checked if an attestation from its parent travels with the chain. The signatures are already present in every chain. Both changes are additive.
On a fake network, every key pair is derived from the ship address.
++ fake-core |=(who=ship (pit:nu:cric:crypto 512 who %b ~))
++ fake-ring |=(who=ship `ring`sec:ex:(fake-core who))
++ fake-pass |=(who=ship `pass`pub:ex:(fake-core who))A verifier determines whether its own ship is fake by asking Jael.
.^(? %j /=fake=)The %puby scry has no special handling for fake ships. On a fake ship it returns ~ for nearly every peer, because a fake ship holds no Azimuth snapshot. Without further handling, every message in a development environment would receive the verdict %unverified.
A verifier MUST therefore check %fake first. When the answer is true, the verifier MUST derive the peer's public key as pub:ex:(pit:nu:cric:crypto 512 who %b ~) and MUST NOT scry %puby. The %deed scry handles fake ships in the same way.
Two consequences follow.
life field has no effect on key retrieval, because the derivation takes only the ship address. A message that claims life 99 from a fake ship verifies on another fake ship. A test that needs the verdict %unverified on a fake ship has to name a signer that the verifier deliberately omits from its key table. See Section 11.A chain is a list of messages and nothing more. It has no header, no thread identifier, and no sender field. It carries no ordering guarantee. The order of a chain is chosen by the ship that sent it, and that ship may be hostile.
A chain represents one path from a thread root to one message. It does not represent a whole thread. Threads branch whenever two participants answer the same message. A chain holds the conversation that leads to one message, which is what a recipient needs in order to verify that message in context.
A thread identifier is the message identifier of the thread's root message. Two ships that hold the same conversation agree on its identifier without coordination, because the root message is identical on both.
A receiver determines the thread of an incoming chain with the procedure +thread-key, under the following rules.
The identity of an established thread is immutable. If the pair [id sig] of any message in the incoming chain matches a copy that the receiver already holds in some thread, the chain belongs to that thread, whatever else the chain contains.
Otherwise, the chain is a first contact, and the thread is identified by the unique root in the chain. The roots are the messages with prev equal to ~. They are counted by distinct identifier and not by message.
=/ roots (skim c |=(m=msg ?=(~ prev.unsigned.m)))
=/ root-ids
(~(gas in *(set msg-id)) (turn roots |=(m=msg (id unsigned.m))))
?. =(1 ~(wyt in root-ids)) ~|(%auspex-no-unique-root !!)
(snag 0 ~(tap in root-ids))Counting by identifier is required. A relay may forward the genuine root together with a copy that carries a modified signature. Both copies have prev equal to ~, because prev is part of the signed content that they share. Counting by message would reject that chain, and the chain is otherwise legitimate.
A receiver MUST refuse a chain that contains no root, and a chain that contains more than one distinct root identifier. The reference procedure raises an error in both cases. A receiver MUST contain that error and treat it as a refusal. See Section 8.4.
A receiver MUST NOT derive thread identity from the order of the incoming list or from any sent value. An attacker controls both.
The procedure +root of the reference library returns the first message of a chain as supplied. It is intended for a chain that the local ship built. An implementation MUST NOT apply it to an incoming chain.
A sender MUST transmit the messages on the path from the thread root to the message being answered or forwarded, together with the new message. It MUST NOT transmit anything else.
++ path-chain
|= [c=chain i=msg-id]
^- chain
=/ keep=(set msg-id) (~(gas in *(set msg-id)) (ancestors (prev-map c) i))
(merge ~ (skim c |=(m=msg (~(has in keep) (id unsigned.m)))))The following rules apply.
prev upward from the message. The walk ends at a root, at a prev value that does not resolve within the stored thread, or at a bound equal to the number of distinct identifiers. See Section 6.8.The result is a valid chain. Every prev in it resolves within it, and it contains the unique root. The recipient therefore assigns it to the same thread identifier, and every message in it verifies independently.
One case requires an addition to the path.
++ with-root
|= [c=chain p=chain]
^- chain
?: (lien p |=(m=msg ?=(~ prev.unsigned.m))) p
(merge p (skim c |=(m=msg ?=(~ prev.unsigned.m))))A well-formed path already ends at a root, and +with-root leaves it unchanged. The procedure acts only on the path of an orphan (Section 6.8). Such a path contains no root, and a recipient would refuse it under rule 3 of Section 6.2. The send would appear successful to the sender and would be discarded by the recipient. The procedure adds the thread root in that case. Every participant already holds the root, so adding it discloses nothing.
A sender MUST apply to its outgoing chain the same limits that a receiver applies on arrival (Section 6.7). Every send carries the path it answers. One oversized message would therefore cause every later message on the same path to be refused by every recipient, with no indication to anyone. The moment of composition is the only point at which a person can still correct the problem.
A receiver merges an incoming chain into the chain it has stored for the same thread.
++ merge
|= [old=chain new=chain]
^- chain
=/ key |=(m=msg [(id unsigned.m) sig.m])The procedure +merge removes duplicates by the pair [id sig]. It MUST NOT remove duplicates by identifier alone. A message is a duplicate of another only when both the contents and the signature match.
The reason is an attack that is described in Section 13.3. In outline, two messages can share an identifier and carry different signatures, one valid and one not. If duplicates were detected by identifier alone, the copy that arrived first would displace the other. With the pair as the key, both copies are retained, each receives its own verdict, and the reader decides what to display.
The same pair identifies a copy wherever a copy is named.
[id sig].[id sig].[id sig].The table that maps each message to its parent is the one deliberate exception. It is indexed by identifier, because prev is inside unsigned and every copy of one identifier therefore carries the same prev. Two copies that differ in signature are a single node of the thread tree.
The procedure also removes duplicates within the incoming chain itself. A peer may supply a chain that repeats a message, and on first contact there is no stored chain to compare against.
The merged chain is sorted by sent, then by identifier, then by signature. This is a total order. Two ships that merge the same messages produce the same list.
A receiver holds at most 4 copies of any one message identifier. The procedure +prune enforces this bound.
++ prune
|= [c=chain vs=(map [msg-id @ux] verdict) max-copies=@ud]
^- chainThe following requirements apply.
%verified copies are retained first, then %unverified copies, then %forged copies. A %verified copy MUST NOT be discarded while a copy of lower rank remains.%unverified above %forged is required. Under version 1, every message from a moon or a comet is %unverified (Section 5.5). Without the ranking, 4 junk copies could displace the single genuine copy of such a message.%unverified.The general rule is as follows, and an implementation MUST follow it. A limit on incoming data is enforced by refusal. A limit on merged state is enforced by discarding. Refusing a delivery because of the state of the store penalizes a legitimate sender for excess that may consist entirely of an attacker's data.
The procedure +freeze combines the verdicts from a new delivery with the verdicts already stored.
?: ?=(?(%verified %forged) (~(gut by acc) -.i.new %unverified))
$(new t.new)The verdicts %verified and %forged are definitive for a given pair [id sig]. The digest and the key are both fixed by the pair, so the two verdicts can never contradict each other. A stored %verified or %forged verdict MUST NOT be overwritten.
The verdict %unverified is not a finding about the signature. It records only that the key was absent at one moment. A stored %unverified verdict MUST remain replaceable by the verdict of a later delivery that arrives after the key has become available.
A receiver MUST enforce each of the following limits on an incoming chain. A receiver MUST refuse a chain that exceeds a limit. It MUST NOT truncate the chain or any field in it. A chain that violates a limit is not partly trustworthy, and an altered signed field is a forgery.
| Name | Value | Bounds |
|---|---|---|
max-chain |
1000 | messages in one chain, and distinct identifiers in one thread |
max-body |
100000 | bytes in one body |
max-subj |
1000 | bytes in one subject |
max-to |
100 | recipients of one message |
max-mime |
128 | bytes in body-mime, and in an attachment mime |
max-attach |
16 | attachments on one message |
max-blob |
262144 | claimed size in bytes of one attachment |
max-name |
256 | bytes in an attachment name |
max-depth |
64 | ancestors between a root and a leaf |
max-signers |
128 | distinct pairs [ship life] in one chain |
max-copies |
4 | signed copies of one identifier. Enforced by discarding (Section 6.5). |
max-threads |
10000 | distinct threads held by one ship. Applies to new threads only (Section 6.9). |
The values of two limits were chosen for reasons that an implementer would not otherwise see.
The value of max-depth reflects a cost on the read path. The reference implementation stores a message under its ancestry. The storage key of a message has one segment of about 34 bytes for each ancestor, and the keys are rebuilt whenever the mail tree is read. The cost is quadratic in depth. With max-chain as the only bound, one hostile linear chain could hold a thread at depth 1000 permanently. In measurement, 200 messages at depth 200 cost about 1.8 times as much as 200 messages at depth 2. The limit is checked on the incoming chain and again on the merged chain, because two chains that are each within the limit can combine to exceed it. The cost of the limit is that a thread deeper than 64 messages accepts no further replies.
The limit max-signers is the only limit that bounds work performed outside the receiving application. Every other limit bounds bytes or nodes. This limit bounds key retrievals, because a key costs one scry for each distinct pair [ship life]. Without the limit, one junk chain could name a thousand signers and cause a thousand sequential retrievals and a thousand signature verifications [RFC8032]. A forged signature costs exactly as much to check as a valid one. The value was chosen with reference to max-to. One message may name 100 recipients, so a thread in which every recipient replies once has 100 signers and is legitimate. The value 128 accommodates that case and leaves room for key rotations, which cause one ship to appear as two signers.
A receiver's limits on its own attachment store are not part of the protocol. The reference implementation holds at most 1000 blobs and at most 33554432 bytes of blobs. Attachment bytes enter a ship only through a local action and never through a delivered chain.
An orphan is a message whose prev names an identifier that is not present in the chain or in the stored thread. The ancestry walk stops at an orphan. A receiver MUST store an orphan and place it as a root of its own subtree. It MUST NOT drop it. Only hostile input produces an orphan, because a forwarded path is complete by construction and pruning never discards the last copy of an identifier. Refusing to store a message is nevertheless a worse outcome than filing it at the wrong depth. When the parent of an orphan later arrives, the orphan takes its place beneath the parent.
A cycle of prev values would require a hash preimage loop and does not occur in practice. The ancestry walk operates on input supplied by an attacker all the same. An implementation MUST bound the walk so that it terminates. The reference implementation bounds it by the number of distinct identifiers in the chain.
The placement of orphans does not relax rule 3 of Section 6.2. A chain that contains no root at all is refused, because there is nothing from which to derive a thread identity.
A receiver MUST process an incoming chain in the order given in this section. Limits are checked first, then signatures are verified, then the thread is identified, then the chain is merged, pruned, and stored. Nothing is written until every signature in the incoming chain has a verdict.
Decode. Interpret the payload as a $chain. A payload that is not a well-formed chain MUST be refused without an error escaping the handler. See Section 8.4. An empty chain requires no action.
Check the limits on the incoming chain. Check max-chain, max-body, max-subj, max-to, max-attach together with the attachment descriptor limits, max-mime, max-depth, and max-signers, in that order. A chain that exceeds any limit is refused whole. The max-signers check comes last among the limits and before any key retrieval, because key retrieval is the cost that it bounds.
Build the key table. Perform one retrieval for each distinct pair [ship life] in the chain. Check whether the local ship is fake once, and retrieve keys as described in Sections 5.3 and 5.6.
Verify every message. Produce one verdict for each pair [id sig].
Identify the thread as described in Section 6.2. A chain without a unique root is refused at this step.
Merge the incoming chain with the stored chain of that thread, as described in Section 6.4.
Check the limits on the merged state.
max-chain, the delivery is refused. Discarding a distinct identifier that is not a root would orphan later messages. This refusal is a known weakness. See Section 13.9.max-depth, the delivery is refused.max-threads threads, the delivery is refused. A delivery to an existing thread MUST NOT be refused on this ground. A reply is never refused because unrelated threads filled the store.Combine the verdicts, then prune. Verdicts are combined first (Section 6.6), because pruning needs a verdict for every message in the merged chain. That includes messages stored by an earlier delivery and absent from this one. The merged chain is then pruned (Section 6.5).
Store the pruned chain. Each copy is stored under its pair [id sig] together with its verdict. A delivery that results in no write MUST be treated as no change.
Apply local filing. New mail returns an archived thread to the inbox. The receiver's own filters may then add labels and may archive the thread again. Nothing in this step can suppress a message, because the chain and its verdicts are already stored.
Signal the change. An implementation that notifies its clients of new mail MUST do so only if stored state changed. A notification for a refused delivery, or for a repeated delivery that wrote nothing, would allow any remote ship to impose the cost of a full refresh on every connected client.
In the reference implementation, every refusal in this procedure is a normal return that records a reason in a local log. The poke is acknowledged and the chain is discarded. Only a payload that cannot be decoded, or a failure of the transport, produces a nack. A sender therefore cannot distinguish a refused chain from an accepted one. An implementation MAY nack a refusal instead. An implementation MUST NOT terminate abnormally on a refusal. See Section 8.4.
+$ attachment
$: name=@t :: original filename
size=@ud :: bytes
mime=@t :: content type
hash=@uv :: (sham octs) over the contents
==An attachment descriptor is the right-nested four-tuple [name [size [mime hash]]]. Descriptors are held in the attachments field of unsigned. They are therefore covered by the signature and by the message identifier. A descriptor adds about 100 bytes to a message whatever the size of the file. A chain is transmitted whole on every send, and for that reason the bytes of an attachment are never part of a chain.
++ blob-hash
|= =octs
^- @uv
(sham octs)The hash field is (sham octs), where octs is the cell [length bytes]. The hash is computed over that cell and not over the byte atom alone. An atom does not record leading zero bytes, so two files that differ only in leading zeros are the same atom. The octs cell carries the length, and the two files therefore have different addresses. The construction also binds the size to the address. A false statement of the size is a false statement of the address.
A blob is acceptable if and only if it hashes to the address under which it was requested.
++ blob-ok
|= [=octs h=@uv]
^- ?
=(h (blob-hash octs))The limits max-blob, max-attach, max-name, and max-mime of Section 6.7 apply to descriptors. The reference predicate for one descriptor is as follows.
++ attach-ok
|= a=attachment
^- ?
?& (lte size.a max-blob)
(text-ok name.a max-name)
(text-ok mime.a max-mime)
==The value of max-blob is 262144 bytes. A blob is served over remote scry, which divides a response into network packets and has no means of resuming a partial transfer. The value keeps a single retrieval small.
A receiver MUST NOT treat the size field as evidence of the size of the bytes. The field is checked against max-blob on arrival because refusing an oversized claim at the boundary costs less than discovering it during retrieval. The hash is the authority. A receiver measures and hashes the bytes on arrival, whatever size stated.
Attachment bytes are never pushed to a recipient. A receiver stores a chain as soon as it arrives. A ship that wants an attachment it does not hold retrieves the bytes on demand, by hash.
Retrieval MUST be the result of an explicit local action. Delivery of a chain MUST NOT cause retrieval. A chain from an unknown ship that names many attachments must not cause the receiving ship to issue many requests.
Retrieval uses a keen. Remote scry is the only read channel on the network that requires no permission. Other read mechanisms require a prior grant, and between ships with no grant they stall and do not fail. The asymmetry suits this protocol. Any ship that holds the bytes can serve them, and the hash proves them.
A ship that holds a blob publishes it in the namespace of its hosting agent at the following path.
++ blob-spur
|= h=@uv
^- path
/auspex/blob/[(scot %uv h)]A retrieving ship requests the following path, which MUST correspond exactly to the published path.
++ blob-keen-path
|= [agent=@ta h=@uv case=@ud]
^- path
%+ weld `path`[%g %x (scot %ud case) agent %$ %'1' ~]
(blob-spur h)The request path has the form /g/x/<case>/<agent>//1/auspex/blob/<hash>. Its segments are as follows.
g, x<case><agent>%grubbery, the hosting agent, and not the nexus. An implementation hosted by another agent names that agent.
1The path has no segment that identifies a revision of the content. The bytes of a blob are fixed by its name, so no mechanism for discovering revisions is needed. Such mechanisms require permission.
The response is a pair of a mark and a noun. The mark MUST be %auspex-blob and the noun MUST be an $octs. A retrieving ship MUST discard any other response.
Case 1 serves every blob that has never been withdrawn. Withdrawing a published path consumes its case number permanently. A blob that is restricted (Section 7.5) and later published again is bound at case 2. The reference implementation requests cases 1 through 3 in turn. Each miss costs one timeout, and only a blob that has been restricted incurs that cost.
A publisher MUST NOT publish to a path without first establishing that the path is unbound. Publishing to a bound path binds the next case number. A publisher that republished on every user request would move the binding beyond the range that peers probe. The attachment would then be unavailable to every peer, permanently, and no error would be reported to anyone.
A retrieving ship MUST validate a response before accepting it. The reference implementation applies the following checks in order, and discards the response at the first failure.
?~ res.b → discard: the fetch missed
?. (lte p.u.res.b max-blob:uc) → discard: 'blob too large'
?. (gte p.u.res.b (met 3 q.u.res.b)) → discard: 'blob malformed'
?. (blob-ok:uc u.res.b hash.b) → discard: 'blob hash mismatch'A blob whose contents do not hash to the requested address MUST be discarded. The failure MUST NOT be attributed to the author of the message. The ship named in a retrieval request is only an indication of where the bytes may be found, and it confers nothing. The hash proves the bytes. The blob store is a cache. The loss of a blob is the loss of a file, and never the loss of a message or a signature.
A ship that accepts a blob publishes it in turn, because it is now one of the ships that hold the bytes.
A blob is public by default. A chain can be forwarded to anyone, and an attachment readable only by the original recipients would make every forward carry a file that its recipient cannot read.
A ship may restrict a blob. Restriction withdraws the ship's own copy from its published namespace. Every ship that retrieves a blob republishes it, so the first successful retrieval creates a second source that the first ship cannot withdraw.
Restriction therefore provides exactly one guarantee. Bytes that a ship has not yet served can no longer be obtained from that ship. An implementation MUST present restriction to the user as unpublishing. It MUST NOT present restriction as revocation or as access control. The property that makes a chain portable also makes a blob impossible to recall, and the design accepts this consequence of using the hash as the authority.
The visibility of a blob is local state and is never transmitted. See Section 10.
The HTTP interface through which the reference client uploads and downloads blobs is outside the scope of this document.
A sender delivers a chain by poking the recipient's Auspex nexus with the mark %auspex-chain and the chain as the payload noun. The poke is carried by the command interface of the Grubbery agent, whose mark is %grub-cmd. A conformant peer runs Grubbery with the Auspex nexus installed. Version 1 defines no other endpoint.
Within the nexus, the chain is addressed to the grub named main.sig at the nexus root, which is the writer process of the nexus. The Grubbery message type for the delivery is the following pair.
[/ %auspex-chain]The type is a top-level mark whose name begins with auspex-. It is not a mark beneath a path prefix. The command interface reduces a message type to a bare mark name, and a dojo poke also names a bare mark. A type of the form [/auspex %chain] cannot be addressed from another ship, and a delivery that uses it is not received. The auspex- prefix prevents a top-level name in a shared tree from colliding with a name that belongs to Grubbery.
To deliver a chain, a sending agent pokes the recipient's %grubbery agent with the mark %grub-cmd and the following noun.
:- chan=@ta :: the sender's return channel
:^ %poke
path=<the recipient's nexus root>
name=%'main.sig' :: the writer grub
:- mark=%auspex-chain
noun=<the chain>The sender first subscribes to the path /client/<chan> on the same agent. The outcome of the poke arrives on that subscription as a %grub-fact that carries [%ack err=(unit tang)].
The recipient's nexus root is a path in the recipient's own tree. The reference implementation uses the following conventional installation path.
/apps/'shell.shell'/desks/'auspex.desk'/desk/data/'auspex.auspex_app'This path is a convention and not a guarantee. The recipient chooses the name of the desk at installation, and delivery to a ship that installed the nexus at another path fails. Nothing is delivered to the wrong place in that case, because a poke to a path that does not exist is refused. It is expected that a later revision of the transport will resolve the recipient's nexus root through a published alias. Such a change affects the transport only and does not affect the message format.
Within Grubbery, the reference sender reaches the remote writer by rewriting the absolute path of the writer into the recipient's subtree, at /sys/ames/ships/<ship>/root/<path>.
Delivery is open to every ship. The nexus grants the right to poke its writer to the public. A receiver MUST NOT require that the delivering ship be a participant in the thread, and MUST NOT require that it be the author of any message in the chain. The signatures are the authority, and the courier is irrelevant. This is what makes a chain portable.
The grant applies to the writer as a whole and not to a single mark. A peer that can reach the writer can also address to it the marks that are reserved for the local user. A receiver MUST refuse a locally reserved action unless the poke originates from the receiver's own ship. The receiver MUST determine the origin from the transport. It MUST NOT determine the origin from the payload.
An implementation can conform at two independent levels.
%grub-cmd poke of Section 8.1 and follows the procedure of Section 9.4. A receiver conforms if it is a Grubbery nexus that accepts %auspex-chain at its writer and publishes its capabilities as required by Section 9.1.
A ship that does not run the nexus cannot receive Auspex mail under version 1. Nothing in the format depends on Grubbery. No Grubbery type is transmitted, and the payload is a bare list of messages.
The handler that receives deliveries MUST NOT terminate abnormally on any input. Every refusal is a normal return. In the reference implementation, the one procedure that can raise an error on hostile input is +thread-key, and it is invoked within an error trap.
A payload that is not a well-formed chain MUST be refused by a normal return. For this reason the wire marks of the reference implementation accept any noun, and the payload is checked against the chain type inside the handler, within an error trap. Appendix B records why a typed wire mark is unsafe on the reference transport.
As a consequence, a delivery that is refused because of a limit, a missing root, or a full store is acknowledged, and the reason is recorded locally. Only a transport failure or an undecodable payload is visible to the sender. The protocol provides no delivery receipt. A send that the network cannot deliver fails without any indication to the sender. This applies to remote delivery only. A send that the sender's own ship refuses is refused locally, where the user can see the refusal.
A sender MUST bound its wait for each recipient. The reference implementation waits 20 seconds. An unreachable recipient MUST NOT block the sender, and a nack from any peer MUST NOT cause the sender to fail. Section 9.5 defines how a sender interprets a poke that is not acknowledged.
On the reference transport, a poke of a mark that the recipient does not implement is never acknowledged. The sender observes a timeout. A recipient that is offline produces the same observation, and so does a recipient that runs an incompatible version. The three conditions cannot be distinguished, and none of them tells the sender what to do.
Each nexus therefore publishes the versions it speaks, and a sender reads that publication before it pokes.
Every nexus MUST publish one noun of the following type.
+$ proto-caps
$: max-blob=@ud
max-attach=@ud
max-chain=@ud
max-body=@ud
max-subj=@ud
max-to=@ud
max-depth=@ud
max-signers=@ud
max-mime=@ud
max-name=@ud
==
::
+$ proto [%auspex versions=(list @ud) marks=(list @tas) caps=proto-caps]As a noun, this is [%auspex [versions [marks caps]]], where caps is a right-nested ten-tuple of atoms in the order shown. A version 1 nexus publishes the following value.
[%auspex ~[1] ~[%auspex-chain] [262.144 16 1.000 100.000 1.000 100 64 128 128 256]]The following requirements apply.
%auspex. It is not a version number. It identifies the noun as an Auspex publication within a namespace that other applications share. A reader MUST check it.versions and marks are parallel. The mark for a version is the entry in marks at the index of that version in versions. A reader MUST refuse a publication in which the two lists differ in length. A reader MUST refuse a publication in which versions is empty. A refused publication is treated as an absent one. See Section 9.8. Two lists are used in place of a map because a list has a shape that any implementation can traverse.versions and marks MUST NOT exceed 64 entries, and a reader MUST refuse a publication that exceeds this bound. A reader MUST check the bound before it traverses either list in full. The noun is published by an unknown party, and a traversal of unbounded length is work whose amount an attacker chooses.caps are the limits that the publisher enforces, with the names and meanings of Section 6.7. A publisher MUST NOT publish a limit that it does not enforce. A published limit that differs from the enforced one causes a sender to transmit a chain that the receiver then discards. A publisher MUST republish when its limits change.The mark under which the reference implementation stores this noun accepts any noun. A typed mark would validate the stored value against the current type on every read, and adding a field to $proto would then make the stored publication unreadable.
The noun is published in the namespace of the hosting agent at the following path.
++ proto-spur ^-(path /auspex/proto)A reader requests the following path.
++ proto-keen-path
|= [agent=@ta case=@ud]
^- path
%+ weld `path`[%g %x (scot %ud case) agent %$ %'1' ~]
proto-spurThe request path has the form /g/x/<case>/<agent>//1/auspex/proto. Each segment has the meaning given in Section 7.4. The requirement concerning the empty segment applies here also. The mark of the response MUST be %auspex-proto. A reader MUST discard any other response.
This is the same permissionless read by which attachment bytes are retrieved, and it is used for the same reason. A keen is the only channel that answers a peer that holds no grant.
A reader SHOULD request cases 1 through 8 in turn, with a deadline of 4 seconds for each, and SHOULD accept the first case that answers. This range is wider than the range used for blobs, and the deadline is shorter. A blob never changes, so its binding moves only when it is restricted. A capability publication moves by one case whenever a ship changes its versions or its limits, which is an ordinary event for a deployed protocol. A namespace read is answered from a cache or by the publisher's kernel, with no application involved. A read that is slow is therefore unlikely to arrive at all. A complete miss costs 32 seconds, which is close to the 30 seconds that a complete miss of a blob costs. That total delays queued mail on first contact, and the total was the quantity held fixed when these numbers were chosen.
A publisher MUST NOT publish unconditionally. Publishing to a bound path binds the next case number. A publisher that published on every deployment would move the binding beyond the range that readers probe. The publication would then be unreadable by every peer, permanently, and no error would be reported.
A publisher reads back the value that is currently bound, through the same request that a peer would issue, and then acts as follows.
| Bound value | Action |
|---|---|
| Nothing is bound. | Publish. The value binds at case 1. |
| A value is bound and equals the value to be published. | Do nothing. A deployment that changes nothing consumes no case. |
| A value is bound and differs from the value to be published. | Withdraw the bound value, then publish. A reader accepts the first case that answers, counting up from 1. If the old value were left bound at a lower case, it would answer every reader, and the new value would never be read. |
| A value is bound and cannot be read. | Do nothing, and report the condition to the operator. Publishing after a failed read consumes a case and gains no information. |
The result is one new case for each change of protocol capabilities and not one for each deployment. The period during which the published limits and the enforced limits can differ is one deployment.
A sender MUST NOT poke a recipient for which it has no discovery result. If a sender transmitted version 1 while its discovery read was still outstanding, outcome 2 below could never occur on first contact. First contact is the send that is most likely to reach a ship that runs another version. The chain waits until the result arrives.
A sender obtains the recipient's $proto and then proceeds according to the first of the following outcomes that applies.
The recipient publishes a version that the sender also implements. The sender pokes the mark for the highest common version. It MUST select the highest common version and not the first one listed. The recipient chooses the order of its list, and a sender that took the first common entry would be bound by that order.
The recipient publishes no version that the sender implements. The sender MUST NOT poke. The send to this recipient fails with a message of the following form.
no common protocol version: ~ship speaks <their versions>, this ship speaks <ours>The send exceeds a limit that the recipient publishes. The sender MUST NOT poke. The error message names the recipient's number, as in the following example.
~ship accepts at most N attachmentsThe same applies to the size of an attachment, the length of the chain, the number of recipients, the subject, the body, the body media type, the depth, and the number of distinct signers. The check MUST use the limits that the recipient published. It MUST NOT use the sender's own limits.
The discovery read returned nothing usable. This covers a timeout, an absent publication, a response with the wrong mark, and a noun that is not a well-formed $proto. The sender makes one attempt under version 1, because an absent publication denotes version 1 (Section 9.8). The sender SHOULD report the condition before the poke and not after a failure, with a message of the following form.
~ship has not answered discovery; sent as version 1A sender MUST reach this outcome only from the empty result of a discovery read that it actually performed. It MUST NOT reach this outcome because a cache entry is absent. The two conditions are different findings, and a sender that conflates them reports a failure that it did not observe.
The discovery read MUST NOT be performed on the process that serializes the ship's writes. A keen is a network round trip, and a keen that times out may still deliver a late response to the process that issued it. The reference implementation consults its cache on the writer. For a recipient with no fresh result, it queues the chain and starts a short-lived process. That process performs the read, applies the four outcomes above, and then pokes or refuses.
Two sends to the same unknown recipient that are issued before the discovery result arrives MUST both be delivered, in the order in which they were written. A queue that released its contents in reverse order would deliver a reply before the message it answers. Every recipient files by prev in any case, so nothing would report the error.
Two of the published limits cannot be checked before a message is signed. A sender that screens a message at composition works from the single message, while max-chain and max-depth describe the whole path that the message joins. These two limits can be checked only at the time of sending. The limit max-name is published for completeness. The receiver enforces it at its boundary, and a sender need not screen for it. The limit max-threads is a property of the receiver's store, and no chain can be measured against it.
A poke that is not acknowledged is not a failed send, and a sender MUST NOT report it as one. On the reference transport, the acknowledgement of a delivery poke does not return to the sending process at all. The sender's deadline therefore expires on every successful send. In repeated measurement between two live ships, every send between the ships timed out at the sender. Every one of them also arrived, was verified, and was stored at the receiver.
A sender MUST NOT treat a send timeout as evidence about the recipient. A sender MUST NOT discard its cached discovery result because of a timeout. A sender SHOULD NOT present a timeout to the user. It is sufficient to record the timeout for an operator, in a form such as the following.
~ship did not ack within 20s; it may still arriveDiscovery is the indication that a peer is alive, and the acknowledgement is not. A recipient that answered discovery did so recently, over a channel that returns a response. If a recipient did not answer discovery, the sender reported that fact before it poked, under outcome 4. In neither case does the timeout add information.
An explicit nack is different in kind. It is a statement by the recipient. A sender reports a nack, and a sender MUST discard its cached discovery result for a recipient that nacks, so that the next send reads the publication again.
A sender SHOULD cache the discovery result for each recipient. Two lifetimes apply.
| Record | Lifetime | Reason |
|---|---|---|
| A record that permits sending. This is a record with a common version, or a recorded absence of a publication. | 1 day | The send itself tests the record. A nack discards the record, so an incorrect record is corrected at its first use. |
| A record that refuses sending, because no version is shared. | 1 hour | Nothing tests the record, because no poke is sent. A record that is never tested is given a shorter lifetime. |
A sender SHOULD record a failed discovery read as such and SHOULD NOT simply leave the cache empty. A recipient that runs a build without discovery then costs one read per day and not one read per send.
A refusal under outcome 3 depends on the message as well as on the record. It cannot be derived from the record alone, and the record keeps the ordinary lifetime. An implementation SHOULD provide a means to discard a cached record on request. Discarding a record MUST NOT discard mail that is queued for the same recipient.
A refusal under Section 9.4 applies only to the recipient that it names. A refusal MUST NOT prevent delivery to the other recipients of the same message. One recipient that published a max-chain of 0 could otherwise block a send to a hundred recipients.
The to field MUST NOT be altered to remove a refused recipient. The field is signed and names the audience that the author chose. Delivery omits the refused recipient. The message does not.
A peer that publishes no capabilities is treated as a version 1 peer. Auspex was deployed before discovery was defined, so the absence of a publication has exactly one possible meaning. The limits of such a peer are the values in Section 6.7. A send to such a peer is still checked against those limits.
This rule makes discovery an additive change. A sender that implements discovery and a receiver that does not implement it interoperate without modification, in both directions.
A receiver accepts every mark that it lists in marks and no other mark. Discovery changes nothing else about a receiver.
Nothing local is ever signed, and nothing signed is ever local.
| Signed and transmitted | Local and never transmitted |
|---|---|
from, life, to, subj, body, body-mime, sent, prev, attachments |
read state, labels, archive state, drafts, filters, mailing lists, blob visibility, blob arrival time, inbox order, the record of blind copies, verdicts, the discovery cache |
An implementation MUST NOT infer any item in the right-hand column from a chain. It MUST NOT serialize any such item into a message. It MUST NOT treat a disagreement between two ships about such an item as an error. Two ships may disagree about every item in the right-hand column. They cannot disagree about who signed what. This separation is what permits the unsigned structure to be frozen.
The following specific rules apply.
from, no life, no sent, and no signature. Signing occurs once, at the moment of sending, over the fields as they stand at that moment. A client MUST NOT present a draft as a message alongside signed messages. A client SHOULD enforce the distinction structurally. The reference implementation stores drafts outside the message tree, in a form that the decoder for stored messages rejects.to field of a message sent to a list contains the member ships, exactly as if the author had entered each of them.$proto publication is public. What a ship remembers about a peer's publication is local state. A ship MUST NOT accept discovery records from a peer. A peer that could write another ship's cache could claim a version that a recipient does not speak, or limits that a recipient does not enforce. Either claim would cause mail to be sent that is silently lost.The file protocol/vectors/v1.json in [AUSPEX-VECTORS] contains deterministic test cases that a second implementation is expected to reproduce byte for byte. Every ship named in the file is a fake ship. The key pair of a fake ship is derived from its address (Section 5.6), so the cases can be reproduced anywhere, with no network and no Azimuth snapshot.
The file is a JSON object of the following form.
{ "version": 1,
"digest_tag": "auspex",
"caps": { … every limit, as a number … },
"cases": [ … ] }Each case has the following members.
| Member | Meaning |
|---|---|
name |
The identifier of the case. |
kind |
One of message, chain, or cap. |
unsigned |
The nine fields, in readable form. |
jam |
(jam unsigned), printed as @uw. |
msg_id |
(sham unsigned), printed as @ux. |
digest |
(shaf %auspex (sham unsigned)), printed as @ux. |
sig |
The signature, printed as @ux. |
signer |
The ship whose key produced the signature. Where this differs from from, the case is a forgery. |
verdict |
The verdict that a verifier MUST produce when it holds the fake keys of the ships listed in the case as known. |
Version 1 of the file contains the following cases: root, reply, two-branch-tree, forged-copy, same-id-pair, unverifiable-life, orphan-chain, three-recipients, two-attachments, cap-max-chain, cap-max-body, cap-max-subj, cap-max-to, cap-max-mime-length, cap-mime-control-byte, cap-max-attach, cap-max-blob, cap-max-depth, and cap-max-signers.
Each case of kind cap gives the jam of the unsigned of the offending message and the length of the offending chain. Where that value would be larger than the rest of the file, the case gives a sample_recipe that states exactly how to construct the input.
The file has two further top-level members.
set_nounthree-recipients case with the mug of each, the jam of the set that they form, and the jam of the empty set. This member makes the encoding of Section 4.3 testable.
protojam in @uw and in @ux. The member also gives the published path, the response mark, the request path at case 1 with the empty segment written as //, the limits, the cache lifetime, and the compatibility rule of Section 9.8 as silent_peer_is_version: 1. The $proto noun is the only noun in this protocol that crosses the network without being hashed. Its bytes are therefore the contract. A field added to it, or a reordering of $proto-caps, would change those bytes without changing any identifier or digest.
An implementation conforms to the format if it satisfies both of the following.
jam, the same msg_id, the same digest, the same sig from the key of the stated signer, and the same verdict from the stated set of known keys.jam values in set_noun and in proto.An implementer SHOULD compare jam first. The jam is the whole unsigned noun. A difference in jam isolates a failure to the order or the type of a field, before any hashing or cryptography is involved.
The vectors are produced from the reference library by the generator gen/auspex-vectors.hoon in [AUSPEX-VECTORS]. The generator writes the JSON document into the ship's file system, and the published file is a byte-for-byte copy of that output.
The reference test tests/lib/auspex-vectors.hoon reads the published file itself. It asserts that the library reproduces every value in the file, including the descriptive strings. A test that generates its own expected values proves nothing about the file that an implementer downloads, and an earlier revision of the vectors lost two characters in manual transcription while such a test continued to pass. A change to a single character of the published file now fails the test.
This section records the status of known implementations at the time of publication, in the manner described in [RFC7942].
There is one implementation. It is the reference implementation at https://github.com/nisfeb/auspex, and its author is the author of this document. It consists of a Grubbery nexus, a web client, a desktop client, and an extension for a third-party mail client. It implements every part of this document. At the time of publication, its test suites held 89 tests of the chain library, 48 tests of the client interface, and 15 tests of conformance to the vectors.
Two limits on the evidence are stated here because they bear on interoperability.
%urmail (Section 4.6). It demonstrates the key interfaces and not the version 1 tag.%puby retrieval path of the nexus had not been executed on a live ship.No independent implementation is known. A report of one is welcome at the address given in the Status of This Memo.
The protocol assumes that any party may deliver any noun to a receiver and may read anything that a ship publishes. It assumes that the author of a message controls every field of that message, signed or not. The considerations below follow from these assumptions.
The key that signs Auspex messages is the ship's networking key. The same key authenticates network packets and signs attestations. A signature made with that key over bytes that an attacker chose could be presented in another context if the two contexts shared a space of preimages. The %auspex salt in the digest (Section 4.6) keeps the space of Auspex preimages disjoint from every other use of the key. An Auspex signature cannot be replayed in another context, and a signature from another context cannot be presented as an Auspex signature. The salt is a requirement and not an option for this reason.
A valid signature establishes authorship. It does not establish that a value is safe to use. The fields discussed in Section 4.4 can carry header injection and path traversal payloads under a valid signature. Because a signed field cannot be repaired, the receiving boundary is the only place where such a value can be refused. The sent field cannot be refused at all. An implementation that uses sent for any decision other than display order gives the author of a message control over that decision.
Two messages can share an identifier and carry different signatures, one valid and one invalid. Suppose that an implementation detected duplicates by identifier alone. A malicious forwarder could then deliver the invalid copy first. The genuine copy would later be discarded as a duplicate. The recipient would hold a %forged message attributed to an innocent ship and would never see the genuine one. The forwarder would have framed the author as a forger.
The pair [id sig] (Section 6.4) defeats this attack in merging. The same attack can be attempted at the level of stored state, by recording one verdict per identifier, and by exhausting storage. For this reason verdicts are recorded per pair, and the ranked discarding of Section 6.5 never removes a %verified copy in favour of a copy of lower rank.
Suppose that the per-identifier copy bound were enforced by refusing deliveries. An attacker could deliver the maximum number of forged copies of a thread's root to a ship that had never seen the thread. This would create the thread under its genuine identifier, which is derived from content, filled with junk. When the genuine chain arrived later from any participant, the count would exceed the bound and the chain would be refused. Every send carries its whole path. Every later message in that thread would therefore also be refused, permanently, and the attack would cost a few messages with junk signatures. Section 6.5 requires discarding for this reason.
The bound on distinct identifiers in a thread is still enforced by refusal. It remains open to a related attack, which is described in Section 13.9.
If thread identity were derived from the order of a chain or from sent values, a single delivery could duplicate an existing conversation under a new identifier. It could also move an established conversation to an identifier of the attacker's choice. Rule 1 of Section 6.2 prevents the second attack. A delivery that combines one message the receiver already holds with a new root message is filed in the established thread.
Delivery is open, and every limit in Section 6.7 therefore bounds work that an anonymous peer can impose. Three points deserve attention.
max-signers limit is checked before any retrieval for that reason.max-signers limit bounds the cost of one poke. It does not bound the cost that one sender can impose. A peer that sends a thousand pokes imposes a thousand times the cost. A budget applied to each source ship is the appropriate defence, and the reference implementation does not have one.In the same way, the list bound of Section 9.1 is required to be checked before the lists are traversed, because the publication comes from an unknown party.
The identity of the delivering ship has no meaning in this protocol, and an implementation MUST NOT assign it one. Two checks depend on information from the transport. The check of the origin of a locally reserved action (Section 8.2) is one, and any rate limiting applied to a source ship is the other. Both MUST use the origin that the transport reports. Neither may use any field of the payload.
The verdict %forged is an accusation against a named ship. The separation of %unverified from %forged in Section 5.4 ensures that the accusation is made only when a key was actually held and the signature actually failed against it. A modified life field can prevent a message from being verified. It cannot make a genuine message read as forged.
The following weaknesses are known and are not addressed in version 1.
max-chain. No valid signature is required, because forged messages are stored and counted. Every legitimate message is refused from then on. The remedy is to discard distinct identifiers as well, with a preference for retaining %verified messages. That remedy is deferred. It has not been shown to be unnecessary.This section follows the guidance of [RFC6973].
Disclosure by forwarding. A chain is designed to be forwarded, and everything in it is signed, permanent, and attributable to its author. Section 6.3 limits a send to a single path for this reason. If a sender transmitted a whole thread, a side conversation between two participants on one branch would be disclosed to a third party when a message on a different branch was forwarded. The third party would receive that side conversation with signatures attached.
Recipients. The to field is signed and is visible to every recipient, including every later recipient of a forward. A blind copy is not recorded in the message in any form, and the protocol carries no commitment to it.
Discovery. The capability publication of Section 9.1 is readable at a fixed path by any ship on the network. The read is not logged, and the publisher cannot refuse it. A reader learns that the ship runs Auspex. It also learns a version list and ten numbers, which together may identify a software build. This disclosure is accepted. A sender needs the information before its first poke to a ship with which it has no prior relationship, and a keen is the only channel that serves such a sender. Restricting the read would require a grant, and a grant presupposes the relationship that first contact lacks. The publication contains nothing that is specific to a user and nothing about mail. The same information can be inferred by any party that is willing to send the ship a message.
Attachments. A blob is published by content hash. A party that knows the hash can retrieve the bytes from any ship that publishes them. Restriction (Section 7.5) withdraws one ship's copy and cannot recall copies that other ships have already retrieved.
Metadata. This protocol does not conceal from the network the fact that two ships communicate. It relies on the confidentiality that the underlying transport provides between the two ships. A relaying courier sees the whole chain that it carries.
This document has no IANA actions.
The table below pairs rules of this document with the tests in tests/lib/auspex-chain.hoon of the reference implementation that exercise them. It is provided so that the specification and the test suite can be compared. Tests of behaviour that is purely local to the reference client, such as search, pagination, labels, and storage layout, are omitted.
| Rule | Test |
|---|---|
| A signature made with a ship's key verifies against that ship's key. | test-sign-verify-roundtrip |
| A signature does not verify against the key of a different ship. | test-sign-wrong-key-fails |
| A signature does not verify against a different message. | test-sign-wrong-message-fails |
The digest is (shaf %auspex (sham unsigned)). |
test-digest-is-salted-sham |
| The digest differs from the unsalted hash and from the same noun salted for another protocol. | test-digest-domain-separated |
| The message identifier covers each of the nine signed fields. | test-msg-id-covers-every-field |
| A third party that has never communicated with the author verifies the author's signature in a forwarded chain. | test-third-party-verifies-forwarded-chain |
A modified body is %forged and not %unverified. |
test-tampered-body-is-forged |
A modified life is %unverified and not %forged. |
test-tampered-life-is-unverified |
A missing key is %unverified and never %forged. |
test-missing-key-is-unverified |
| One chain yields a separate verdict for each message. | test-mixed-verdicts-per-message |
| A message signed under life 2 verifies against the life 2 key of its author. | test-verifies-under-rotated-life |
A verdict is recorded per pair [id sig]. |
test-verdict-keyed-on-id-and-sig |
| Merging the same chain twice changes nothing. | test-merge-dedupes |
Merging orders by sent, whatever the order of arrival. |
test-merge-orders-by-sent |
| Two copies with one identifier and different signatures both survive merging. | test-merge-keeps-both-copies-on-sig-collision |
| An incoming chain that repeats a message is deduplicated against itself. | test-merge-dedupes-within-new |
| Pruning discards the excess and does not refuse the chain. | test-prune-sheds-excess-rather-than-rejecting |
Pruning never discards a %verified copy, in either order of input. |
test-prune-never-sheds-a-verified |
Pruning retains %unverified in preference to %forged. |
test-prune-prefers-unverified-over-forged |
The full ranking is %verified, then %unverified, then %forged. |
test-prune-ranks-verified-then-unverified-then-forged |
| A definitive verdict is never overwritten. | test-freeze-keeps-a-definitive-verdict |
A later delivery can replace an %unverified verdict. |
test-freeze-upgrades-an-unverified |
| The thread identifier is the identifier of the root message, and every ship computes the same one. | test-root-is-first-message-id |
| Thread identity comes from content and never from the order of the supplied list. | test-thread-key-from-content-not-list-order |
Thread identity never derives from sent. |
test-thread-key-ignores-sent |
| The identity of an established thread is immutable. | test-thread-key-established-thread-is-immutable |
| A root that arrives with a modified copy of itself still resolves on first contact. | test-thread-key-tolerates-a-shadowed-root |
| The limits on incoming data refuse and do not truncate, and one bad message condemns the chain. | test-input-caps-reject-on-any-message |
| The capacity bound counts distinct identifiers and not copies. | test-distinct-ids-counts-ids-not-copies |
| The depth limit measures the deepest path and not the number of messages. | test-max-ancestry-is-the-deepest-path |
The signer limit counts distinct pairs [ship life] and not messages. |
test-signer-cap-counts-keys-not-messages |
| The ancestry of a message is the list of identifiers from the root down to it, inclusive. | test-ancestors-are-root-first |
| The parent table is indexed by identifier and not by signature. | test-prev-map-is-keyed-by-id-not-signature |
| A forward carries the path and not the sibling branch. | test-path-chain-omits-the-sibling-branch |
| A forwarded branch travels whole. | test-path-chain-carries-the-whole-path |
| A forwarded path is a valid chain with a unique root and the same thread identifier. | test-path-chain-is-a-fileable-chain |
| Every copy at a node on the path travels. | test-path-chain-keeps-both-copies-of-a-node |
| An orphan is placed as its own root and is not dropped. | test-orphan-is-its-own-root |
A cycle of prev values terminates. |
test-ancestors-survives-a-cycle |
+with-root adds the root only when the path lacks one. |
test-with-root-only-adds-when-the-root-is-missing |
| The content address covers the length as well as the bytes. | test-blob-hash-covers-length |
| A blob is accepted only if its bytes hash to the requested address. | test-blob-ok-rejects-wrong-bytes |
| A descriptor is held to the same limits as the file it describes. | test-attach-ok-enforces-the-same-caps |
The attachment count limit is max-attach. |
test-attaches-ok-caps-the-count |
name and mime refuse control bytes at the boundary. |
test-text-ok-refuses-control-bytes |
Replacing an attachment makes the message %forged. |
test-swapped-attachment-is-forged |
| The attachment limits apply per message and refuse the chain whole. | test-incoming-attachment-caps |
| The blob paths are content-addressed, have no revision segment, and contain the empty segment. | test-blob-paths-are-content-addressed |
| A draft is not a stored message, and the decoder refuses one. | test-a-draft-is-not-a-stored-message |
| A filter cannot suppress a forged message. | test-a-filter-cannot-hide-a-forgery |
| A search finds a forged message and labels it. | test-search-finds-and-names-the-forged-message |
| A sender selects the highest common version. | test-common-version-picks-the-highest |
| No common version is a refusal and not a fallback. | test-common-version-answers-none |
| A peer with no publication is version 1, with the limits of version 1. | test-a-peer-with-no-proto-is-version-1 |
versions and marks are parallel, and a publication whose lists disagree is refused. |
test-mark-for-follows-the-parallel-lists |
| The limit check before sending uses the limits of the peer. | test-peer-cap-check-uses-the-peers-caps-not-ours |
| The published limits are the enforced limits. | test-published-caps-are-the-enforced-caps |
| A cached discovery result expires after a day, and a record dated in the future is not fresh. | test-a-peer-record-expires |
| A refusing record expires after an hour, and a recorded absence is not a refusal. | test-a-refusing-record-expires-sooner |
| The capability path and its request path correspond, including the empty segment. | test-proto-paths-mirror-each-other |
| The pending queue preserves the order of writing. | test-the-pending-queue-drains-in-order |
| The publication check bounds its traversal before measuring, and refuses lists of unequal length. | test-proto-ok-bounds-the-walk |
This appendix is not normative. It records experience from the reference implementation that another implementer on the same platform is likely to need.
Typed wire marks. The reference implementation first declared its wire mark with the chain type, so that the runtime would validate a payload before the handler ran. Grubbery performs that validation before any nexus code runs. A failure of that validation terminates the writer process, and the restarted process consumes the next poke without handling it. A typed wire mark therefore never refused a malformed chain. It silently destroyed the valid delivery that followed, and delivery is open to every ship on the network. The behaviour was reproduced, and the correction was confirmed between two ships. An implementation whose transport treats a failure of payload validation as a failure of the process MUST NOT type its wire mark. The same reasoning applies to the mark of the capability publication (Section 9.1).
Separation of pure and effectful code. Jael answers only at the current time (Section 5.3). The reference library is therefore divided into pure procedures and a thin layer that the running nexus calls. The pure procedures take keys as arguments and perform all hashing, signing, and verification. The thin layer performs the scries. Every procedure quoted in this document belongs to the pure part. The pure part can be tested without a network.
Discovery off the write path. A keen that times out can leave a late response for the process that issued it. On a long-lived process, such responses accumulate. The reference implementation issues every discovery read and every blob retrieval from a short-lived process, and never from the writer.
Case numbers. The publication rules of Sections 7.4 and 9.3 were written after a test deployment moved a binding beyond the probed range by publishing repeatedly. No error is reported to any party when this occurs. An implementation on the same namespace should test for it explicitly.