theme: Switch to docsy theme

10 files changed, 474 insertions, 0 deletions

diff --git a/content/en/docs/Concepts/_index.md b/content/en/docs/Concepts/_index.md
new file mode 100644
index 0000000..ea9b33c
--- /dev/null
+++ b/content/en/docs/Concepts/_index.md
@@ -0,0 +1,12 @@
+---
+title: "Concepts"
+linkTitle: "Concepts"
+weight: 40
+description: >
+    The concepts underpinning recursive networks in general
+    and Ouroboros in particular.
+---
+
+{{% pageinfo %}}
+Under construction, subpages are accessible.
+{{% /pageinfo %}}
diff --git a/content/en/docs/Concepts/aschenbrenner.png b/content/en/docs/Concepts/aschenbrenner.png
new file mode 100644
index 0000000..b9b11f6
--- /dev/null
+++ b/content/en/docs/Concepts/aschenbrenner.png
diff --git a/content/en/docs/Concepts/creating_layers.jpg b/content/en/docs/Concepts/creating_layers.jpg
new file mode 100644
index 0000000..fe60019
--- /dev/null
+++ b/content/en/docs/Concepts/creating_layers.jpg
diff --git a/content/en/docs/Concepts/dependencies.jpg b/content/en/docs/Concepts/dependencies.jpg
new file mode 100644
index 0000000..eaa9e79
--- /dev/null
+++ b/content/en/docs/Concepts/dependencies.jpg
diff --git a/content/en/docs/Concepts/elements.md b/content/en/docs/Concepts/elements.md
new file mode 100644
index 0000000..bfc7fab
--- /dev/null
+++ b/content/en/docs/Concepts/elements.md
@@ -0,0 +1,96 @@
+---
+title: "Elements of a recursive network"
+author: "Dimitri Staessens"
+date:  2019-07-11
+weight: 2
+description: >
+    The building blocks for recursive networks.
+---
+
+This section describes the high-level concepts and building blocks are
+used to construct a decentralized [recursive network](/docs/what):
+layers and flows. (Ouroboros has two different kinds of layers, but
+we will dig into all the fine details in later posts).
+
+A __layer__ in a recursive network embodies all of the functionalities
+that are currently in layers 3 and 4 of the OSI model (along with some
+other functions). The difference is subtle and takes a while to get
+used to (not unlike the differences in the term *variable* in
+imperative versus functional programming languages). A recursive
+network layer handles requests for communication to some remote
+process and, as a result, it either provides a handle to a
+communication channel -- a __flow__ endpoint --, or it raises some
+error that no such flow could be provided.
+
+A layer in Ouroboros is built up from a bunch of (identical) programs
+that work together, called Inter-Process Communication (IPC) Processes
+(__IPCPs__). The name "IPCP" was first coined for a component of the
+[LINCS]
+(https://www.osti.gov/biblio/5542785-delta-protocol-specification-working-draft)
+hierarchical network architecture built at Lawrence Livermore National
+Laboratories and was taken over in the RINA architecture. These IPCPs
+implement the core functionalities (such as routing, a dictionary) and
+can be seen as small virtual routers for the recursive network.
+
+{{<figure width="60%" src="/docs/concepts/rec_netw.jpg">}}
+
+In the illustration, a small 5-node recursive network is shown. It
+consists of two hosts that connect via edge routers to a small core.
+There are 6 layers in this network, labelled __A__ to __F__.
+
+On the right-hand end-host, a server program __Y__ is running (think a
+mail server program), and the (mail) client __X__ establishes a flow
+to __Y__ over layer __F__ (only the endpoints are drawn to avoid
+cluttering the image).
+
+Now, how does the layer __F__ get the messages from __X__ to __Y__?
+There are 4 IPCPs (__F1__ to __F4__) in layer __F__, that work
+together to provide the flow between the applications __X__ and
+__Y__. And how does __F3__ get the info to __F4__? That is where the
+recursion comes in. A layer at some level (its __rank__), will use
+flows from another layer at a lower level. The rank of a layer is a
+local value. In the hosts, layer __F__ is at rank 1, just above layer
+__C__ or layer __E_. In the edge router, layer __F__ is at rank 2,
+because there is also layer __D__ in that router. So the flow between
+__X__ and __Y__ is supported by flows in layer __C__, __D__ and __E__,
+and the flows in layer __D__ are supported by flows in layers __A__
+and __B__.
+
+Of course these dependencies can't go on forever. At the lowest level,
+layers __A__, __B__, __C__ and __E__ don't depend on a lower layer
+anymore, and are sometimes called 0-layers. They only implement the
+functions to provide flows, but internally, they are specifically
+tailored to a transmission technology or a legacy network
+technology. Ouroboros supports such layers over (local) shared memory,
+over the User Datagram Protocol, over Ethernet and a prototype that
+supports flows over an Ethernet FPGA device. This allows Ouroboros to
+integrate with existing networks at OSI layers 4, 2 and 1.
+
+If we then complete the picture above, when __X__ sends a packet to
+__Y__, it passes it to __F3__, which uses a flow to __F1__ that is
+implemented as a direct flow between __C2__ and __C1__. __F1__ then
+forwards the packet to __F2__ over a flow that is supported by layer
+__D__. This flow is implemented by two flows, one from __D2__ to
+__D1__, which is supported by layer A, and one from __D1__ to __D3__,
+which is supported by layer __B__. __F2__ will forward the packet to
+__F4__, using a flow provided by layer __E__, and __F4__ then delivers
+the packet to __Y__.  So the packet moves along the following chain of
+IPCPs: __F3__ --> __C2__ --> __C1__ --> __F1__ --> __D2__ --> __A1__
+--> __A2__ --> __D1__ --> __B1__ --> __B2__ --> __D3__ --> __F2__ -->
+__E1__ --> __E2__ --> __F4__.
+
+{{<figure width="40%" src="/docs/concepts/dependencies.jpg">}}
+
+A recursive network has __dependencies__ between layers in the
+network, and between IPCPs in a __system__. These dependencies can be
+represented as a directed acyclic graph (DAG). To avoid problems,
+these dependencies should never contain cycles (so a layer I should
+not directly or indirectly depend on itself). The rank of a layer is
+defined (either locally or globally) as the maximum depth of this
+layer in the DAG.
+
+---
+Changelog:
+
+2019 07 11: Initial version.<br>
+2019 07 23: Added dependency graph figure
diff --git a/content/en/docs/Concepts/layers.jpg b/content/en/docs/Concepts/layers.jpg
new file mode 100644
index 0000000..5d3020c
--- /dev/null
+++ b/content/en/docs/Concepts/layers.jpg
diff --git a/content/en/docs/Concepts/layers.md b/content/en/docs/Concepts/layers.md
new file mode 100644
index 0000000..e7c1441
--- /dev/null
+++ b/content/en/docs/Concepts/layers.md
@@ -0,0 +1,90 @@
+---
+title: "Recursive layers"
+author: "Dimitri Staessens"
+#description: "IRMd"
+date:  2019-07-23
+weight: 20
+#type:  page
+draft: false
+description: >
+    How to build a recursive network.
+---
+
+The most important structure in recursive networks are the layers,
+that are built op from [elements](/docs/elements/) called
+Inter-Process Communication Processes (IPCPs). (Note again that the
+layers in recursive networks are not the same as layers in the OSI
+model).
+
+{{<figure width="50%" src="/docs/concepts/creating_layers.jpg">}}
+
+Now, the question is, how do we build these up these layers? IPCPs are
+small programs (think of small virtual routers) that need to be
+started, configured and managed. This functionality is usually
+implemented in some sort of management daemon. Current RINA
+implementations call it the *IPC manager*, Ouroboros calls it the
+__IPC Resource Management daemon__ or __IRMd__ for short. The IRMd
+lies at the heart of each system that is participating in an Ouroboros
+network, implementing the core function primitives. It serves as the
+entry point for the system/network administrator to manage the network
+resources.
+
+We will describe the functions of the Ouroboros IRMd with the
+Ouroboros commands for illustration and to make things a bit more
+tangible.
+
+The first set of primitives, __create__ (and __destroy__), allow
+creating IPCPs of a given *type*. This just runs the process without
+any further configuration. At this point, that process is not part of
+any layer.
+
+```
+$ irm ipcp create type unicast name my_ipcp
+$ irm ipcp list
++---------+----------------------+------------+----------------------+
+|     pid |                 name |       type |                layer |
++---------+----------------------+------------+----------------------+
+|    7224 |              my_ipcp |    unicast |         Not enrolled |
++---------+----------------------+------------+----------------------+
+```
+
+The example above creates a unicast IPCP and gives that IPCP a name
+(we called it "my_ipcp"). A listing of the IPCPs in the system shows
+that the IPCP is running as process 7224, and it is not part of a
+layer ("*Not enrolled*").
+
+To create a new functioning network layer, we need to configure the
+IPCP, using a primitive called __bootstrapping__. Bootstrapping sets a
+number of configuration optionss for the layer (such as the routing
+algorithm to use) and activates the IPCP to allow it to start
+providing flows.  The Ouroboros command line allows creating an IPCP
+with some default values, that are a bit like a vanilla IPv4 network:
+32-bit addresses and shortest-path link-state routing.
+
+```
+$ irm ipcp bootstrap name my_ipcp layer my_layer
+$ irm ipcp list
++---------+----------------------+------------+----------------------+
+|     pid |                 name |       type |                layer |
++---------+----------------------+------------+----------------------+
+|    7224 |              my_ipcp |    unicast |             my_layer |
++---------+----------------------+------------+----------------------+
+```
+
+Now we have a single node-network. In order to create a larger
+network, we connect and configure new IPCPs using a third primitive
+called __enrollment__. When enrolling an IPCP in a network, it will
+create a flow (using a lower layer) to an existing member of the
+layer, download the bootstrapping information, and use it to configure
+itself as part of this layer.
+
+The final primitive is the __connect__ (and __disconnect__)
+primitive. This allows to create *adjacencies* between network nodes.
+
+An example of how to create a small two-node network is given in
+[tutorial 2](/docs/tutorials/tutorial-2/)
+
+---
+Changelog:
+
+2019-07-23: Initial version
diff --git a/content/en/docs/Concepts/protocols.md b/content/en/docs/Concepts/protocols.md
new file mode 100644
index 0000000..16c72cb
--- /dev/null
+++ b/content/en/docs/Concepts/protocols.md
@@ -0,0 +1,130 @@
+---
+title: "The protocols"
+author: "Dimitri Staessens"
+#description: protocols
+date:  2019-09-06
+#type:  page
+draft: false
+description: >
+    A brief introduction to the main protocols.
+---
+
+# Network protocol
+
+As Ouroboros tries to preserve privacy as much as possible, it has an
+*absolutely minimal network protocol*:
+
+```
+  0                   1                   2                   3
+  0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+ |                                                               |
+ +                      Destination Address                      +
+ |                                                               |
+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+ | Time-to-Live  |      QoS      |     ECN       |     EID       |
+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+ |      EID      |
+ +-+-+-+-+-+-+-+-+
+```
+
+The 5 fields in the Ouroboros network protocol are:
+
+* **Destination address**: This specifies the address to forward the
+  packet to. The width of this field is configurable based on various
+  preferences and the size of the envisioned network. The Ouroboros
+  default is 64 bits. Note that there is _no source address_, this is
+  agreed upon during _flow allocation_.
+
+* **Time-to-Live**: Similar to IPv4 and IPv6 (where this field is called
+  Hop Limit), this is decremented at each hop to ensures that packets
+  don't get forwarded forever in the network, for instance due to
+  (transient) loops in the forwarding path. The Ouroboros default for
+  the width is one octet (byte).
+
+* **QoS**: Ouroboros supports Quality of Service via a number of methods
+  (out of scope for this page), and this field is used to prioritize
+  scheduling of the packets when forwarding. For instance, if the
+  network gets congested and queues start filling up, higher priority
+  packets (e.g. a voice call) get scheduled more often than lower
+  priority packets (e.g. a file download). By default this field takes
+  one octet.
+
+* **ECN**: This field specifies Explicit Congestion Notification (ECN),
+  with similar intent as the ECN bits in the Type-of-Service field in
+  IPv4 / Traffic Class field in IPv6. The Ouroboros ECN field is by
+  default one octet wide, and its value is set to an increasing value
+  as packets are queued deeper and deeper in a congested routers'
+  forwarding queues. Ouroboros enforces Forward ECN (FECN).
+
+* **EID**: The Endpoint Identifier (EID) field specified the endpoint for
+  which to deliver the packet. The width of this field is configurable
+  (the figure shows 16 bits). The values of this field is chosen by
+  the endpoints, usually at _flow allocation_. It can be thought of as
+  similar to an ephemeral port. However, in Ouroboros there is no
+  hardcoded or standardized mapping of an EID to an application.
+
+# Transport protocol
+
+Packet switched networks use transport protocols on top of their
+network protocol in order to deal with lost or corrupted packets.
+
+The Ouroboros Transport protocol (called the _Flow and Retransmission
+Control Protocol_, FRCP) has only 4 fields:
+
+```
+  0                   1                   2                   3
+  0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+ |            Flags              |            Window             |
+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+ |                        Sequence Number                        |
+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+ |                    Acknowledgment Number                      |
+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+
+```
+
+* **Flags**: There are 7 flags defined for FRCP.
+
+  - **DATA**: Indicates that the packet is carrying data (this allows
+          for 0 length data).
+
+  - **DRF** : Data Run Flag, indicates that there are no unacknowledged
+          packets in flight for this connection.
+
+  - **ACK** : Indicates that this packet carries an acknowledgment.
+  - **FC**  : Indicates that this packet updates the flow control window.
+  - **RDVZ**: Rendez-vous, this is used to break a zero-window deadlock
+          that can arise when an update to the flow control window
+          gets lost. RDVZ packets must be ACK'd.
+  - **FFGM**: First Fragment, this packet contains the first fragment of
+          a fragmented payload.
+  - **MFGM**: More Fragments, this packet is not the last fragment of a
+          fragmented payload.
+
+* **Window**: This updates the flow control window.
+
+* **Sequence Number**: This is a monotonically increasing sequence number
+                   used to (re)order the packets at the receiver.
+
+* **Acknowledgment Number**: This is set by the receiver to indicate the
+                         highest sequence number that was received in
+                         order.
+
+# Operation
+
+The operation of the transport protocol is based on the [Delta-t
+protocol]((https://www.osti.gov/biblio/5542785-delta-protocol-specification-working-draft)),
+which is a timer-based protocol that is a bit simpler in operation
+than the equivalent functionalities in TCP. In contrast with TCP/IP,
+Ouroboros does congestion control purely in the network protocol, and
+fragmentation and flow control purely in the transport protocol.
+
+
+
+
+--- Changelog:
+
+2019 09 05: Initial version.<br>
+2019 09 06: Added section on transport protocol.
\ No newline at end of file
diff --git a/content/en/docs/Concepts/rec_netw.jpg b/content/en/docs/Concepts/rec_netw.jpg
new file mode 100644
index 0000000..bddaca5
--- /dev/null
+++ b/content/en/docs/Concepts/rec_netw.jpg
diff --git a/content/en/docs/Concepts/what.md b/content/en/docs/Concepts/what.md
new file mode 100644
index 0000000..bacd4d6
--- /dev/null
+++ b/content/en/docs/Concepts/what.md
@@ -0,0 +1,146 @@
+---
+title: "What is a recursive network?"
+author: "Dimitri Staessens"
+
+date:  2019-07-06
+weight: 1
+description: >
+     Introduction to recursive networks.
+---
+
+# The current networking paradigm
+{{<figure width="40%" src="/docs/concepts/aschenbrenner.png">}}
+
+Every computer science class that deals with networks explains the
+[7-layer OSI model](https://www.bmc.com/blogs/osi-model-7-layers/).
+Open Systems Interconnect (OSI) defines 7 layers, each providing an
+abstraction for a certain *function* that a network application may
+need.
+
+From top to bottom, the layers provide (roughly) the following
+functions:
+
+The __application layer__ implements the details of the application
+protocol (such as HTTP), which specifies the operations and data that
+the application understands (requesting a web page).
+
+The __presentation layer__ provides independence of data representation,
+and may also perform encryption.
+
+The __session layer__ sets up and manages sessions (think of a session
+as a conversation or dialogue) between the applications.
+
+The __transport layer__ handles individual chunks of data (think of them
+as words in the conversation), and can ensure that there is end-to-end
+reliability (no words or phrases get lost).
+
+The __network layer__ forwards the packets across the network, it
+provides such things as addressing and congestion control.
+
+The __datalink layer__ encodes data into bits and moves them between
+hosts. It handles errors in the physical layer. It has two sub-layers:
+Media access control layer (MAC), which says when hosts can transmit
+on the medium, and logical link control (LLC) that deals with error
+handling and control of transmission rates.
+
+Finally, the __physical layer__ is responsible for translating the
+bits into a signal (e.g. laser pulses in a fibre) that is carried
+between endpoints.
+
+This functional layering provides a logical order for the steps that
+data passes through between applications. Indeed, every existing
+(packet) network goes through these steps in roughly this order
+(however, some may be skipped). There can be some small variations on
+where the functions are implemented. For instance, TCP does congestion
+control, but is a Layer 4 protocol.
+
+However, when looking at current networking solutions, things are not
+as simple as these 7 layers seem to indicate. Just consider this
+realistic scenario for a software developer working remotely. Usually
+it goes something like this: You connect to the company __Virtual
+Private Network__ (VPN), setup an SSH __tunnel__ over the development
+server to your virtual machine and then SSH into that virtual machine.
+
+The use of VPNs and various tunneling technologies draw a picture
+where the __functions__ of Layers 2, 3, and 4, and often layers 5 and
+6 (encryption) are __repeated__ a number of times in the actual
+functional path that data follows through the network stack.
+
+Enter __*recursive networks*__.
+
+# The recursive network paradigm
+
+The functional repetition in the network stack is discussed in
+detail in the book __*"Patterns in Network Architecture: A Return to
+Fundamentals"*__. From the observations in the book, a new architecture
+was proposed, called the "__R__ecursive __I__nter__N__etwork
+__A__rchitecture", or [__RINA__](http://www.pouzinsociety.org).
+
+__Ouroboros__ follows the recursive principles of RINA, but deviates
+quit a bit from its internal design. There are resources on the
+Internet explaining RINA, but here we will focus
+on its high level design and what is relevant for Ouroboros.
+
+Let's look at a simple scenario of an employee contacting an internet
+corporate server over a Layer 3 VPN from home. Let's assume for
+simplicity that the corporate LAN is not behind a NAT firewall. All
+three networks perform (among some other things):
+
+__Addressing__: The VPN hosts receive an IP address in the VPN, let's
+say some 10.11.12.0/24 address. The host will also have a public IP
+address, for instance in the 20.128.0.0/16 range . Finally that host
+will have an Ethernet MAC address. Now the addresses __differ in
+syntax and semantics__, but for the purpose of moving data packets,
+they have the same function: __identifying a node in a network__.
+
+__Forwarding__: Forwarding is the process of moving packets to a
+destination __with intent__: each forwarding action moves the data
+packet __closer__ to its destination node with respect to some
+__metric__ (distance function).
+
+__Network discovery__: Ethernet switches learn where the endpoints are
+through MAC learning, remembering the incoming interface when it sees
+a new SRC address; IP routers learn the network by exchanging
+informational packets about adjacency in a process called *routing*;
+and a VPN proxy server relays packets as the central hub of a network
+connected as a star between the VPN clients and the local area
+network (LAN) that is provides access to.
+
+__Congestion management__: When there is a prolonged period where a
+node receives more traffic than can forward forward, for instance
+because there are incoming links with higher speeds than some outgoing
+link, or there is a lot of traffic between different endpoints towards
+the same destination, the endpoints experience congestion. Each
+network could handle this situation (but not all do: TCP does
+congestion control for IP networks, but Ethernet just drops traffic
+and lets the IP network deal with it. Congestion management for
+Ethernet never really took off).
+
+__Name resolution__: In order not having to remember addresses of the
+hosts (which are in a format that make it easier for a machine to deal
+with), each network keeps a mapping of a name to an address. For IP
+networks (which includes the VPN in our example), this is done by the
+Domain Name System (DNS) service (or, alternatively, other services
+such as *open root* or *namecoin*). For Ethernet, the Address
+Resolution Protocol maps a higher layer name to a MAC (hardware)
+address.
+
+{{<figure width="50%" src="/docs/concepts/layers.jpg">}}
+
+Recursive networks take all these functions to be part of a network
+layer, and layers are mostly defined by their __scope__. The lowest
+layers span a link or the reach of some wireless technology. Higher
+layers span a LAN or the network of a corporation e.g. a subnetwork or
+an Autonomous System (AS). An even higher layer would be a global
+network, followed by a Virtual Private Network and on top a tunnel
+that supports the application. Each layer being the same in terms of
+functionality, but different in its choice of algorithm or
+implementation. Sometimes the function is just not implemented
+(there's no need for routing in a tunnel!), but logically it could be
+there.
+
+
+---
+Changelog:
+
+2019 07 09: Initial version.
\ No newline at end of file