Control protocol transport layer.

appendix.md

@@ -2,20 +2,23 @@
 
 ## ZMQ
 
-Some useful hints regarding the working of the zmq package.
+Some useful hints regarding how the zmq package works.
 
 (router-sockets)=
 ### Particularities of ROUTER sockets
 
 A small introduction to ROUTER sockets, for more details see [zmq guide chapter 3](https://zguide.zeromq.org/docs/chapter3/#Exploring-ROUTER-Sockets).
 
-A router socket assigns a random _identity_ to each connecting peer.
-If, for example, two Components `CA`, `CB` connect to the ROUTER sockets, the socket assigns the identities, which will be called `IA`, `IB` here (they can be any byte sequence).
+The ROUTER socket is mostly used in a server role as it can maintain connections to many peers.
+In order to distinguish peers, it assigns a random _identity_ to each connected peer.
+Application code does not know which peer gets which identity, however, the identity of a peer stays the same for the lifetime of the connection.
 
-Whenever a message is sent to the ROUTER socket from a peer, the socket prepends that identity in front of the message frames.
-For example if `CA` sends the message `Request A`, the ROUTER socket will read `IA|Request A`.
-That way, the answer to that message can be returned to the correct peer and not another one (a ROUTER socket can have many connected peers).
-If the ROUTER's send command is called with `IA|Reply A`, the socket will send `Reply A` to the peer, whose connection is `IA`, in this case `CA`.
+If, for example, two Components `CA`, `CB` connect to a ROUTER socket, the socket assigns identities, which will be called `IA`, `IB` here (they can be any byte sequence).
+
+Whenever a message is sent to a ROUTER socket from a peer, the socket prepends that identity in front of the message frames, before handing the message to the application code.
+For example, if `CA` sends the message `Request A`, the ROUTER socket will read `IA|Request A`.
+That way, an answer to that message can be returned to this same peer, and not another one (a ROUTER socket may have many connected peers).
+Consequently, in order to send such an answer, the identity has to be prepended to the frames to send: Calling the ROUTER's send command with `IA|Reply A`, the socket will send `Reply A` to the peer, whose identity is `IA`, in this case that is `CA`.
 
 The following diagram shows this example communication with two Components:
 sequenceDiagram

control_protocol.md

@@ -1,6 +1,6 @@
 # Control protocol
 
-The control protocol transmits messages via its {ref}`transport-layer` from one Component to another one.
+The control protocol transmits messages via its {ref}`transport-layer` from one Component to another.
 The {ref]}`message-layer` is the common language to understand commands, thus creating a remote procedure call.
 
 
@@ -15,7 +15,7 @@ The transport layer ensures that a message arrives at its destination.
 #### Socket Configuration
 
 Each {ref}`Coordinator <components.md#coordinator>` shall offer one {ref}`ROUTER<appendix.md#router-sockets>` socket, bound to an address.
-The address consists of a host (this can be the host name, an IP address of the device, or "\*" for all IP addresses of the device) and a port number, for example `*:12345` for all IP addresses and the port `12345`.
+The address consists of a host (this can be the host name, an IP address of the device, or "\*" for all IP addresses of the device) and a port number, for example `*:12345` for all IP addresses at the port `12345`.
 
 {ref}`Components <components.md#components>` shall have one DEALER socket connecting to one Coordinator's ROUTER socket.
 
@@ -26,24 +26,28 @@ This DEALER socket shall connect to the other Coordinator's ROUTER socket.
 While the number of DEALER sockets thus required scales badly with the number of Connectors in a LECO Network, the scope of the protocol means that at most a few Coordinators will be involved.
 :::
 
-Messages must be sent to a Coordinator's ROUTER socket.
+Communicating with a Coordinator, messages must be sent to a Coordinator's ROUTER socket.
+Only for acknowledging a {ref}`coordinator-sign-in`, it is permitted to send a message to a Coordinator's DEALER socket.
 
 
 #### Naming scheme
 
 Each Component must have an individual name, given by the user, the _Component name_.
-A Component name must be a series of bytes, without the ASCII character "." (byte value 46).
+A Component name must be a series of ASCII characters, without the character ".".
 Component names must be unique in a {ref}`Node <network-structure.md#node>`, i.e. among the Components (except other Coordinators) connected to a single Coordinator.
-A Coordinator itself must have the Component name `COORDINATOR` (ASCII encoded).
+A Coordinator itself must have the Component name `COORDINATOR`.
 
 Similarly, every Node must have a name, the _Namespace_.
 Every Namespace must be unique in the Network.
+It must be a series of ASCII characters, without the character ".".
 As each Component belongs to exactly one Node, it is fully identified by the combination of Namespace and Component name, which is globally unique.
 This _Full name_ is the composition of Namespace, ".", and Component name.
 For example `N1.CA` is the Full name of the Component `CA` in the Node `N1`.
 
 The receiver of a message may be specified by Component name alone if the receiver belongs to the same Node as the sender.
-In all other cases, the receiver of a message must be specified by Full name.
+In all other cases, the receiver of a message must be specified by the Full name.
+
+The sender of a message must be specified by Full name, except during SIGNIN, when the Component name alone is sufficient.
 
 
 #### Message composition
@@ -58,10 +62,10 @@ A message consists of 4 or more frames.
 
 #### Directory
 
-Each Coordinator shall have a list of the Components (including other Connectors) connected to it.
-This is its _Directory_.
+Each Coordinator shall have a list of the Components (including other Coordinators) connected to it.
+This is its _local Directory_.
 
-The _Glossary_ is the combination of the Directories of all Coordinators in a Network.
+The _global Directory_ is the combination of the Directories of all Coordinators in a Network.
 
 
 ### Conversation protocol
@@ -81,12 +85,12 @@ In the exchange of messages, only the messages over the wire are shown, the conn
 
 After connecting to a Coordinator (`Co1`), a Component (`CA`) shall send a SIGNIN message indicating its Component name.
 The Coordinator shall indicate success/acceptance with an ACKNOWLEDGE response, giving the Namespace and other relevant information, or reply with an ERROR, e.g. if the Component name is already taken.
-In that case, the Coordinator may indicate a suitable still available variation on the indicated Component name.
+In that case, the Coordinator may indicate a suitable, still available variation on the indicated Component name.
 The Component may retry SIGNIN with a different chosen name.
 
 After a successful handshake, the Coordinator shall store the (zmq) connection identity and corresponding Component name in its {ref}`directory`.
 It shall also notify the other Coordinators in the network that this Component signed in, see {ref}`Coordinator coordination<coordinator-coordination>`.
-Similarly, the Component shall store the Namespace and use it from this moment to generate its Full name.
+Similarly, the Component shall store the Namespace and use it from this moment on, to generate its Full name.
 
 If a Component does send a message to someone without having signed in, the Coordinator shall refuse message handling and return an error.
 
@@ -122,22 +126,22 @@ A Component should and a Coordinator shall send a PING and wait some time before
 A Coordinator shall follow the {ref}`sign out routine<signing-out>` for a signed in Component considered dead.
 
 :::{note}
-TBD: Respond to every non empty message with an empty one?
+TBD: Heartbeat details are still to be determined.
 :::
 
 
 ##### Signing out
 
-A Component should tell a Coordinator when it stops participating in the network with a SIGNOUT message.
-The Coordinator shall ACKNOWLEDGE the sign-out and remove the Name from its Directory.
+A Component should send a SIGNOUT message to its Coordinator when it stops participating in the Network.
+The Coordinator shall ACKNOWLEDGE the sign-out and remove the Component name from its local {ref}`directory`.
 It shall also notify the other Coordinators in the network that this Component signed out, see {ref}`Coordinator coordination<coordinator-coordination>`.
 
 :::{mermaid}
 sequenceDiagram
     CA ->> N1: V|COORDINATOR|N1.CA|H|SIGNOUT
     participant N1 as N1.COORDINATOR
     N1 ->> CA: V|N1.CA|N1.COORDINATOR|H|ACKNOWLEDGE
-    Note right of N1: Removes "CA" with identity "IA"<br> from Directory
+    Note right of N1: Removes "CA" with identity "IA"<br> from local Directory
     Note right of N1: Notifies other Coordinators about sign-out of "CA"
     Note left of CA: Shall not send any message anymore except SIGNIN
 :::
@@ -146,7 +150,6 @@ sequenceDiagram
 #### Communication with other Components
 
 The following two examples show how a message is transferred between two components `CA`, `CB` via one or two Coordinators.
-Coordinators shall send messages from their DEALER socket to other Coordinator's ROUTER socket.
 
 Coordinators shall route the message to the corresponding Coordinator or connected Component.
 
@@ -186,7 +189,7 @@ Prerequisites of Communication between two Components are:
 - Both Components are either connected to the same Coordinator (example one), or their Coordinators are connected to each other (example two).
 
 
-The following flow chart shows the decision scheme and message modification in a Coordinator `Co1` of Node `N1`.
+The following flow chart shows the decision scheme and message modification in the Coordinator `Co1` of Node `N1`.
 Its Full name is `N1.Coordinator`.
 `nS`, `nR` are placeholders for sender and recipient Namespaces.
 `recipient` is a placeholder for the recipient Component name.
@@ -201,8 +204,8 @@ flowchart TB
     C0([nS.COORDINATOR DEALER]) == "V|nR.recipient|nS.CA|H|Content" ==> R0
     R0[receive] == "iA|V|nR.recipient|nS.CA|H|Content" ==> CnS{nS == N1?}
     CnS-->|no| RemIdent
-    CnS-->|yes| Clocal{CA in <br>Directory?}
-    Clocal -->|yes| CidKnown{iA is CA's identity<br> in Directory?}
+    CnS-->|yes| Clocal{CA in <br>local Directory?}
+    Clocal -->|yes| CidKnown{iA is CA's identity<br> in local Directory?}
     CidKnown -->|yes| RemIdent
     Clocal -.->|no| E1[ERROR: Sender unknown] ==>|"iA|V|nS.CA|N1.COORDINATOR|H|ERROR: Sender unknown"| S
     S[send] ==> WA([N1.CA DEALER])
@@ -212,7 +215,7 @@ flowchart TB
     CnR{nR?} -- "== N1"--> Local
     Local{recipient<br>==<br>COORDINATOR?} -- "yes" --> Self[Message for Co1<br> itself]
     Self == "V|nR.recipient|nS.CA|H|Content" ==> SC([Co1 Message handling])
-    Local -- "no" --> Local2a{recipient in Directory?}
+    Local -- "no" --> Local2a{recipient in local Directory?}
     Local2a -->|yes, with Identity iB| Local2
     Local2[add recipient identity iB] == "iB|V|nR.recipient|nS.CA|H|Content" ==> R1[send]
     R1 == "V|nR.recipient|nS.CA|H|Content" ==> W1([Wire to N1.recipient DEALER])
@@ -235,26 +238,23 @@ flowchart TB
 
 #### Coordinator coordination
 
-Each Coordinator shall keep an up-to-date {ref}`Glossary<directory>` with the Names of all Components in the Network.
+Each Coordinator shall keep an up-to-date global {ref}`directory` with the Full names of all Components in the Network.
 For this, Coordinators shall notify each other about sign-ins and sign-outs of Components and Coordinators.
-On request, Coordinators shall send the Names of their Directory, or of their Glossary, depending on the request type.
+On request, Coordinators shall send the Names of their local or global Directory, depending on the request type.
 
 For the format of the Messages, see {ref}`message-layer`.
 
 
 ##### Coordinator sign-in
 
-A Coordinator `Co1`joining a network follows a few steps:
-1. It signs in to one Coordinator `Co2` of the Network.
-2. It sends a CO_TELL_ALL message to `Co2`, to tell all other Coordinators about `Co1`s address.
-3. `Co2` tells all the Coordinators signed in (`Co3`, `Co4`...) about `Co1` with a CO_NEW message.
-4. These other Coordinators (`Co3`, `Co4`...) sign in to `Co1`.
-5. All Coordinators are connected to all others.
-
-Two Coordinators shall follow a more thorough sign-in/sign-out procedure than Components (address is for example host and port).
-The sign-in might happen because of a CO_NEW message arrived or at startup.
+A Coordinator joins a Network by signing in to any Coordinator of that Network.
+The sign-in/sign-out procedure between two Coordinators is more thorough than that of Components.
+During the sign-in procedure, Coordinators exchange their local Directories and shall sign in to all Coordinators, they are not yet signed in.
+The sign-in might happen because the Coordinator learns a new Coordinator address via Directory updates or at startup.
 The sign-out might happen because the Coordinator shuts down.
 
+These are the sign-in/sign-out sequences between Coordinators, where `address` is for example the host name and port number of the Coordinator's ROUTER socket.
+
 :::{mermaid}
 sequenceDiagram
     participant r1 as ROUTER
@@ -268,23 +268,22 @@ sequenceDiagram
     activate d1
     Note left of d1: created with<br> name "temp-NS"
     d1-->>r2: connect to address2
-    d1->>r2: CO_SIGNIN<br>N1, address1,<br>ref:temp-NS
-    par
-        d1->>r2: GET Directory
-    and
-        Note right of r2: stores N1 identity
-        activate d2
-        Note left of d2: created with<br>name "N1"
-        d2-->>r1: connect to address1
-        d2->>r1: CO_SIGNIN<br>N2, address2<br>your ref:temp-NS
-        Note right of r1: stores N2 identity
-        Note left of d1: name changed<br>from "temp-NS"<br>to "N2"
-        d2->>r1: GET Directory
-    end
-    d2->>r1: Here is my<br>Directory
-    Note right of r1: Updates<br>Glossary
-    d1->>r2: Here is my<br>Directory
-    Note right of r2: Updates<br>Glossary
+    d1->>r2: V|COORDINATOR|N1.COORDINATOR|H|<br>CO_SIGNIN
+    Note right of r2: stores N1 identity
+    r2->>d1: V|N1.COORDINATOR|N2.COORDINATOR|H|ACK
+    Note left of d1: DEALER name <br>set to "N2"
+    d1->>r2: V|N1.COORDINATOR|N2.COORDINATOR|H|<br>Here is my local directory
+    Note right of r2: Updates global <br>Directory and signs <br>in to all unknown<br>Coordinators,<br>also N1
+    Note over d1,r2: Mirror of above sign-in procedure
+    activate d2
+    Note left of d2: created with<br>name "N1"
+    d2-->>r1: connect to address1
+    d2->>r1: V|COORDINATOR|N2.COORDINATOR|H|<br>CO_SIGNIN
+    Note right of r1: stores N2 identity
+    r1->>d2: V|N2.COORDINATOR|N1.COORDINATOR|H|ACK
+    Note left of d2: Name is already "N1"
+    d2->>r1: V|N2.COORDINATOR|N1.COORDINATOR|H|<br>Here is my local directory
+    Note right of r1: Updates global <br>Directory and signs <br>in to all unknown<br>Coordinators
     Note over r1,d2: Sign out between two Coordinators
     Note right of r1: shall sign out from N2
     d1->>r2: CO_SIGNOUT
@@ -294,12 +293,16 @@ sequenceDiagram
     deactivate d1
 :::
 
+:::{note}
+Note that the DEALER socket responds with the local Directory to the received Acknowledgment.
+:::
+
 
 ##### Coordinator updates
 
-Whenever a Component signs in to or out of its Coordinator, the Coordinator shall send a CO_UPDATE message regarding this event to all the other Coordinators.
+Whenever a Component signs in to or out from its Coordinator, the Coordinator shall send a CO_UPDATE message regarding this event to all the other Coordinators.
 The message shall contain the Full name of the Component and the event type (sign in or out)
-The other Coordinators shall update their Glossary according to this message (add or remove an entry).
+The other Coordinators shall update their global Directory according to this message (add or remove an entry).
 
 
 (message-layer)=
@@ -316,8 +319,6 @@ The other Coordinators shall update their Glossary according to this message (ad
 - PING
 - CO_SIGNIN
 - CO_SIGNOUT
-- CO_TELL_ALL
-- CO_NEW
 - CO_UPDATE
 
 

glossary.md

@@ -23,11 +23,14 @@ Device
 Director
     A Component which takes part in orchestrating a (i.e. LECO-controlled) measurement setup, see {ref}`components.md#director`.
 
+Directory
+    Each Coordinator maintains a local Directory with all the Components connected to it, and a global Directory with all Components in the Network, see {ref}`control_protocol.md#directory`.
+
 Driver
     An object that takes care of communicating with a Device. This object is external to LECO, for example coming from and instrument control library like `pymeasure`, `instrumentkit` or `yaq`. See {ref}`components.md#driver`.
 
 Full name
-    The name of a Component unique in the whole name.
+    The name of a Component unique for the whole setup.
     It consists of the {ref}`namespace` and {ref}`component-name`, see {ref}`control_protocol.md#naming-scheme`.
 
 LECO

network-structure.md

@@ -53,7 +53,7 @@ The Node-local Message Layer can have a local or distributed mode.
 ### Distributed Message Transport (DMT)
 
 The Distributed Message Transport works within or across Nodes.
-It uses [Zmq](https://zeromq.org/) sockets for the communication. For more details see the [zmq guide](https://zguide.zeromq.org/) or [zmq API](http://api.zeromq.org/)
+Currently, the only defined transport layer uses [Zmq](https://zeromq.org/) sockets for the communication. For more details see the [zmq guide](https://zguide.zeromq.org/) or [zmq API](http://api.zeromq.org/)
 
 Zmq messages consist in a series of frames, each is a byte sequence.
 In this documentation, the separation between frames is indicated by `|`.
@@ -68,6 +68,6 @@ For some useful information see our {ref}`appendix.md#zmq`.
 The Local Message Transport only works within a Node _and_ within a process.
 Local Message Transport options include queues between threads/processes and zeromq inproc.
 :::{admonition} Warning
-LMT details are still notional and not to be relied upon this will be fleshed out at a later date. 
+LMT details are still notional and not to be relied upon, this will be fleshed out at a later date. 
 The list of LMT options is not definitive yet.
 :::