doc'd new-in-4.0 keyserver &registry command groups & revamped `r…

…emote`

Signed-off-by: jason yang <[email protected]>

endpoint.rst

@@ -1,3 +1,5 @@
+.. _endpoint:
+
 ################
 Remote Endpoints
 ################

index.rst

@@ -65,7 +65,9 @@ Sharing & Online Services
    :maxdepth: 1
 
    Remote Endpoints <endpoint>
-   Library API Registries <library_api>
+   Keyserver Management <keyserver>
+   OCI Registries <registry>
+   Sylabs Cloud Library <cloud_library>
 
 **************
 Advanced Usage

keyserver.rst

@@ -1,20 +1,110 @@
 .. _keyserver:
 
-###########
-Key Servers
-###########
+####################
+Keyserver Management
+####################
 
-Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor
-incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis
-nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat.
-Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu
-fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in
-culpa qui officia deserunt mollit anim id est laborum.
+By default, {Singularity} will use the keyserver defined by the active
+:ref:`remote endpoint<endpoint>`'s service discovery file. This behavior can be
+changed or supplemented using the ``key`` command and, in particular, its
+subcommands ``add`` and ``remove``. These allow an administrator to create a
+global list of keyservers that will be used to verify container signatures by
+default. When verifying container signatures, {Singularity} consults them
+according to a configured *order* (with the keyserver whose order is ``1``
+consulted first, then the one whose order is ``2``, and so forth). Other
+operations performed by {Singularity} that reach out to a keyserver will only
+use the first one (whose order is ``1``).
 
 .. note::
 
    In previous versions of {Singularity}, the functionality described here was
-   grouped together with :ref:`remote endpoint management<remote>` under the
+   grouped together with :ref:`remote endpoint management<endpoint>` under the
    ``remote`` command group. Beginning with version 4.0, this functionality has
    been given its own top-level command group, ``keyserver``.
 
+The ``list`` subcommand allows the user to examine the set of currently
+configured keyservers:
+
+.. code:: console
+
+   $ singularity keyserver list
+
+   SylabsCloud*^
+      #1  https://keys.sylabs.io  🔒
+
+   (* = system endpoint, ^ = default endpoint,
+    + = user is logged in directly to this keyserver)
+
+We can see in the output of the ``list`` subcommand that "SylabsCloud" is the
+*default* remote endpoint (in other words, the endpoint that will be used by all
+{SingularityCE} commands unless otherwise specified), and that it is a *global*
+(in other words, system-level) endpoint. Furthermore, the lock icon next to
+``https://keys.sylabs.io`` indicates that TLS will be used when communicating
+with this keyserver.
+
+We can add a key server to list of keyservers as follows:
+
+.. code:: console
+
+   $ sudo singularity keyserver add https://pgp.example.com
+   $ singularity keyserver list
+
+   SylabsCloud*^
+      #1  https://keys.sylabs.io   🔒
+      #2  https://pgp.example.com  🔒
+
+   (* = system endpoint, ^ = default endpoint,
+    + = user is logged in directly to this keyserver)
+
+Here, we see that the ``https://pgp.example.com`` keyserver was
+added to the list. We can specify the order in the list in which this keyserver
+should be added, by using the ``--order`` flag:
+
+.. code:: console
+
+   $ sudo singularity keyserver add --order 1 https://pgp.example.com
+   $ singularity keyserver list
+
+   SylabsCloud*^
+      #1  https://pgp.example.com  🔒
+      #2  https://keys.sylabs.io   🔒
+
+   (* = system endpoint, ^ = default endpoint,
+    + = user is logged in directly to this keyserver)
+
+Since we specified ``--order 1``, the ``https://pgp.example.com`` keyserver was
+added as the first entry in the list, and the default keyserver was moved to
+second in the list. With this keyserver configuration, all default image
+verification performed by {Singularity} will, when searching for public keys,
+reach out to ``https://pgp.example.com`` first, and only then to
+``https://keys.sylabs.io``.
+
+If a keyserver requires authentication prior to being used, users can login
+as follows, supplying the password or an API token at the prompt:
+
+.. code:: console
+
+   $ singularity keyserver login --username myname https://pgp.example.com
+   Password / Token:
+   INFO:    Token stored in /home/myuser/.singularity/remote.yaml
+
+The output of `keyserver list` will now show that we are logged in to
+``https://pgp.example.com``:
+
+.. code:: console
+
+   $ singularity keyserver list
+
+   SylabsCloud *^
+      #1  https://pgp.example.com          🔒  +
+      #2  https://keys.sylabs.io           🔒
+
+   (* = system endpoint, ^ = default endpoint,
+    + = user is logged in directly to this keyserver)
+
+.. note::
+
+   It is important for users to be aware that the ``keyserver login`` command
+   will store the supplied credentials or tokens unencrypted in your home
+   directory.
+

registry.rst

@@ -4,17 +4,81 @@
 OCI Image Registries
 ####################
 
-Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor
-incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis
-nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat.
-Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu
-fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in
-culpa qui officia deserunt mollit anim id est laborum.
+It is common for users of {Singularity} to use `OCI
+<https://opencontainers.org/>`__ registries as sources for their container
+images. Some registries require credentials to access certain images or even the
+registry itself. Previously, the only method in {Singularity} to supply
+credentials to registries was to supply credentials for each command or set
+environment variables to contain the credentials for a single registry. See
+:ref:`Authentication via Interactive Login
+<sec:authentication_via_docker_login>` and :ref:`Authentication via Environment
+Variables <sec:authentication_via_environment_variables>`.
+
+Starting with {Singularity} 4.0, users can supply credentials
+on a per-registry basis with the ``registry`` command.
+
+.. note::
+
+   In versions of {Singularity} starting from 3.7 but before 4.0, the
+   functionality described here was grouped together with :ref:`remote endpoint
+   management<endpoint>` under the ``remote`` command group. Beginning with
+   version 4.0, this functionality has been given its own top-level command
+   group, ``registry``.
+
+Users can login to an OCI registry with the ``registry login`` command by
+specifying a ``docker://`` prefix to the registry hostname:
+
+.. code:: console
+
+   $ singularity registry login --username myuser docker://docker.com
+   Password / Token:
+   INFO:    Token stored in /home/myuser/.singularity/remote.yaml
+
+   $ singularity registry list
+
+   URI                  SECURE?
+   docker://docker.com  ✓
+
+{Singularity} will automatically supply the configured credentials when
+interacting with DockerHub. The checkmark in the ``SECURE?`` column indicates
+that {Singularity} will use TLS when communicating with the registry.
+
+A user can be logged-in to multiple OCI registries at the same time:
+
+.. code:: console
+
+   $ singularity registry login --username myuser docker://registry.example.com
+   Password / Token:
+   INFO:    Token stored in /home/myuser/.singularity/remote.yaml
+
+   $ singularity registry list
+
+   URI                            SECURE?
+   docker://docker.com            ✓
+   docker://registry.example.com  ✓
+
+{Singularity} will supply the correct credentials for the registry based on the
+hostname used, whenever one of the following commands is used with a
+``docker://`` or ``oras://`` URI:
+
+`pull
+<https://www.sylabs.io/guides/{version}/user-guide/cli/singularity_pull.html>`__,
+`push
+<https://www.sylabs.io/guides/{version}/user-guide/cli/singularity_push.html>`__,
+`build
+<https://www.sylabs.io/guides/{version}/user-guide/cli/singularity_build.html>`__,
+`exec
+<https://www.sylabs.io/guides/{version}/user-guide/cli/singularity_exec.html>`__,
+`shell
+<https://www.sylabs.io/guides/{version}/user-guide/cli/singularity_shell.html>`__,
+`run
+<https://www.sylabs.io/guides/{version}/user-guide/cli/singularity_run.html>`__,
+`instance
+<https://www.sylabs.io/guides/{version}/user-guide/cli/singularity_instance.html>`__
 
 .. note::
 
-   In previous versions of {Singularity}, the functionality described here was
-   grouped together with :ref:`remote endpoint management<remote>` under the
-   ``remote`` command group. Beginning with version 4.0, this functionality has
-   been given its own top-level command group, ``registry``.
+   It is important for users to be aware that the ``registry login`` command
+   will store the supplied credentials or tokens unencrypted in your home
+   directory.
 

signNverify.rst

@@ -422,7 +422,7 @@ Signing with a PEM key
 To sign a container using a private key in PEM format, provide the private key
 material to the ``sign`` command using the ``--key`` flag. 
 
-.. code:: 
+.. code::
 
    $ {command} sign --key $KEYD/rsa-private.pem lolcow.sif 
    INFO:    Signing image with key material from 'rsa_pri.pem'
@@ -431,7 +431,7 @@ material to the ``sign`` command using the ``--key`` flag.
 
 The DSSE signature descriptor can now be seen by inspecting the SIF file:
 
-.. code:: 
+.. code::
 
    $ {command} sif list lolcow.sif 
    ------------------------------------------------------------------------------
@@ -448,23 +448,25 @@ The DSSE signature descriptor can now be seen by inspecting the SIF file:
    "payloadType": "application/vnd.{command}.sif-metadata+json",
    ...
 
-Attempting to ``verify`` the image without options will fail, as it is not signed with a PGP key:
+Attempting to ``verify`` the image without options will fail, as it is not
+signed with a PGP key:
 
-.. code:: 
+.. code::
 
    $ {command} verify lolcow.sif 
    INFO:    Verifying image with PGP key material
    FATAL:   Failed to verify container: integrity: key material not provided for DSSE envelope signature
 
-Note that the error message shows that the container image has a DSSE signature present.
+Note that the error message shows that the container image has a DSSE signature
+present.
 
 Verifying with a PEM key
 ========================
 
 To verify a container using a PEM public key directly, provide the key material
 to the ``verify`` command using the ``key`` flag:
 
-.. code:: 
+.. code::
 
    $ {command} verify --key $KEYD/rsa-public.pem lolcow.sif 
    INFO:    Verifying image with key material from 'rsa_pub.pem'
@@ -481,11 +483,11 @@ to the ``verify`` command using the ``key`` flag:
 Verifying with an X.509 certificate
 ===================================
 
-To verify a container that was signed with a PEM private key, using an X.509 certificate,
-pass the certificate to the ``verify`` command using the ``--certificate`` flag.
-If the certificate is part of a chain, provide intermediate and valid root
-certificates with the ``--certificate-intermediates`` and
-``--certificate-roots`` flags:
+To verify a container that was signed with a PEM private key, using an X.509
+certificate, pass the certificate to the ``verify`` command using the
+``--certificate`` flag. If the certificate is part of a chain, provide
+intermediate and valid root certificates with the
+``--certificate-intermediates`` and ``--certificate-roots`` flags:
 
 .. code::
 
@@ -495,7 +497,7 @@ certificates with the ``--certificate-intermediates`` and
       --certificate-roots $CERTD/root.pem \
       lolcow.sif 
 
-.. note:: 
+.. note::
 
    The certificate must have a usage field that allows code signing in order to
    verify container images.
@@ -507,22 +509,22 @@ When verifying a container using X.509 certificates, {Project} can perform
 online revocation checks using the Online Certificate Status Protocol (OCSP). To
 enable OCSP checks, add the ``--ocsp-verify`` flag to your ``verify`` command:
 
-.. code:: 
+.. code::
 
    $ {command} verify \
       --certificate $CERTD/leaf.pem \
       --certificate-intermediates $CERTD/intermediate.pem \
       --certificate-roots $CERTD/root.pem \
       --ocsp-verify
-      lolcow.sif 
+      lolcow.sif
 
 {Project} will then attempt to contact the prescribed OCSP responder for
 each certificate in the chain, in order to check that the relevant certificate
 has not been revoked. In the event that an OCSP responder cannot be contacted,
 or a certificate has been revoked, verification will fail with a validation
 error:
 
-.. code:: 
+.. code::
 
    INFO:    Validate: cert:leaf  issuer:intermediate
    FATAL:   Failed to verify container: OCSP verification has failed