Update docs workflows to use CLI

fixes: #107

CHANGES/107.doc

@@ -0,0 +1 @@
+Added CLI commands to documented workflows.

docs/_scripts/add_content_repo.sh

@@ -0,0 +1,5 @@
+# Add created Gem content to repository
+pulp gem repository content add --repository foo --href $CONTENT_HREF
+
+# After the task is complete, it gives us a new repository version
+pulp gem repository version show --repository foo

docs/_scripts/artifact.sh

@@ -0,0 +1,7 @@
+# Get a Gem package or choose your own
+curl -O https://fixtures.pulpproject.org/gem/gems/amber-1.0.0.gem
+
+# Upload it to Pulp
+pulp gem content upload --file "amber-1.0.0.gem"
+
+CONTENT_HREF=$(pulp gem content list | jq -r .[0].pulp_href)

docs/_scripts/distribution.sh

@@ -0,0 +1,3 @@
+# Distributions are created asynchronously. Create one, and specify the publication that will
+# be served at the base path specified.
+pulp gem distribution create --name foo --base-path foo --publication "$PUBLICATION_HREF"

docs/_scripts/publication.sh

@@ -0,0 +1,6 @@
+# Create a new publication specifying the repository_version.
+# Alternatively, you can specify just the repository, and Pulp will assume the latest version.
+pulp gem publication create --repository foo --version 1
+
+# Publications can only be referenced through their pulp_href
+PUBLICATION_HREF=$(pulp gem publication list | jq -r .[0].pulp_href)

docs/_scripts/pullthrough.sh

@@ -0,0 +1,2 @@
+# Add remote to distribution to enable pull-through caching
+pulp gem distribution update --name foo --remote gem

docs/_scripts/remote.sh

@@ -0,0 +1,2 @@
+# Create a remote that syncs all versions of panda into your repository.
+pulp gem remote create --name gem --url https://index.rubygems.org/ --includes '{"panda":null}'

docs/_scripts/repo.sh

@@ -0,0 +1,2 @@
+# Start by creating a new repository named "foo":
+pulp gem repository create --name foo

docs/_scripts/sync.sh

@@ -0,0 +1,8 @@
+# Using the Remote we just created, we kick off a sync task
+pulp gem repository sync --name foo --remote gem
+
+# The sync command will by default wait for the sync to complete
+# Use Ctrl+c or the -b option to send the task to the background
+
+# Show the latest version when sync is done
+pulp gem repository version show --repository foo

docs/installation.rst

@@ -1,43 +1,32 @@
 User Setup
 ==========
 
-All REST API examples below use `httpie <https://httpie.org/doc>`__ to
-perform the requests.
+All workflow examples use the Pulp CLI. Install and setup from PyPI:
 
 .. code-block:: bash
 
-    machine localhost
-    login admin
-    password admin
+    pip install pulp-cli[pygments] pulp-cli-gem
+    pulp config create -e
+    pulp status # Check that CLI can talk to Pulp
 
 If you configured the ``admin`` user with a different password, adjust the configuration
 accordingly. If you prefer to specify the username and password with each request, please see
-``httpie`` documentation on how to do that.
+``Pulp CLI`` documentation on how to do that.
 
 
-Install ``pulpcore``
---------------------
+Containerized Installation
+**************************
 
-Follow the `installation
-instructions <docs.pulpproject.org/en/3.0/nightly/installation/instructions.html>`__
-provided with pulpcore.
-
-Install plugin
---------------
-
-This document assumes that you have
-`installed pulpcore <https://docs.pulpproject.org/en/3.0/nightly/installation/instructions.html>`_
-into a the virtual environment ``pulpvenv``.
-
-Users should install from **either** PyPI or source.
+Follow the `Pulp in One Container <https://pulpproject.org/pulp-in-one-container/>`_ instructions to get started with Pulp by
+leveraging OCI images. Further details are discussed in the `pulpcore documentation <https://docs.pulpproject.org/pulpcore/installation/instructions.html>`_.
 
 From Source
 ***********
 
 .. code-block:: bash
 
    sudo -u pulp -i
-   source ~/pulpvenv/bin/activate
+   source ~/pulp/bin/activate
    cd pulp_gem
    pip install -e .
    django-admin runserver 24817

docs/workflows/index.rst

@@ -5,25 +5,18 @@ If you have not yet installed the `gem` plugin on your Pulp installation, please
 :doc:`../installation`. These documents will assume you have the environment installed and
 ready to go.
 
-The REST API examples here use `httpie <https://httpie.org/doc>`_ to perform the requests.
-The ``httpie`` commands below assume that the user executing the commands has a ``.netrc`` file
-in the home directory. The ``.netrc`` should have the following configuration:
+The example workflows here use the Pulp CLI. Get and setup the Pulp CLI from PyPI with the following
+commands. For more information about setting up the Pulp CLI please read the `installation and configuration
+doc page <https://github.com/pulp/pulp-cli/blob/develop/docs/install.md>`_.
 
 .. code-block:: bash
 
-    machine localhost
-    login admin
-    password admin
+    pip install pulp-cli-gem
+    pulp config create -e
 
 If you configured the ``admin`` user with a different password, adjust the configuration
 accordingly. If you prefer to specify the username and password with each request, please see
-``httpie`` documentation on how to do that.
-
-To make these workflows copy/pastable, we make use of environment variables. The first variable to
-set is the hostname and port::
-
-   $ export BASE_ADDR=http://<hostname>:24817
-
+``Pulp CLI`` documentation on how to do that.
 
 .. toctree::
    :maxdepth: 2

docs/workflows/publish.rst

@@ -7,30 +7,74 @@ This section assumes that you have a repository with content in it. To do this,
 Create a Publication
 --------------------
 
-Publications contain extra settings for how to publish.::
+Kick off a publish task by creating a new publication. The publish task will generate all the
+metadata that ``gem`` needs to install packages (although it will need to be hosted through a
+Distribution before it is consumable).
 
-$ http POST $BASE_ADDR/pulp/api/v3/publications/gem/gem/ name=bar
+.. literalinclude:: ../_scripts/publication.sh
+    :language: bash
 
-Response::
+Publication Create Response::
 
     {
-        "pulp_href": "http://localhost:24817/pulp/api/v3/publications/gem/gem/bar/",
-        ...
+      "pulp_href": "/pulp/api/v3/publications/gem/gem/0188e510-cbbe-7586-b182-927bacc89bdb/",
+      "pulp_created": "2023-06-22T21:44:23.230932Z",
+      "repository_version": "/pulp/api/v3/repositories/gem/gem/0188e4e3-1429-7411-89d7-c87288edf51a/versions/1/",
+      "repository": "/pulp/api/v3/repositories/gem/gem/0188e4e3-1429-7411-89d7-c87288edf51a/"
     }
 
-
 Host a Publication (Create a Distribution)
 --------------------------------------------
 
-To host a publication, (which makes it consumable by a package manager), users create a distribution which
-will serve the associated publication at ``/pulp/content/<distribution.base_path>``::
+To host a publication, (which makes it consumable by ``gem``), users create a distribution which
+will serve the associated publication at ``/pulp/content/<distribution.base_path>``
 
-$ http POST $BASE_ADDR/pulp/api/v3/distributions/gem/gem/ name='baz' base_path='foo' publication=$BASE_ADDR/publications/5fcb3a98-1bd1-445f-af94-801a1d563b9f/
+.. literalinclude:: ../_scripts/distribution.sh
+    :language: bash
 
 Response::
 
     {
-        "pulp_href": "http://localhost:24817/pulp/api/v3/distributions/2ac41454-931c-41c7-89eb-a9d11e19b02a/",
-       ...
+      "pulp_href": "/pulp/api/v3/distributions/gem/gem/0188e513-594c-7c44-aaff-beed0f97363e/",
+      "pulp_created": "2023-06-22T21:47:10.540670Z",
+      "base_path": "foo",
+      "base_url": "http://localhost:5001/pulp/content/foo/",
+      "content_guard": null,
+      "hidden": false,
+      "pulp_labels": {},
+      "name": "foo",
+      "repository": null,
+      "publication": "/pulp/api/v3/publications/gem/gem/0188e510-cbbe-7586-b182-927bacc89bdb/",
+      "remote": null
     }
 
+.. note::
+    Alternatively you could specify the repository when creating a distribution in which case Pulp
+    will automatically distribute the latest available publication for the greatest repository version
+
+Enable Pull-Through Caching:
+----------------------------
+
+Only gems present in your repository will be available from your index, but adding a remote source to
+your distribution will enable the pull-through cache feature. This feature allows you to install any gem
+from the remote source and have Pulp store that gem as orphaned content.
+
+.. literalinclude:: ../_scripts/pullthrough.sh
+    :language: bash
+
+.. warning::
+    Support for pull-through caching is provided as a tech preview in Pulp 3.
+    Functionality may not work or may be incomplete. Also, backwards compatibility when upgrading
+    is not guaranteed.
+
+Use the newly created distribution
+-----------------------------------
+
+The metadata and packages can now be retrieved from the distribution::
+
+    $ http $BASE_ADDR/pulp/content/foo/specs.4.8
+    $ http $BASE_ADDR/pulp/content/foo/gems/panda-1.1.0.gem
+
+The content is now gem installable::
+
+    $ gem install --clear-sources --source $BASE_ADDR/pulp/content/foo/ panda

docs/workflows/sync.rst

@@ -7,82 +7,102 @@ their repository.
 Create a Repository
 -------------------
 
-Start by creating a new repository named "foo"::
+Start by creating a new repository named "foo"
 
-    $ http POST $BASE_ADDR/pulp/api/v3/repositories/gem/gem/ name=foo
+.. literalinclude:: ../_scripts/repo.sh
+   :language: bash
 
 Response::
 
     {
-        "pulp_href": "http://localhost:24817/pulp/api/v3/repositories/gem/gem/9b19ceb7-11e1-4309-9f97-bcbab2ae38b6/",
-        ...
+      "pulp_href": "/pulp/api/v3/repositories/gem/gem/0188e4e3-1429-7411-89d7-c87288edf51a/",
+      "pulp_created": "2023-06-22T20:54:27.113947Z",
+      "versions_href": "/pulp/api/v3/repositories/gem/gem/0188e4e3-1429-7411-89d7-c87288edf51a/versions/",
+      "pulp_labels": {},
+      "latest_version_href": "/pulp/api/v3/repositories/gem/gem/0188e4e3-1429-7411-89d7-c87288edf51a/versions/0/",
+      "name": "foo",
+      "description": null,
+      "retain_repo_versions": null,
+      "remote": null
     }
 
-
 Create a Remote
 ---------------
 
 Creating a remote object informs Pulp about an external content source.
 
-``$ http POST $BASE_ADDR/pulp/pulp/api/v3/remotes/gem/gem/ name='bar' url='http://some.url/somewhere/'``
+.. literalinclude:: ../_scripts/remote.sh
+   :language: bash
 
 Response::
 
-    {
-        "pulp_href": "http://localhost:24817/pulp/pulp/api/v3/remotes/gem/gem/ 9c757d65-3007-4884-ac5b-c2fd93873289/",
-        ...
-    }
-
+  {
+      "pulp_href": "/pulp/api/v3/remotes/gem/gem/0188e505-157c-7565-8474-e607e0dbc4a0/",
+      "pulp_created": "2023-06-22T21:31:35.676442Z",
+      "name": "gem",
+      "url": "https://index.rubygems.org",
+      "ca_cert": null,
+      "client_cert": null,
+      "tls_validation": true,
+      "proxy_url": null,
+      "pulp_labels": {},
+      "pulp_last_updated": "2023-06-22T21:31:35.676454Z",
+      "download_concurrency": null,
+      "max_retries": null,
+      "policy": "immediate",
+      "total_timeout": null,
+      "connect_timeout": null,
+      "sock_connect_timeout": null,
+      "sock_read_timeout": null,
+      "headers": null,
+      "rate_limit": null,
+      "hidden_fields": [...],
+      "prereleases": false,
+      "includes": {
+        "panda": null
+      },
+      "excludes": null
+  }
+
+.. note::
+
+    You can also not specify anything in `includes` and have Pulp try to sync all gems available on the remote.
+
+.. note::
+
+    `includes` and `excludes` fields are JSON dictionaries with the key being the gem name and the value being
+    the version specifier string, or `null` for syncing all versions.
 
 Sync repository foo with remote
 -------------------------------
 
 Use the remote object to kick off a synchronize task by specifying the repository to
-sync with. You are telling pulp to fetch content from the remote and add to the repository::
-
-    $ http POST $BASE_ADDR/pulp/api/v3/repositories/gem/gem/9b19ceb7-11e1-4309-9f97-bcbab2ae38b6/sync/' remote=$REMOTE_HREF
-
-Response::
-
-    {
-        "pulp_href": "http://localhost:24817/pulp/api/v3/tasks/3896447a-2799-4818-a3e5-df8552aeb903/",
-        "task_id": "3896447a-2799-4818-a3e5-df8552aeb903"
-    }
-
-You can follow the progress of the task with a GET request to the task href. Notice that when the
-synchroinze task completes, it creates a new version, which is specified in ``created_resources``::
+sync with. You are telling pulp to fetch content from the remote and add to the repository
 
-    $  http GET $BASE_ADDR/pulp/api/v3/tasks/3896447a-2799-4818-a3e5-df8552aeb903/
+.. literalinclude:: ../_scripts/sync.sh
+   :language: bash
 
-Response::
+Repository Version Show Response (when complete)::
 
     {
-        "pulp_href": "http://localhost:24817/pulp/api/v3/tasks/3896447a-2799-4818-a3e5-df8552aeb903/",
-        "pulp_created": "2018-05-01T17:17:46.558997Z",
-        "created_resources": [
-            "http://localhost:24817/pulp/api/v3/repositories/gem/gem/9b19ceb7-11e1-4309-9f97-bcbab2ae38b6/versions/6/"
-        ],
-        "error": null,
-        "finished_at": "2018-05-01T17:17:47.149123Z",
-        "parent": null,
-        "progress_reports": [
-            {
-                "done": 0,
-                "message": "Add Content",
-                "state": "completed",
-                "suffix": "",
-                "total": 0
-            },
-            {
-                "done": 0,
-                "message": "Remove Content",
-                "state": "completed",
-                "suffix": "",
-                "total": 0
-            }
-        ],
-        "spawned_tasks": [],
-        "started_at": "2018-05-01T17:17:46.644801Z",
-        "state": "completed",
-        "worker": "http://localhost:24817/pulp/api/v3/workers/eaffe1be-111a-421d-a127-0b8fa7077cf7/"
+      "pulp_href": "/pulp/api/v3/repositories/gem/gem/0188e4e3-1429-7411-89d7-c87288edf51a/versions/1/",
+      "pulp_created": "2023-06-22T21:40:00.488466Z",
+      "number": 1,
+      "repository": "/pulp/api/v3/repositories/gem/gem/0188e4e3-1429-7411-89d7-c87288edf51a/",
+      "base_version": null,
+      "content_summary": {
+        "added": {
+          "gem.gem": {
+            "count": 33,
+            "href": "/pulp/api/v3/content/gem/gem/?repository_version_added=/pulp/api/v3/repositories/gem/gem/0188e4e3-1429-7411-89d7-c87288edf51a/versions/1/"
+          }
+        },
+        "removed": {},
+        "present": {
+          "gem.gem": {
+            "count": 33,
+            "href": "/pulp/api/v3/content/gem/gem/?repository_version=/pulp/api/v3/repositories/gem/gem/0188e4e3-1429-7411-89d7-c87288edf51a/versions/1/"
+          }
+        }
+      }
     }