Improvement documentation for using Object Storage containers

* Add guidelines for naming containers and objects. Generally there are two standards offered to customers:
    * A restrictive naming guideline (mostly based on the S3 Bucket Naming Rules), that ensures the containers will be compatible with all current and future APIs and features.
    * The (less restrictive) specific requirements for container naming, based on the client they are planning to use, if the customer requires more naming flexibility at the expense of compatibility.
* Add notes about our plans to introduce subdomain-based container URLs, and virtual-host-style requests for S3 API.
* Rewrite the section on using Object Storage via the CLI, with more detail, nicer formatting, and ease of navigation via anchors.
* Add documentation for using the Swift Client to manage containers.
* Some other related small fixes and improvements.

source/identity-access-management/roles.rst

@@ -1,7 +1,7 @@
 .. _access_control:
 
 ##############
-Access control
+Access Control
 ##############
 
 .. _project_users:

source/object-storage/cli/aws-cli/container-create.rst

@@ -0,0 +1,23 @@
+.. note::
+
+  It is **not recommended** to create an Object Storage container
+  using the AWS CLI, as the :ref:`storage policy <object-storage-storage-policies>`
+  for the container cannot be set.
+
+  Containers created using the AWS CLI will always use the default storage policy
+  (``nz--o1--mr-r3``).
+
+The preferred method to create a container using the AWS CLI
+is to use the ``aws s3api create-bucket`` command,
+specifying the container name with ``--bucket``.
+
+.. code-block:: bash
+
+  aws s3api create-bucket --bucket mycontainer-1
+
+``aws s3 mb`` can also be used,
+specifying the container as a bucket name using an ``s3://`` URL.
+
+.. code-block:: bash
+
+  aws s3 mb s3://mycontainer-1

source/object-storage/cli/aws-cli/container-delete.rst

@@ -0,0 +1,23 @@
+The preferred method is to use the ``aws s3api delete-bucket`` command,
+specifying the container name with ``--bucket``.
+This command can only be used to delete empty containers
+(all objects must be deleted before the container itself can be deleted),
+so is a safer option to ensure data is not accidentally deleted.
+
+.. code-block:: bash
+
+  aws s3api delete-bucket --bucket mycontainer-1
+
+An alternative is to use the ``aws s3 rb`` command,
+specifying the container as a bucket name using an ``s3://`` URL.
+
+.. code-block:: bash
+
+  aws s3 rb s3://mycontainer-1
+
+Unlike ``aws s3api delete-bucket``, this command can be used to delete non-empty containers
+using the ``--force`` option.
+
+.. code-block:: bash
+
+  aws s3 rb s3://mycontainer-1 --force

source/object-storage/cli/aws-cli/container-list.rst

@@ -0,0 +1,24 @@
+Use either the ``aws s3 ls`` or the ``aws s3api list-buckets`` commands to return a list of all containers.
+
+.. code-block:: console
+
+  $ aws s3 ls
+  2009-02-04 05:45:09 mycontainer-1
+  2009-02-04 05:45:09 mycontainer-2
+  $ aws s3api list-buckets
+  {
+      "Buckets": [
+          {
+              "Name": "mycontainer-1",
+              "CreationDate": "2009-02-03T16:45:09+00:00"
+          },
+          {
+              "Name": "mycontainer-2",
+              "CreationDate": "2009-02-03T16:45:09+00:00"
+          }
+      ],
+      "Owner": {
+          "DisplayName": "example.com:[email protected]",
+          "ID": "example.com:[email protected]"
+      }
+  }

source/object-storage/cli/aws-cli/container-show.rst

@@ -0,0 +1,3 @@
+Object Storage container details are not available using the AWS CLI.
+
+To get container details on the command line, use the Catalyst Cloud Client.

source/object-storage/cli/aws-cli/object-create.rst

@@ -0,0 +1,20 @@
+The ``aws s3 cp`` command can be used to upload an object to a container,
+by specifying the local source path as the first argument
+and the destination ``s3://`` URL for the object as the second argument.
+
+.. code-block:: console
+
+  $ aws s3 cp file-1.txt s3://mycontainer-1/file-1.txt
+  upload: ./file-1.txt to s3://mycontainer-1/file-1.txt
+
+Alternatively ``aws s3api put-object`` can be used as shown below,
+using ``--body`` to specify the source file path,
+``--bucket`` to specify the container name
+and ``--key`` to specify the unique object name.
+
+.. code-block:: console
+
+  $ aws s3api put-object --body file-1.txt --bucket mycontainer-1 --key file-1.txt
+  {
+      "ETag": "\"746308829575e17c3331bbcb00c0898b\""
+  }

source/object-storage/cli/aws-cli/object-delete.rst

@@ -0,0 +1,15 @@
+The ``aws s3 rm`` command can be used to delete an object from a container,
+specifying the ``s3://`` URL for the object.
+
+.. code-block:: console
+
+  $ aws s3 rm s3://mycontainer-1/file-1.txt
+  delete: s3://mycontainer-1/file-1.txt
+
+Alternatively ``aws s3api delete-object`` can be used as shown below,
+using ``--bucket`` to specify the container name
+and ``--key`` to specify the unique object name.
+
+.. code-block:: bash
+
+  aws s3api delete-object --bucket mycontainer-1 --key file-1.txt

source/object-storage/cli/aws-cli/object-list.rst

@@ -0,0 +1,57 @@
+A list of objects can be returned using the ``aws s3 ls`` command,
+specifying the container as a bucket name using an ``s3://`` URL.
+
+.. code-block:: console
+
+  $ aws s3 ls s3://mycontainer-1
+                             PRE images/
+  2024-08-16 14:22:25         14 file-1.txt
+  2023-08-25 11:31:27    1908226 image-1.png
+
+Just like on AWS, objects with directory-style prefixes (e.g. ``images/image-2.png``)
+are not listed by default. Use the ``--recursive`` option to recursively list all objects
+in the container.
+
+.. code-block:: console
+
+  $ aws s3 ls s3://mycontainer-1
+  2024-08-16 14:22:25         14 file-1.txt
+  2023-08-25 11:31:27    1908226 image-1.png
+  2024-08-16 15:55:09     723424 images/image-2.png
+
+For more information on how to use the ``aws s3 ls`` command, see the `AWS CLI documentation <aws s3 ls>`_.
+
+To get the list of objects in a format suitable for ingestion
+by a script, use the ``aws s3api list-objects-v2`` command.
+
+.. code-block:: console
+
+  $ aws s3api list-objects-v2 --bucket mycontainer-1
+  {
+      "Contents": [
+          {
+              "Key": "file-1.txt",
+              "LastModified": "2024-08-16T02:22:25.304000+00:00",
+              "ETag": "\"746308829575e17c3331bbcb00c0898b\"",
+              "Size": 14,
+              "StorageClass": "STANDARD"
+          },
+          {
+              "Key": "image-1.png",
+              "LastModified": "2024-08-16T02:22:28.010000+00:00",
+              "ETag": "\"90cef9cada92ce2cf05cbde9499afbdb\"",
+              "Size": 1908226,
+              "StorageClass": "STANDARD"
+          },
+          {
+              "Key": "images/image-2.png",
+              "LastModified": "2024-08-16T03:55:09.486000+00:00",
+              "ETag": "\"746308829575e17c3331bbcb00c0898b\"",
+              "Size": 723424,
+              "StorageClass": "STANDARD"
+          }
+      ],
+      "RequestCharged": null
+  }
+
+.. _`aws s3 ls`: https://awscli.amazonaws.com/v2/documentation/api/latest/reference/s3/ls.html

source/object-storage/cli/aws-cli/object-save.rst

@@ -0,0 +1,30 @@
+The ``aws s3 cp`` command can be used to download an object from a container,
+by specifying the source ``s3://`` URL for the object as the first argument
+and the local destination path as the second argument.
+
+.. code-block:: console
+
+  $ aws s3 cp s3://mycontainer-1/file-1.txt file-1.txt
+  download: s3://mycontainer-1/file-1.txt to ./file-1.txt
+
+The file will be saved to the target path.
+
+.. code-block:: console
+
+  $ cat file-1.txt
+  Hello, world!
+
+Alternatively ``aws s3api get-object`` can be used as shown below,
+using ``--bucket`` to specify the container name
+and ``--key`` to specify the unique object name.
+
+.. code-block:: console
+
+  $ aws s3api get-object --bucket mycontainer-1 --key file-1.txt file-1.txt
+  {
+      "LastModified": "2024-08-16T04:28:01+00:00",
+      "ContentLength": 14,
+      "ETag": "\"746308829575e17c3331bbcb00c0898b\"",
+      "ContentType": "binary/octet-stream",
+      "Metadata": {}
+  }

source/object-storage/cli/aws-cli/object-show.rst

@@ -0,0 +1,16 @@
+Some of the object metadata is available from the S3 CLI.
+
+To get this metadata run the ``aws s3api head-object`` command,
+using ``--bucket`` to specify the container name
+and ``--key`` to specify the unique object name.
+
+.. code-block:: console
+
+  $ aws s3api head-object --bucket mycontainer-1 --key file-1.txt
+  {
+      "LastModified": "2024-08-16T02:22:26+00:00",
+      "ContentLength": 14,
+      "ETag": "\"746308829575e17c3331bbcb00c0898b\"",
+      "ContentType": "text/plain",
+      "Metadata": {}
+  }

source/object-storage/cli/aws-cli/prerequisites.rst

@@ -0,0 +1,76 @@
+Once you have downloaded and installed the AWS CLI
+(see `AWS CLI - Getting Started <https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html>`_ for more information),
+you will need to use the Catalyst Cloud Client to create the credentials for the AWS CLI.
+
+First, run ``openstack ec2 credentials create`` to create an EC2 credential.
+
+.. code-block:: console
+
+  $ openstack ec2 credentials create
+  +-----------------+------------------------------------------------------------------------------------------------------------------------------------------------------+
+  | Field           | Value                                                                                                                                                |
+  +-----------------+------------------------------------------------------------------------------------------------------------------------------------------------------+
+  | access          | ee55dd44cc33bb2211aaee55dd44cc33                                                                                                                     |
+  | access_token_id | None                                                                                                                                                 |
+  | app_cred_id     | None                                                                                                                                                 |
+  | links           | {'self': 'https://api.nz-por-1.catalystcloud.io:5000/v3/users/1a2b3c4d5e1a2b3c4d5e1a2b3c4d5e1a/credentials/OS-EC2/e5d4c3b2a1e5d4c3b2a1e5d4c3b2a1e5'} |
+  | project_id      | 1a2b3c4d5e1a2b3c4d5e1a2b3c4d5e1a                                                                                                                     |
+  | secret          | 11aa22bb33cc44dd55ee11aa22bb33cc                                                                                                                     |
+  | trust_id        | None                                                                                                                                                 |
+  | user_id         | e5d4c3b2a1e5d4c3b2a1e5d4c3b2a1e5                                                                                                                     |
+  +-----------------+------------------------------------------------------------------------------------------------------------------------------------------------------+
+
+If you have an existing EC2 credential you'd like to use instead,
+run ``openstack ec2 credentials list`` to fetch the access key ID and secret access key for the credendial.
+
+.. code-block:: console
+
+  $ openstack ec2 credentials list
+  +----------------------------------+----------------------------------+----------------------------------+----------------------------------+
+  | Access                           | Secret                           | Project ID                       | User ID                          |
+  +----------------------------------+----------------------------------+----------------------------------+----------------------------------+
+  | ee55dd44cc33bb2211aaee55dd44cc33 | 11aa22bb33cc44dd55ee11aa22bb33cc | 1a2b3c4d5e1a2b3c4d5e1a2b3c4d5e1a | e5d4c3b2a1e5d4c3b2a1e5d4c3b2a1e5 |
+  +----------------------------------+----------------------------------+----------------------------------+----------------------------------+
+
+Take the values for the ``access`` and ``secret`` fields, and use them to define the following environment variables
+in your terminal session.
+
+.. tabs::
+
+  .. group-tab:: Linux / macOS
+
+    .. code-block:: bash
+
+      export AWS_ACCESS_KEY_ID=${access}
+      export AWS_SECRET_ACCESS_KEY=${secret}
+
+  .. group-tab:: Windows
+
+    .. code-block:: powershell
+
+      $Env:AWS_ACCESS_KEY_ID = "${access}"
+      $Env:AWS_SECRET_ACCESS_KEY = "${secret}"
+
+In addition, you will need to set the ``AWS_ENDPOINT_URL`` environment variable to tell the AWS CLI
+the location of the Catalyst Cloud Object Storage S3 API.
+
+.. include:: tutorial-scripts/endpoint-urls.rst
+
+Once you have determined the correct endpoint URL to use, set the ``AWS_ENDPOINT_URL``
+environment variable in your terminal session using the following command.
+
+.. tabs::
+
+  .. group-tab:: Linux / macOS
+
+    .. code-block:: bash
+
+      export AWS_ENDPOINT_URL=${endpoint_url}
+
+  .. group-tab:: Windows
+
+    .. code-block:: powershell
+
+      $Env:AWS_ENDPOINT_URL = "${endpoint_url}"
+
+At this point, you should now be able to use the AWS CLI to interact with Catalyst Cloud Object Storage.

source/object-storage/cli/catalystcloud-client/container-create.rst

@@ -0,0 +1,30 @@
+To create a new Object Storage container, run ``openstack container create``
+followed by the unique name for the new container.
+
+.. code-block:: console
+
+  $ openstack container create mycontainer-1
+  +---------------------------------------+---------------+------------------------------------+
+  | account                               | container     | x-trans-id                         |
+  +---------------------------------------+---------------+------------------------------------+
+  | AUTH_1a2b3c4d5e1a2b3c4d5e1a2b3c4d5e1a | mycontainer-1 | tx415f9620d24e4ab28f01f-0066beb32f |
+  +---------------------------------------+---------------+------------------------------------+
+
+If you'd like to create the container using a specific
+:ref:`storage policy <object-storage-storage-policies>`, use the ``--storage-policy`` option
+followed by the storage policy name.
+
+.. code-block:: console
+
+  $ openstack container create mycontainer-2 --storage-policy nz-hlz-1--o1--sr-r3
+  +---------------------------------------+---------------+------------------------------------+
+  | account                               | container     | x-trans-id                         |
+  +---------------------------------------+---------------+------------------------------------+
+  | AUTH_1a2b3c4d5e1a2b3c4d5e1a2b3c4d5e1a | mycontainer-2 | tx580e7f6582674260b9261-0066beb4a6 |
+  +---------------------------------------+---------------+------------------------------------+
+
+Other useful options:
+
+* ``--public`` - Make the container publicly accessible.
+  **Be careful when using this option, as no authentication
+  will be required to access the container's objects.**

source/object-storage/cli/catalystcloud-client/container-delete.rst

@@ -0,0 +1,14 @@
+To delete one or more containers, use the ``openstack container delete`` command,
+followed by the names of the containers to delete.
+
+.. code-block:: bash
+
+  openstack container delete mycontainer-1
+
+By default, only containers that contain no objects can be deleted.
+To delete the containers **and all of the objects contained within them**,
+use the ``--recursive`` option.
+
+.. code-block:: bash
+
+  openstack container delete mycontainer-1 --recursive

source/object-storage/cli/catalystcloud-client/container-list.rst

@@ -0,0 +1,7 @@
+Use the ``openstack container list`` command to return a list of all containers.
+
+.. code-block:: console
+
+  $ openstack container list
+  mycontainer-1
+  mycontainer-2

source/object-storage/cli/catalystcloud-client/container-show.rst

@@ -0,0 +1,15 @@
+To view the details for a specific container,
+use ``openstack container show`` followed by the container name.
+
+.. code-block:: console
+
+  $ openstack container show mycontainer-1
+  +----------------+---------------------------------------+
+  | Field          | Value                                 |
+  +----------------+---------------------------------------+
+  | account        | AUTH_1a2b3c4d5e1a2b3c4d5e1a2b3c4d5e1a |
+  | bytes_used     | 1908226                               |
+  | container      | mycontainer-1                         |
+  | object_count   | 1                                     |
+  | storage_policy | nz--o1--mr-r3                         |
+  +----------------+---------------------------------------+