Skip to content

Commit c961741

Browse files
committed
create beamline tutorial complete
1 parent d7d50a3 commit c961741

File tree

9 files changed

+490
-184
lines changed

9 files changed

+490
-184
lines changed

docs/user/explanations/cli-tools.rst

Lines changed: 0 additions & 39 deletions
This file was deleted.

docs/user/explanations/repositories.rst

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@ Source and Registry Locations
99
- Beamline Source repos: ``https://gitlab.diamond.ac.uk/controls/containers/beamline/<beamline>``
1010
- Accelerator Source repos: ``https://gitlab.diamond.ac.uk/controls/containers/accelerator/<domain>``
1111
- Generic IOC Container Images: ``ghcr.io/epics-containers/<Generic ioc>``
12-
- IOC Instance Helm Charts: ``helm-test.diamond.ac.uk/iocs/<domain>/<ioc instance>``
12+
- epics-containers Helm Charts: ``https://github.com/orgs/epics-containers/packages?repo_name=ec-helm-charts``
1313

1414
Where to Keep Source Code
1515
-------------------------
@@ -64,13 +64,13 @@ kept in a public repository as it is a live example of a domain repo.
6464
Where to put Registries
6565
-----------------------
6666

67-
Generic IOC Container Images
68-
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
67+
Generic IOC Container Images and Source Repos
68+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
6969

70-
TODO: usually GHCR but internal supported for license e.g. Nexus Repository Manager
70+
Usually GHCR but internal supported for license e.g. Nexus Repository Manager
7171

72-
IOC Instance Helm Charts
73-
~~~~~~~~~~~~~~~~~~~~~~~~
72+
IOC Instance Domain Repos
73+
~~~~~~~~~~~~~~~~~~~~~~~~~
7474

75-
TODO: Can be GHCR but internal supported supported e.g. Nexus Repository Manager
75+
Internal git registry or private GitHub registry.
7676

docs/user/index.rst

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,7 @@ side-bar.
1818
tutorials/create_beamline
1919
tutorials/deploy_example
2020
tutorials/ioc_changes
21+
tutorials/release_beamline
2122
tutorials/generic_ioc
2223
tutorials/debug_generic_ioc
2324
tutorials/test_generic_ioc
@@ -58,7 +59,6 @@ side-bar.
5859
explanations/kubernetes_cluster
5960
explanations/docs-structure
6061
explanations/repositories
61-
explanations/cli-tools
6262
explanations/repos
6363

6464
+++
@@ -72,6 +72,7 @@ side-bar.
7272
:maxdepth: 1
7373

7474
reference/faq
75+
reference/environment
7576
reference/cli
7677
reference/ioc_helm_chart
7778
reference/k8s_resources

docs/user/reference/cli.rst

Lines changed: 67 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -3,19 +3,74 @@
33
Command Line Interface for IOC Management
44
=========================================
55

6-
Built in to the devcontainer is epics-containers-cli, a command line interface
7-
for assisting with building and managing IOCs in the cluster.
6+
The python project ``epics-containers-cli`` is installed as part of the
7+
Python section of the initial tutorial `../tutorials/setup_workstation`.
8+
It provides a command line function ``ec`` with support for:
89

9-
The CLI is just a thin wrapper around these tools:-
10+
- building and debugging Generic IOC Containers
11+
- deploying, managing and debugging IOC Instances
1012

11-
- kubectl
12-
- helm
13-
- podman (or docker)
13+
The CLI is just a thin wrapper around the underlying tools that do the real
14+
work:
1415

15-
To help with learning the above tools. The CLI will print out the commands it
16-
executes before running them. This can be suppressed with the ``--quiet`` flag,
17-
or by setting the ``K8S_QUIET=true`` in your environment.
16+
:kubectl: the command line interface to the Kubernetes APIs
17+
:helm: the command line interface to the Kubernetes Helm package manager
18+
:podman (or docker): CLIs for container engines
19+
:git: the git version control system client
1820

19-
The CLI entrypoint is ``ec``. To see the available commands, run ``ec --help``.
20-
Much of the functionality is available through subcommands dev and ioc.
21-
Below is pasted the current version of help for the CLI.
21+
``ec`` is useful because it saves typing and provides a consistent interface
22+
when working on multiple beamlines. This is because it uses the environment
23+
setup by the beamline repo's ``environment.sh`` script. See `environment`.
24+
25+
To see the available commands, run ``ec --help``.
26+
Much of the functionality is available through subcommands dev and ioc. To get
27+
full details of a given command type the full command plus ``--help``. For
28+
example:
29+
30+
.. raw:: html
31+
32+
<pre>$ ec dev launch --help
33+
<b> </b>
34+
<b> </b><font color="#A2734C"><b>Usage: </b></font><b>ec dev launch [OPTIONS] IOC_INSTANCE </b>
35+
<b> </b>
36+
Launch an IOC instance using configuration from a domain repo. Or by passing a
37+
generic IOC image ID. Can be used for local testing of IOC instances. You may
38+
find the devcontainer a more convenient way to do this.
39+
40+
<font color="#AAAAAA">╭─ Arguments ───────────────────────────────────────────────────────────────────╮</font>
41+
<font color="#AAAAAA">│ </font><font color="#C01C28">*</font> ioc_instance <font color="#A2734C"><b>DIRECTORY</b></font> local IOC definition folder from domain │
42+
<font color="#AAAAAA">│ repo │</font>
43+
<font color="#AAAAAA">│ [default: None] │</font>
44+
<font color="#AAAAAA">│ </font><font color="#80121A">[required] </font> │
45+
<font color="#AAAAAA">╰───────────────────────────────────────────────────────────────────────────────╯</font>
46+
<font color="#AAAAAA">╭─ Options ─────────────────────────────────────────────────────────────────────╮</font>
47+
<font color="#AAAAAA">│ </font><font color="#2AA1B3"><b>--execute</b></font> <font color="#A2734C"><b>TEXT </b></font> command to execute in the container. │
48+
<font color="#AAAAAA">│ Defaults to executing the IOC │</font>
49+
<font color="#AAAAAA">│ [default: /epics/ioc/start.sh; bash] │</font>
50+
<font color="#AAAAAA">│ </font><font color="#2AA1B3"><b>--target</b></font> <font color="#6C4C32"><b>[developer|runtime]</b></font> choose runtime or developer target │
51+
<font color="#AAAAAA">│ [default: developer] │</font>
52+
<font color="#AAAAAA">│ </font><font color="#2AA1B3"><b>--image</b></font> <font color="#A2734C"><b>TEXT </b></font> override container image to use │
53+
<font color="#AAAAAA">│ </font><font color="#2AA1B3"><b>--tag</b></font> <font color="#A2734C"><b>TEXT </b></font> override image tag to use. │
54+
<font color="#AAAAAA">│ [default: None] │</font>
55+
<font color="#AAAAAA">│ </font><font color="#2AA1B3"><b>--args</b></font> <font color="#A2734C"><b>TEXT </b></font> Additional args for podman/docker, │
56+
<font color="#AAAAAA">│ &apos;must be quoted&apos; │</font>
57+
<font color="#AAAAAA">│ </font><font color="#2AA1B3"><b>--ioc-name</b></font> <font color="#A2734C"><b>TEXT </b></font> container name override. Use to run │
58+
<font color="#AAAAAA">│ multiple instances │</font>
59+
<font color="#AAAAAA">│ [default: test-ioc] │</font>
60+
<font color="#AAAAAA">│ </font><font color="#2AA1B3"><b>--help</b></font> <font color="#A2734C"><b> </b></font> Show this message and exit. │
61+
<font color="#AAAAAA">╰───────────────────────────────────────────────────────────────────────────────╯</font>
62+
</pre>
63+
64+
It may be instructive to understand the underlying tools and how they are
65+
are being called. For this reason ``ec`` supports a ``-v`` option to show
66+
the underlying commands being executed. e.g.
67+
68+
.. raw:: html
69+
70+
<pre>$ ec -v ps
71+
<font color="#5F8787">kubectl get namespace p38-iocs -o name</font>
72+
<font color="#5F8787">kubectl -n p38-iocs get pod -l is_ioc==True -o custom-columns=IOC_NAME:metadata.labels.app,VERSION:metadata.labels.ioc_version,STATE:status.phase,RESTARTS:status.containerStatuses[0].restartCount,STARTED:metadata.managedFields[0].time</font>
73+
IOC_NAME VERSION STATE RESTARTS STARTED
74+
bl38p-ea-ioc-03 2023.10.6 Running 0 2023-10-25T14:07:44Z
75+
bl38p-ea-panda-02 2023.10.25-b16.24 Running 0 2023-10-25T15:24:41Z
76+
</pre>

docs/user/reference/environment.rst

Lines changed: 128 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,128 @@
1+
The Environment Configuration File
2+
==================================
3+
4+
``environment.sh`` is a configuration file that is provided in each domain
5+
(beamline or accelerator) repository. It is used to set up the environment
6+
such that the ``epics-containers-cli`` will interact with the correct
7+
cluster (or local server if you are not using Kubernetes).
8+
9+
An important part of creating a new domain repository is to edit the
10+
``environment.sh`` so that it suits the domain you are targetting.
11+
12+
There are 3 sections to the file as follows:
13+
14+
Environment Variables Setup
15+
---------------------------
16+
17+
The first section defines a number of environment variables. These should
18+
be adjusted to suit your domain. The variables are as follows:
19+
20+
21+
Required Variables
22+
~~~~~~~~~~~~~~~~~~
23+
24+
- **EC_REGISTRY_MAPPING**: defines a mapping between git source repositories and
25+
container repositories. This is used to determine where a Generic IOC
26+
container image is pushed to (or pulled from). If you are using GitHub then
27+
you could use the following value:
28+
29+
- ``EC_REGISTRY_MAPPING='github.com=ghcr.io'``
30+
31+
Which means containers whose source repository is at:
32+
33+
- ``GitHub/organisation/my_generic_ioc``
34+
35+
will be pushed to:
36+
37+
- ``ghcr.io/organisation/my_generic_ioc:VERSION``.
38+
39+
You can have multiple mappings if needed by separating them with a space.
40+
41+
- **EC_NAMESPACE**: defines the namespace in a Kubernetes Cluster that your IOC
42+
Instances will be deployed to. When you come to set up a cluster you will
43+
need to create a namespace for your domain. This is the name you should
44+
use here. If you are not using Kubernetes then you can leave this as
45+
``EC_NAMESPACE=`` and this will deploy IOC Instances to the local server's
46+
docker or podman instance.
47+
48+
- **EC_DOMAIN_REPO**: this is a link back to the repository that defines this
49+
domain. For example the bl38p reference beamline repository uses
50+
``[email protected]:epics-containers/bl38p.git``. This variable
51+
is used by ``ec`` to fetch versioned IOC Instance from the repo and deploy
52+
time.
53+
54+
55+
Optional Variables
56+
~~~~~~~~~~~~~~~~~~
57+
58+
- **EC_LOG_URL**: if you have a centralized logging service with a web UI then
59+
you can set this variable to the URL of the web UI. This will then be
60+
displayed when the command ``ec ioc log-history <ioc-name>`` is run. The
61+
ioc name is added to the URL using ``{ioc-name}`` as a placeholder e.g.
62+
63+
- ``EC_LOG_URL='https://graylog2.diamond.ac.uk/search?rangetype=relative&fields``
64+
``=message%2Csource&width=1489&highlightMessage=&relative=172800&q=pod_name%3A``
65+
``{ioc_name}*'``
66+
67+
- **EC_CONTAINER_CLI**: this sets the name of the container CLI to use. supported
68+
options are ``podman`` or ``docker``. If not set then ``ec`` will try to
69+
determine which one to use. You would only need this variable if you have
70+
both podman and docker installed and you want to use one over the other, or
71+
if you want to use a different container CLI such as ``singularity``.
72+
IMPORTANT: the application you reference must have docker compatible CLI
73+
(at least for common functions).
74+
75+
- **EC_DEBUG**: causes the ``ec`` command to output debug information for all
76+
commands. For more targetted debugging you can use ``ec -d ...``.
77+
78+
Installation of ``ec``
79+
----------------------
80+
81+
The second section of the ``environment.sh`` file is used to install the
82+
``ec`` command by pip installing the ``epics-containers-cli`` package. The
83+
``blxxi-template`` project comes with a suggested way of doing this, but
84+
it would probably be best to have ``ec`` installed globally on your
85+
workstation and then omit this section from your ``environment.sh`` files.
86+
87+
Perhaps the simplest way to achieve this is to install ``ec`` into your user
88+
space using the following command:
89+
90+
.. code:: bash
91+
92+
pip install --user epics-containers-cli
93+
94+
Then add the following to your ``.bashrc`` file:
95+
96+
.. code:: bash
97+
98+
export PATH=$PATH:$HOME/.local/bin
99+
100+
Connecting to a Namespace on your Kubernetes Cluster
101+
----------------------------------------------------
102+
103+
The third section of the ``environment.sh`` sets up how the ``kubectl`` command
104+
will connect to a namespace on your Kubernetes cluster. This usually involves
105+
setting the ``KUBECONFIG`` environment variable to point to a file that contains
106+
the cluster configuration.
107+
108+
When we set up a cluster in the tutorials we will create a namespace for your
109+
and discuss how to update ``environment.sh`` to connect to it.
110+
111+
If you are connecting to your own facility's cluster then you will need to
112+
ask your admins for the correct configuration.
113+
114+
If you are not using Kubernetes then you can leave this section out.
115+
116+
.. note::
117+
118+
DLS users: the module system is used to connect us to each beamline/accelerator
119+
cluster. The example ``environment.sh`` file in ``bl38p`` shows how to do this
120+
see https://github.com/epics-containers/bl38p/blob/main/environment.sh
121+
122+
One other thing that is useful is to set up command line completion for the
123+
Kubernetes tools ``kubectl`` and ``helm``. See the end of the bl38p
124+
``environment.sh`` file for an example of how to do this at
125+
https://github.com/epics-containers/bl38p/blob/main/environment.sh.
126+
127+
128+

0 commit comments

Comments
 (0)