diff --git a/docs/user/explanations/ioc-source.rst b/docs/user/explanations/ioc-source.rst index 1bc1dbd7..b5fa36c2 100644 --- a/docs/user/explanations/ioc-source.rst +++ b/docs/user/explanations/ioc-source.rst @@ -3,6 +3,69 @@ Dev Container vs Runtime Container ================================== -TODO: explain how and why ioc-xxxx is mounted in dev container and what happens -to the ioc folder. +Introduction +------------ + +The dev container is where all development of IOCs and support modules will +take place. The runtime container is where the IOC will run when deployed +to a target system. + +The dev container mounts several host folders into the container to achieve +the following goals: + +- make the developer container look as similar as possible to the runtime + container +- allow the developer to make changes and recompile things without having + to rebuild the container +- make sure that all useful changes occur in the host filesystem so that + they are not lost when the container is rebuilt or deleted + +The details of which folders are mounted where in the container are +shown here: `container-layout`. + +The ioc-XXX project folder is found in the container at ``/workspaces/ioc-XXX``, +along with all of it's peers (because the parent folder is mounted +at ``/workspaces``). + + +The ioc Folder +-------------- + +The ioc folder contains the Generic IOC source code. It is typically the same +for all Generic IOCs but is included in the ioc-XXX repo in /ioc so that it can be +modified if necessary. + +At container build time this folder is copied into the container at +``/epics/generic-source/ioc`` and it is compiled so that the binaries are +available at runtime. + +In the dev container the ``/epics/generic-source`` folder has the project +folder ioc-XXX mounted over the top of it. This means: + +- the project folder ioc-XXX is mounted in two locations in the container + - ``/workspaces/ioc-XXX`` + - ``/epics/generic-source`` +- the ioc source folder ``/epics/generic-source/ioc`` is also mounted over + and now contains the source only. The compiled binaries are no longer + visible inside the dev container. + +It is for this reason that a newly created dev container needs to have the IOC +binaries re-compiled. But this is a good thing, because now any changes you +make to the IOC source code can be compiled and tested, but also those +changes are now visible on the host filesystem inside the project folder +``ioc-XXX/ioc``. This avoids loss of work. + +Finally the ``ioc`` folder is always soft linked from ``/epics/ioc`` so that +the source and binaries are always in a known location. + +Summing Up +---------- + +The above description makes things sound rather complicated. However, +you can for the most part ignore the details and just remember: + +- use ``/epics/ioc`` to compile and run the IOC. +- you are free to make changes to the above folder and recompile +- the changes you make will be visible on the host filesystem in the + original project folder. diff --git a/docs/user/index.rst b/docs/user/index.rst index 0ac9a85b..0e752863 100644 --- a/docs/user/index.rst +++ b/docs/user/index.rst @@ -75,6 +75,7 @@ side-bar. :maxdepth: 1 reference/faq + reference/troubleshooting reference/configuration reference/environment reference/cli diff --git a/docs/user/reference/troubleshooting.rst b/docs/user/reference/troubleshooting.rst new file mode 100644 index 00000000..93dba9f8 --- /dev/null +++ b/docs/user/reference/troubleshooting.rst @@ -0,0 +1,23 @@ +Troubleshooting +=============== + +Permissions issues with GitHub +------------------------------- + +Problem: in the devcontainer you see the following error: + +.. code-block:: none + + git@github.com: Permission denied (publickey). + fatal: Could not read from remote repository. + +Solution: you may need to add your github ssh key to the ssh-agent as +follows: + +.. code-block:: none + + eval "$(ssh-agent -s)" + ssh-add ~/.ssh/id_rsa + +Where ``id_rsa`` is the name of your private key file you use for connecting +to GitHub. diff --git a/docs/user/tutorials/ioc_changes2.rst b/docs/user/tutorials/ioc_changes2.rst index 7cb804f3..2551e9e9 100644 --- a/docs/user/tutorials/ioc_changes2.rst +++ b/docs/user/tutorials/ioc_changes2.rst @@ -1,10 +1,6 @@ Changing a Generic IOC ====================== -.. warning :: - - TODO: This tutorial is a work in progress. It is not yet complete. - This is a type 2 change from `ioc_change_types`. The changes that you can make in an IOC instance are limited to what @@ -24,69 +20,59 @@ Some of the reasons for doing this are: want to add support for a second device, this is allowed but you should consider the alternative of creating a new Generic IOC. If you keep your Generic IOCs simple and focused on a single device, they - will be smaller and there will be less of them. IOCs can still be - linked via CA and this is preferable to recompiling a Generic IOC + will be smaller and there will be less of them. IOCs' records can still be + linked via CA links and this is preferable to recompiling a Generic IOC for every possible combination of devices. Using Kubernetes to manage multiple small services is cleaner than having a handful of monolithic services. -This tutorial will make some changes to the generic IOC ``ioc-adsample``. -This Generic IOC is a simplified copy of ``ioc-adsimdetector`` tailored for -use in these tutorials. -For this exercise we will initially work locally inside the ``ioc-adsample`` -developer container. +This tutorial will make some changes to the generic IOC ``ioc-adsimdetector`` +that you already used in earlier tutorials. -At the end we will push the changes and see the CI build a new version of the -generic IOC container image. This allows for the demonstration of: - -- Deploying an IOC instance using a new image published by the CI -- Showing how to do a Pull Request back to the original repository. +For this exercise we will work locally inside the ``ioc-adsimdetector`` +developer container. Following tutorials will show how to fork repositories +and push changes back to GitHub For this exercise we will be using an example IOC Instance to test our changes. Instead of working with a beamline repository, we will use the example ioc instance -that comes with ``ioc-adsample``. It is a good idea for Generic IOC authors to +inside ``ioc-adsimdetector``. It is a good idea for Generic IOC authors to include an example IOC Instance in their repository for testing changes in isolation. - Preparation ----------- -Because we want to push our changes we will first make a fork of the -``ioc-adsample`` repository. We will then clone our fork locally and -make the changes there. - -To make a fork go to -`ioc-adsample `_ -and click the ``Fork`` button in the top right corner. This will create a fork -of the repository under your own GitHub account. - -Now, clone the fork, build the container image locally and open the -developer container: +First, clone the ``ioc-adsimdetector`` repository and make sure the container +build is working: .. code-block:: console - git clone git@github.com:/ioc-adsample.git - cd ioc-adsample + git clone git@github.com:epics-containers/ioc-adsimdetector.git + cd ioc-adsimdetector ./build code . - # click the green button in the bottom left corner of vscode and select - # "Reopen in Container" + # Choose "Reopen in Container" + +Note that if you do not see the prompt to reopen in container, you can open +the ``Remote`` menu with ``Ctrl+Alt+O`` and select ``Reopen in Container``. The ``build`` script does two things. - it fetches the git submodule called ``ibek-support``. This submodule is shared between all the EPICS IOC container images and contains the support YAML files that tell ``ibek`` how to build support modules inside the container - environment. + environment and how to use them at runtime. - it builds the Generic IOC container image locally. .. note:: The ``build`` script is a convenience script that is provided in the Generic IOC Template project. It is exactly equivalent to cloning - with ``--recursive`` flag and then running ``ec dev build``. + with ``--recursive`` flag and then running ``ec dev build``. Equally, + opening a vscode dev container will also build the container for you, but it + does require the ``ibek-support`` submodule to be present - using + ``--recursive`` flag to git clone ensures this. Verify the Example IOC Instance is working ------------------------------------------ @@ -94,53 +80,168 @@ Verify the Example IOC Instance is working When a new Generic IOC developer container is opened, there are two things that need to be done before you can run an IOC instance inside of it. -- Build the IOC source code +- Build the IOC binary - Select an IOC instance definition to run -The folder ``ioc`` inside of the ``ioc-adsample`` is where the IOC source code -is created and built. When you open the developer container, this folder does -not yet exist. The following command will create it and build the IOC: +The folder ``ioc`` inside of the ``ioc-adsimdetector`` is where the IOC source code +resided. However our containers always make a symlink to this folder at +``/epics/ioc``. This is so that it is always in the same place and can easily be +found by ibek (and the developer!). Therefore you can build the binary with the +following command: .. code-block:: console - ec ioc build + cd /epics/ioc + make + +.. note:: + + Note that we are required to build the IOC. + This is even though the container you are using already had the IOC + source code built by its Dockerfile (``ioc-adsimdetector/Dockerfile`` + contains the same command). + + For a detailed explanation of why this is the case see `ioc-source` The IOC instance definition is a YAML file that tells ``ibek`` what the runtime assets (ie. EPICS DB and startup script) should look like. Previous tutorials selected the IOC instance definition from a beamline repository. In this case -we will use the example IOC instance that comes with ``ioc-adsample``. The +we will use the example IOC instance that comes with ``ioc-adsimdetector``. The following command will select the example IOC instance: .. code-block:: console - ibek dev instance /epics/ioc-adsample/ioc_examples/bl01t-ea-ioc-02 + ibek dev instance /workspaces/ioc-adsimdetector/ioc_examples/bl01t-ea-ioc-02 -In an earlier tutorial when learning about the dev container, we manually -performed this step, see `choose-ioc-instance`. The above command does -exactly the same thing: removes the existing config folder in ``/epics/ioc`` +The above command removes the existing config folder ``/epics/ioc/config`` and and symlinks in the chosen IOC instance definition's ``config`` folder. Now run the IOC: .. code-block:: console - ibek dev run + cd /epics/ioc + ./start.sh -You should see a iocShell prompt and no error messages above. +You should see an iocShell prompt and no error messages above. -.. note:: +Let us also make sure we can see the simulation images that the IOC is +producing. For this we need the ``cd2v`` tool that we used earlier. You +can use the same virtual environment that you created earlier, or create +a new one and install again. Note that these commands are to be run +in a terminal outside of the developer container. - The ``ec ioc build`` command required to re-create the IOC source code. - This is even though the container you are using already had the IOC - source code built by its Dockerfile (``ioc-adsample/Dockerfile`` - contains the same command). +.. code-block:: console + + python3 -m venv cd2v + source ~/cd2v/bin/activate + pip install c2dataviewer + +Run the ``cd2v`` tool and connect it to our IOCs PVA output: + +.. code-block:: console + + cd2v --pv BL01T-EA-TST-02:PVA:OUTPUT & + + +Back inside the developer container, you can now start the detector and +the PVA plugin, by opening a new terminal and running the following: + +.. code-block:: console + + caput BL01T-EA-TST-02:PVA:EnableCallbacks 1 + caput BL01T-EA-TST-02:CAM:Acquire 1 + +You should see the moving image in the ``cd2v`` window. We now have a working +IOC instance that we can use to test our changes. + +Making a change to the Generic IOC +---------------------------------- + +One interesting way of changing a Generic IOC is to modify the support YAML +for one of the support modules. The support YAML describes the ``entities`` that +an IOC instance can make use of in its instance YAML file. This will be +covered in much more detail in `generic_ioc`. + +For this exercise we will make a change to the ``ioc-adsimdetector`` support +YAML file. We will change the startup script that it generates so that the +simulation detector is automatically started when the IOC starts. + +To make this change we just need to have the startup script set the values +of the records ``BL01T-EA-TST-02:CAM:Acquire`` and +``BL01T-EA-TST-02:PVA:EnableCallbacks`` to 1. + +To make this change, open the file +``ibek-support/ADSimDetector/ADSimDetector.ibek.support.yaml`` +and add a ``post_init`` section just after the ``pre_init`` section: + +.. code-block:: yaml + + post_init: + - type: text + value: | + dbpf {{P}}{{R}}Acquire 1 + +Next make a change to the file ``ibek-support/ADCore/ADCore.ibek.support.yaml``. +Find the NDPvaPlugin section and also add a ``post_init`` section: + +.. code-block:: yaml + + post_init: + - type: text + value: | + dbpf {{P}}{{R}}EnableCallbacks 1 + + +If you now go to the terminal where you ran your IOC, you can stop it with +``Ctrl+C`` and then start it again with ``./start.sh``. You should see the +following output at the end of the startup log: + +.. code-block:: console + + dbpf BL01T-EA-TST-02:CAM:Acquire 1 + DBF_STRING: "Acquire" + dbpf BL01T-EA-TST-02:PVA:EnableCallbacks 1 + DBF_STRING: "Enable" + epics> + +You should also see the ``cd2v`` window update with the moving image again. + +If you wanted to publish these changes you would have to commit both the +``ibek-support`` submodule and the ``ioc-adsimdetector`` repository and push +them in that order because of the sub-module dependency. But we won't be +pushing these changes as they are just for demonstration purposes. In later +tutorials we will cover making forks and doing pull requests for when you have +changes to share back with the community. + +Note: this is a slightly artificial example, as it would change the behaviour +for all instances of a PVA plugin and a simDetector. In a real IOC you would +do this on a per instance basis. + +Let us quickly do the instance YAML change to demonstrate the correct approach +to this auto-starting detector. + +Undo the support yaml changes: + +.. code-block:: console + + cd /workspaces/ioc-adsimdetector/ibek-support + git reset --hard + +Add the following to +``/workspaces/ioc-adsimdetector/ioc_examples/bl01t-ea-ioc-02/config/ioc.yaml``: + +.. code-block:: yaml + + - type: epics.dbpf + pv: BL01T-EA-TST-02:CAM:Acquire + value: "1" - For a detailed explanation of why this is the case see - `ioc-source` + - type: epics.dbpf + pv: BL01T-EA-TST-02:PVA:EnableCallbacks + value: "1" +Now restart the IOC and you should see the same behaviour as before. Here +we have made the change on a per instance basis, and used the ``dbpf`` entity +declared globally in ``ibek-support/_global/epics.ibek.support.yaml``. -TODO: complete by adding iocStats and using it in the ioc instance, then -pushing and verifying CI runs and publishes a new image. -TODO: now that cacheing is working, consider using ioc-adsimdetector instead -of ioc-adsample. This is simpler - the change could be the addition of -auto start of the sim detector IOC just like the presentation.