diff --git a/package_list/index.html b/package_list/index.html index 3f87bc0..efea040 100644 --- a/package_list/index.html +++ b/package_list/index.html @@ -1383,6 +1383,10 @@
See the navigation links in the header or side-bar.
Click (top left) on mobile.
"},{"location":"#welcome","title":"Welcome","text":"Welcome to the CGSE framework documentation.
Get started or go straight to the Tutorial
"},{"location":"#what-is-the-cgse","title":"What is the CGSE?","text":"The CGSE is short for Common-EGSE, a framework for managing and running test equipment in a lab. The EGSE stands for Electrical Ground Support Equipment, and this includes all equipment that is used to test or calibration an instrument.
All you need to get started using and building the CGSE.
"},{"location":"getting_started/#requirements","title":"Requirements","text":"You should always work inside a virtual environment to somehow containerize your project such that it doesn't pollute your global environment and you can run different projects next to each other. Create and activate a new virtual environment as follows. It should be Python >= 3.9
$ python -m venv venv\n$ source venv/bin/activate\n
"},{"location":"getting_started/#installation","title":"Installation","text":"The easiest way to install the CGSE is to use the pip
command. Since the CGSE is a monorepo and consists of several packages, you will need to make your choice which package you need for your project. You can however start with the cgse-common
which contains all common code that is generic and useful as a basis for other packages.
$ pip install cgse-common\n
Check the list of packages that are part of the CGSE repo and can be installed with pip
. The packages are described in more detail in the sections Libs and Projects.
To check your installation and set up your environment, here are a few tips.
The version of the core packages and any plugin packages can be verified as follows. The version you installed will probably be higher and more lines will appear when other packages are installed.
$ py -m egse.version\nCGSE version in Settings: 2025.0.5\nInstalled version for cgse-common= 2025.0.5\n
Check your environment with the command below. This will probably print out some warning since you have not defined the expected environment variables yet. There are two mandatory environment variables: PROJECT
and SITE_ID
. The former shall contain the name of your project without spaces and preferably a single word or an acronym like PLATO, ARIEL, MARVEL, MERCATOR. The latter is the name of the site or lab where the tests will be performed. Good names are KUL, ESA, LAB23.
The other environment variables follow the pattern <PROJECT>_...
, i.e. they all start with the project name as defined in the PROJECT environment variable. You should define at least <PROJECT>_DATA_STORAGE_LOCATION
. The configuration data and log file location will be derived from it unless they are explicitly set themselves.
Let's define the three expected environment variables:
$ export PROJECT=ARIEL\n$ export SITE_ID=VACUUM_LAB\n$ export ARIEL_DATA_STORAGE_LOCATION=~/data\n
Rerunning the above command now gives:
$ py -m egse.env\nEnvironment variables:\n PROJECT = ARIEL\n SITE_ID = VACUUM_LAB\n ARIEL_DATA_STORAGE_LOCATION = /Users/rik/data\n ARIEL_CONF_DATA_LOCATION = not set\n ARIEL_CONF_REPO_LOCATION = not set\n ARIEL_LOG_FILE_LOCATION = not set\n ARIEL_LOCAL_SETTINGS = not set\n\nGenerated locations and filenames\n get_data_storage_location() = '/Users/rik/data/VACUUM_LAB' \u27f6 ERROR: The data storage location doesn't exist!\n get_conf_data_location() = '/Users/rik/data/VACUUM_LAB/conf' \u27f6 ERROR: The configuration data location doesn't exist!\n get_conf_repo_location() = None \u27f6 ERROR: The configuration repository location doesn't exist!\n get_log_file_location() = '/Users/rik/data/VACUUM_LAB/log' \u27f6 ERROR: The log files location doesn't exist!\n get_local_settings() = None \u27f6 ERROR: The local settings file is not defined or doesn't exist!\n\nuse the '--full' flag to get a more detailed report, '--doc' for help on the variables.\n
Note
The folders that do not exist (and are not None) can be created by adding the option --mkdir
to the above command.
Settings contains the static configuration of your system, test setup, equipment, including the System Under Test (SUT). You can test your settings with the command below. Let's first also set the ARIEL_LOCALSETTINGS
environment variables:
$ export ARIEL_LOCAL_SETTINGS=~/cgse/local_settings_ariel_vacuum_lab.yaml\n$ py -m egse.settings\nSettings\n\u251c\u2500\u2500 PACKAGES\n\u2502 \u2514\u2500\u2500 CGSE_COMMON: Common classes, functions, decorators, etc. for the CGSE\n\u251c\u2500\u2500 SITE\n\u2502 \u251c\u2500\u2500 ID: VACUUM_LAB\n\u2502 \u251c\u2500\u2500 SSH_SERVER: localhost\n\u2502 \u2514\u2500\u2500 SSH_PORT: 22\n\u2514\u2500\u2500 PROCESS\n \u2514\u2500\u2500 METRICS_INTERVAL: 10\nMemoized locations:\n['/Users/rik/tmp/gettings-started/venv/lib/python3.9/site-packages/cgse_common/settings.yaml', \n'/Users/rik/cgse/local_settings_ariel_vacuum_lab.yaml']\n
These Settings will grow when you add more packages to your installation or when you define settings yourself in the local settings file."},{"location":"help/","title":"Help","text":"The best way to get help for something that you couldn't find in the documentation on this site, is to contact one of the authors.
"},{"location":"help/#bugs-and-feature-requests","title":"Bugs and Feature requests","text":"Report any bugs or issues through GitHub on the CGSE issues page.
"},{"location":"initialize/","title":"Initialize your project","text":"So, we have seen how to get started with some basic commands and only the cgse-common
package. It's time now to initialize your project properly with all the necessary services.
I assume you are in the same environment that we have set up in the previous section where also the cgse-common
package was installed. We will install another package that will provide us with the full functionality of the cgse
command. Install the cgse-tools
package which depends on cgse-core
and will also install that package.
$ pip install cgse-tools\n
You should now have at least the follow three packages installed in your virtual environment:
$ pip list | grep cgse\ncgse-common 2025.0.8\ncgse-core 2025.0.8\ncgse-tools 2025.0.8\n
"},{"location":"initialize/#the-cgse-command","title":"The cgse
command","text":"The two new packages that have been installed (cgse-core
and cgse-tools
) provide the cgse
command that we will use to initialize your environment, but this command is also used to inspect different parts of the system, manage core services and device drivers, etc.
When you run the cgse
command without any arguments, it will show something like this:
$ cgse\n\n Usage: cgse [OPTIONS] COMMAND [ARGS]...\n\n The main cgse command to inspect, configure, monitor the core services and device control servers.\n\n\u256d\u2500 Options \u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u256e\n\u2502 --install-completion Install completion for the current shell. \u2502\n\u2502 --show-completion Show completion for the current shell, to copy it or customize the installation. \u2502\n\u2502 --help Show this message and exit. \u2502\n\u2570\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u256f\n\u256d\u2500 Commands \u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u256e\n\u2502 version Prints the version of the cgse-core and other registered packages. \u2502\n\u2502 init Initialize your project. \u2502\n\u2502 top A top-like interface for core services and device control servers. \u2502\n\u2502 core handle core services: start, stop, status \u2502\n\u2502 show Show information about settings, environment, setup, ... \u2502\n\u2502 check Check installation, settings, required files, etc. \u2502\n\u2502 dev-x device-x is an imaginary device that serves as an example \u2502\n\u2570\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u256f\n
The cgse
command is actually an app that is the starting point for a number of commands that can be used to maintain the system, manage and inspect services and devices. For example, to check the version of the different components, use: $ cgse version\nCGSE-COMMON installed version = 2025.0.8\nCGSE-CORE installed version = 2025.0.8\nCGSE-TOOLS installed version = 2025.0.8\n
We will for now concentrate on the init
command. This command will guide us through a number of steps to define the location of our device data, configuration data, etc. We will basically define some environment variables that are used by the CGSE framework. The PROJECT is he name of the project your will be working on, the SITE_ID is the identifier for the LAB or Setup that you are using to perform the tests. As you see below, the environment variables all start with the project name allowing you to work on different projects simultaneously. If you accept all the defaults, the result of the cgse init
command will look something like this:
$ cgse init --project marvel\nPlease note default values are given between [brackets].\nWhat is the name of the project [MARVEL] ?:\nWhat is the site identifier ?: lab02\nWhere can the project data be stored [~/data/MARVEL/LAB02/] ?:\nWhere will the configuration data be located [~/data/MARVEL/LAB02/conf/] ?:\nWhere will the logging messages be stored [~/data/MARVEL/LAB02/log/] ?:\nWhere shall I create a local settings YAML file [~/data/MARVEL/LAB02/local_settings.yaml] ?:\nShall I add the environment to your ~/bash_profile ? [y/n]: n\n\n# -> Add the following lines to your bash profile or equivalent\n\nexport PROJECT=MARVEL\nexport SITE_ID=LAB02\nexport MARVEL_DATA_STORAGE_LOCATION=~/data/MARVEL/LAB02/\nexport MARVEL_CONF_DATA_LOCATION=~/data/MARVEL/LAB02/conf/\nexport MARVEL_LOG_FILE_LOCATION=~/data/MARVEL/LAB02/log/\nexport MARVEL_LOCAL_SETTINGS=~/data/MARVEL/LAB02/local_settings.yaml\n
If you answered 'Y' to the last question, you should log in to the shell again with exec bash -login
or a similar command for other shells, or you should start a new terminal to activate the environment variables. Add this point you are ready to go and start the core services and any device control servers that you need. You can explore other commands of the cgse
app in the user guide.
The CGSE is a monorepo and consists of numerous packages. Each of these packages are individually installable from PyPI. We maintain a list here with all the packages in the monorepo.
Package Descriptioncgse-common
The common code used by all other packages cgse-core
The core services cgse-coordinates
Coordinate reference Frames cgse-gui
GUI components and styles (PyQt5) cgse-tools
Plugin that adds functions to the cgse
command symetrie-hexapod
Device drivers for the Sym\u00e9trie Hexapods plato-fits
FITS driver with PLATO specific format plato-hdf5
HDF5 driver with PLATO specific format plato-spw
SpaceWire driver with PATO specific packets The following is a non-exhaustive list of known external packages that work well with the CGSE in terms of commanding and monitoring.
Package Descriptioncgse-dummy
Provides a dummy device driver to demonstrate plugins, commands and how to develop an external package for the CGSE."},{"location":"tutorial/","title":"Tutorial","text":"Welcome to the CGSE Tutorial!
By the end of this page you should have a solid understanding of the core features of the CGSE.
"},{"location":"dev_guide/","title":"Developer Guide","text":"Welcome to the CGSE developer guide! An in-depth reference on how to contribute to the CGSE.
First thing to know is that this repository is actually a monorepo, meaning it contains a bunch of related but self-standing packages with a minimum of interdependencies. These packages are
"},{"location":"dev_guide/coding_style/","title":"Style Guide","text":"This part of the developer guide contains instructions for coding styles that are adopted for this project.
The style guide that we use for this project is PEP8. This is the standard for Python code and all IDEs, parsers and code formatters understand and work with this standard. PEP8 leaves room for project specific styles. A good style guide that we can follow is the Google Style Guide.
The following sections will give the most used conventions with a few examples of good and bad.
"},{"location":"dev_guide/coding_style/#tldr","title":"TL;DR","text":"Type Style Example Classes CapWords ProcessManager, ImageViewer, CommandList, Observation, MetaData Methods & Functions lowercase with underscores get_value, set_mask, create_image Variables lowercase with underscores key, last_value, model, index, user_info Constants UPPERCASE with underscores MAX_LINES, BLACK, COMMANDING_PORT Modules & packages lowercase no underscores dataset, commanding, multiprocessing"},{"location":"dev_guide/coding_style/#general","title":"General","text":"user_list
might look good at first, but what if at some point you want to change the list to a set so it can not contain duplicates. Are you going to rename everything into user_set
or would user_info
be a better name?SyntaxError: invalid syntax
.Always use CamelCase (Python uses CapWords) for class names. When using acronyms, keep them all UPPER case.
Manager
or Controller
or Organizer
all mean similar things. Choose one word for the concept and stick to it.Good names are: Observation
, CalibrationFile
, MetaData
, Message
, ReferenceFrame
, URLParser
.
A function or a method does something (and should only do one thing, SRP=Single Responsibility Principle), it is an action, so the name should reflect that action.
Always use lowercase words separated with underscores.
Good names are: get_time_in_ms()
, get_commanding_port()
, is_connected()
, parse_time()
, setup_mask()
.
When working with legacy code or code from another project, names may be in camelCase (with the first letter a lower case letter). So we can in this case use also getCommandPort()
or isConnected()
as method and function names.
Use the same naming convention as functions and methods, i.e. lowercase with underscores.
Good names are: key
, value
, user_info
, model
, last_value
Bad names: NSegments
, outNoise
Take care not to use builtins: list
, type
, filter
, lambda
, map
, dict
, ...
Private variables (for classes) start with an underscore: _name
or _total_n_args
.
In the same spirit as method and function names, the variables can also be in camelCase for specific cases.
"},{"location":"dev_guide/coding_style/#constants","title":"CONSTANTS","text":"Use ALL_UPPER_CASE with underscores for constants. Use constants always within a name space, not globally.
Good names: MAX_LINES
, BLACK
, YELLOW
, ESL_LINK_MODE_DISABLED
Use simple words for modules, preferably just one word like datasets
or commanding
or storage
or extensions
. If two words are unavoidable, just concatenate them, like multiprocessing
or sampledata
or testdata
. If needed for readability, use an underscore to separate the words, e.g. image_analysis
.
from <module> import *
Be careful that you do not name any modules the same as a module in the Python standard library. This can result in strange effects and may result in an AttributeError
. Suppose you have named a module math
in the egse
directory and it is imported and used further in the code as follows:
from egse import math\n\n# in some expression further down the code you might use\n\nmath.exp(a)\n
This will result in the following runtime error:
File \"some_module.py\", line 8, in <module>\n print(math.exp(a))\nAttributeError: module 'egse.math' has no attribute 'exp'\n
Of course this is an obvious example, but it might be more obscure like e.g. in this GitHub issue: 'module' object has no attribute 'Cmd'.
"},{"location":"dev_guide/docs/","title":"Building the documentation","text":"mkdocs serve
from the project root olderdocs/*
I created a virtual environment using pyenv
and when I'm working on the documentation, I start up a shell with this environment. Currently, only mkdocs
and mkdocs-material
are needed. Of course, you need to install these only once.
$ pyenv virtualenv 3.10 cgse-doc-3.10\n$ pyenv shell cgse-doc-3.10\n$ pip install --upgrade pip setuptools wheel\n$ pip install mkdocs\n$ pip install mkdocs-material\n
From this shell, navigate to the project root folder and start the live-reload server of mkdocs
.
$ cd ~/github/cgse\n$ mkdocs serve\n
Now you can update files, create new folders in docs/*
, create new Markdown files and all changes will be reloaded live in the browser.
When you are ready with updating, you will need to build the site and publish it on GitHub pages:
$ mkdocs build\n$ mkdocs gh-deploy -r upstream -m \"documentation update on ..\"\n
"},{"location":"dev_guide/docs/#commands","title":"Commands","text":"mkdocs serve
\u2014 start the live-reloading docs servermkdocs build
\u2014 build the documentation sitemkdocs deploy
\u2014 publish your documentation on GitHub pagesmkdocs -h
\u2014 print a help message for more optionsmkdocs.yml # the mkdocs configuration file\ndocs/\n index.md # the documentation homepage\n ... # other markdown pages, image, folders, ...\n
"},{"location":"dev_guide/installation/","title":"Installation Guide for Developers","text":""},{"location":"dev_guide/plugins/","title":"Plugins","text":"The CGSE is designed to be extensible and uses a few plugin mechanisms to extend its functionally with external contributions. Also within the cgse
monorepo we use the plugin mechanism at several places. The following entry-points are currently defined:
cgse.version
: Each package that provides functionality within the CGSE or adds a device driver registers itself to provide version information.cgse.command.plugins
: Packages can add commands or sub-commands to the cgse
app to manage their functionality from within the cgse
app, e.g. to start or stop the service or to report on its status.cgse.settings
: Package can add their own settings.cgse.explore
: Package provides a set of functions to explore, e.g. if any of the processes it provides are running.cgse.resource
: Packages can register resources.Each of the entry-points knows how to load a module or object and each entry-point group is connected to a specific action or plugin hook like, e.g. add a command or command group to the cgse
app, add package specific settings to the global settings.
When you write a package that you want to integrate with the CGSE, provide a cgse.version
entry-point. The name of the entry-point shall match the package name and is used to read the version from the importlib metadata. The entry-point value is currently not used. The entry-point value can optionally provide additional information about the package, but that is currently not specified.
Add the following to your pyproject.toml
file in your project's root folder, replacing package-name with the name of your project. The entry-point value is currently not used, but you want to use a valid format, the value below is always valid.
[project.entry-points.\"cgse.version\"]\npackage-name = 'egse.version:get_version_installed'\n
"},{"location":"dev_guide/plugins/#extending-the-cgse-app","title":"Extending the cgse
app","text":""},{"location":"dev_guide/plugins/#add-a-command","title":"Add a Command","text":"If your package provides specific functionality that can be added as a command or a command group to the cgse
app, use the cgse.command
entry-point group. Since the cgse
app uses the Typer package to build its commandline interface, adding a command is as simple as writing a function. The function will be added to the cgse
app using the app.command()
function of Typer
, making the function a top-level command of the cgse
app. The function can be defined as a plain function or with Typer's @app.command
decorator.
In the pyproject.toml
file of your project, add the following lines to add the CGSE command:
[project.entry-points.\"cgse.command\"]\nname = 'module:object'\n
Where:
name
is the name of the commandmodule
is a fully qualified module name for your package, a module that can be importedobject
is the name of the function that you want to add as a commandAs an example, for the cgse-tools
package, the init
command of the cgse
app is listed in the pyproject.toml
file as follows:
[project.entry-points.\"cgse.command\"]\ninit = 'cgse_tools.cgse_commands:init'\n
The init
function is defined in the cgse_commands.py
module which is located in the cgse_tools
module in the src
folder of the package:
src\n\u251c\u2500\u2500 cgse_tools\n\u2502 \u251c\u2500\u2500 __init__.py\n\u2502 \u251c\u2500\u2500 cgse_commands.py\n...\n
"},{"location":"dev_guide/plugins/#add-a-command-group","title":"Add a Command group","text":"Some commands are more complicated and define a number of sub-commands. An example is the show
command where you currently have the sub-commands env
and settings
$ cgse show --help\n\n Usage: cgse show [OPTIONS] COMMAND [ARGS]...\n\n Show information about settings, environment, setup, ...\n\n\u256d\u2500 Options \u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u256e\n\u2502 --help Show this message and exit. \u2502\n\u2570\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u256f\n\u256d\u2500 Commands \u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u256e\n\u2502 settings Show the settings that are defined by the installed packages. \u2502\n\u2502 env Show the environment variables that are defined for the project. \u2502\n\u2570\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u256f\n
The show
command is defined as a typer.Typer()
object where env
and settings
are added using the decorator @<app>.command()
.
import typer\n\nshow = typer.Typer(help=\"Show information about settings, environment, setup, ...\")\n\n\n@show.command(name=\"settings\")\ndef show_settings():\n ...\n\n\n@show.command(name=\"env\")\ndef show_env():\n ...\n
To add this command group to the cgse
app, the following entry was used in the pyproject. toml
file of the cgse-tools
project. Notice the [group]
at the end of the entry which indicates this is a command group instead of a single command.
[project.entry-points.\"cgse.command\"]\nshow = 'cgse_tools.cgse_commands:show[group]'\n
"},{"location":"dev_guide/plugins/#add-a-service","title":"Add a Service","text":"If your package provides a device driver or a specific service, use the cgse.service
entry-point group. Service entry-points follow the same scheme as command groups, i.e. they are added to the cgse
app as a Typer()
object. Use the following entry in your pyproject.toml
file:
[project.entry-points.\"cgse.service\"]\nname = 'module:object'\n
where:
name
is the name of the service or device drivermodule
is a fully qualified module name for your package, a module that can be importedobject
is the name of the Typer()
object that you want to add as a serviceThe entry-point cgse.explore
can be used to extend functionality without adding a new command or sub-command to the cgse
app. The idea is that commands that work on different packages can use this entry-point to perform certain tasks on the package. This is currently used for the show procs
command (see below).
The entry-point has the following format:
[project.entry-points.\"cgse.explore\"]\nexplore = \"<package>.cgse_explore\"\n
So, what happens is that a command that wants to apply a functionality on an external package loads the cgse_explore.py
module for that package and checks if a function with a specific name exists in that module. It then executes that function. For the show procs
command, the function show_processes
is expected and it shall return a list of strings which currently are printed to the terminal. This entry-point is currently implemented for cgse-core
and cgse-dummy
(an external demo package) and when you run the cgse show procs
command it looks something like below (the format is from the unix ps -ef
command).
\u279c cgse show procs\n459800007 76849 1 0 11:07PM ttys003 0:03.53 /Users/rik/tmp/test_dummy/venv/bin/python3.9 -m egse.logger.log_cs start\n459800007 76850 1 0 11:07PM ttys003 2:18.60 /Users/rik/tmp/test_dummy/venv/bin/python3.9 -m egse.storage.storage_cs start\n459800007 76851 1 0 11:07PM ttys003 2:20.10 /Users/rik/tmp/test_dummy/venv/bin/python3.9 -m egse.confman.confman_cs start\n459800007 13825 1 0 4:31PM ttys003 0:02.97 /Users/rik/tmp/test_dummy/venv/bin/python3.9 -m cgse_dummy.dummy_sim start\n
"},{"location":"dev_guide/plugins/#register-resources","title":"Register resources","text":"TODO: what if two packages provide a resource icons
?
uv
","text":"uv
is an extremely fast Python package and project manager, written in Rust. We will use uv
as the single tool that replaces pip
, virtualenv
, pyenv
, and more. The main tasks for which we will use uv
are:
uv
","text":"On macOS and Linux you can install uv
using curl
:
$ curl -LsSf https://astral.sh/uv/install.sh | sh\n
If you need more specific information on installing and upgrading uv
, please refer to the official documentation.
The CGSE is guaranteed to work with Python 3.9.x. We will gradually include higher versions of Python, but currently these have not been tested. So, we will for the moment stick with Python 3.9.20. Install this version as follows:
$ uv python install 3.9.20\n
pyenv
When you are using pyenv
to manage your Python versions, make sure you also have the same Python version installed with pyenv
and uv
. Otherwise you will run into the following error. This is a known issue with uv
.
pyenv: version `3.9.20' is not installed (set by /Users/rik/github/cgse/libs/cgse-common/.python-version)\n
You can check which Python versions are installed already on your system:
CommandOutput$ uv python list --only-installed\n
cpython-3.12.8-macos-aarch64-none /Users/rik/Library/Application Support/uv/python/cpython-3.12.8-macos-aarch64-none/bin/python3.12\ncpython-3.10.16-macos-aarch64-none /Users/rik/Library/Application Support/uv/python/cpython-3.10.16-macos-aarch64-none/bin/python3.10\ncpython-3.9.21-macos-aarch64-none /Users/rik/Library/Application Support/uv/python/cpython-3.9.21-macos-aarch64-none/bin/python3.9\ncpython-3.9.20-macos-aarch64-none /Users/rik/Library/Application Support/uv/python/cpython-3.9.20-macos-aarch64-none/bin/python3.9\ncpython-3.9.6-macos-aarch64-none /Library/Developer/CommandLineTools/usr/bin/python3 -> ../../Library/Frameworks/Python3.framework/Versions/3.9/bin/python3\ncpython-3.8.17-macos-aarch64-none /Users/rik/Library/Application Support/uv/python/cpython-3.8.17-macos-aarch64-none/bin/python3.8\n
"},{"location":"dev_guide/uv/#create-a-virtual-environment","title":"Create a virtual environment","text":"Pin a Python version
You can pin a python version with the command:
$ uv python pin 3.9.20\n
uv
will search for a pinned version in the parent folders up to the root folder or your home directory.
You can create a virtual environment with uv
for the specific Python version as follows. The '--python
' is optional and uv
will use the default (pinned) Python version when creating a venv
without this option. We are working in a monorepo or what uv
calls a workspace. There will be only one virtual environment at the root of the monorepo, despite the fact that we have several individual packages in our workspace. Don't worry, uv
will always use the virtual environment at the root and keep it up-to-date with the project your are currently working in.
When creating a virtual environment make sure you are in the root folder, e.g. ~/github/cgse
.
$ cd ~/github/cgse\n$ uv venv --python 3.9.20\n
Now, navigate to the package you will be working in and update the projects' environment, assuming you are going to work in cgse-core
, this will be:
$ cd ~/github/cgse/libs/cgse-core\n$ uv sync\n
Your package(s) from the workspace should be installed as an editable install. You can check this with the command:
$ uv pip list -v\nUsing Python 3.9.20 environment at: /Users/rik/github/cgse/.venv\nPackage Version Editable project location\n----------------- ----------- ---------------------------------------\napscheduler 3.11.0\ncgse-common 0.4.0 /Users/rik/github/cgse/libs/cgse-common\ncgse-core 0.4.0 /Users/rik/github/cgse/libs/cgse-core\n...\n
To install any other project as an editable package:
$ uv pip install -e <project location>\n
Note
If you don't want to use the uv
commands, you can activate the virtual environment and use the original pip
and python
commands as you are used to, but I would recommend you try to get used to uv
for a while to experience its benefits.
$ source ~/github/cgse/.venv/bin/activate\n
Info
In a workspace, maintaining a virtual environment per package might be a hassle and most of the time that is not needed. A good approach is to always use the virtual environment at the workspace root. This venv
which will be automatically created if you run a command or if you use uv sync
in the package folder. With uv sync
you can make sure the virtual environment is up-to-date and contains only those dependencies that are required for the package you are in. So, each time you switch to another package and want to run a comand or a test for that package, use
$ uv sync\n
"},{"location":"dev_guide/uv/#building-and-publishing-all-packages","title":"Building and publishing all packages","text":"We have chosen for one and the same version number for all packages in the cgse
monorepo. That means that whenever we make a change to one of the packages and want to release that change, all packages shall be rebuild and published.
Inline
When working in a workspace, keep in mind that the commands uv run
and uv sync
by default work on the workspace root. That means that when you run the uv run pip install <package>
command, the .venv
at the workspace root will be updated or created if it didn't exist. Similar for the uv sync
command, there is only one uv.lock
file at the root of the workspace.
Fortunately, with uv
, that is done in a few commands.
When you are in the monorepo root folder, you can build all packages at once. They will be placed in the dist
folder of the root package. Before building, make sure you update the version in the pyproject.toml
of the root package and then bump the versions. Before building, clean up the dist
folder, then you can do a default uv publish
afterwards.
$ cd <monorepo root>\n$ uv run bump.py\n$ rm -r dist\n$ uv build --all-packages\n
Publish all packages in the root dist folder to PyPI. The UV_PUBLISH_TOKEN can be defined in a (read protected) ~/. setenv.bash file:
$ uv publish --token $UV_PUBLISH_TOKEN\n
The above command will publish all package to PyPI. If you don't want the token to be in a shell variable, you can omit the --token
in the command above. You will then be asked for a username, use __token__
as the username and then provide the token as a password.
The libraries are those packages that make up the CGSE framework.
The libraries are located under the libs
folder, and we currently find the following packages there:
cgse-common
cgse-core
cgse-coordinates
cgse-gui
This package cgse-common
contains modules that are used by all other packages.
egse.bits
convenience functions to work with bits, bytes and integers egse.calibration
functions to handle conversions and apply correction egse.command
classes and functions to work with commands that operate hardware devices egse.config
convenience functions to configure the system and find folders and files egse.control
defines abstract classes and convenience functions for any control server egse.decorators
a collection of useful decorator functions egse.device
defines the generic interfaces to connect devices egse.env
functionality to work with and check your environment variables egse.exceptions
common Exceptions and Errors egse.hk
functions to retrieve and convert housekeping parameter values egse.metrics
functions to define and update metrics egse.mixin
defines the mixin classes for dynamic commanding egse.monitoring
the monitoring application / function egse.observer
the classic observer and observable egse.obsid
functions to define and work with the OBSID egse.persistence
the persistence layer interface egse.plugin
functions to load plugins and settings from entry-points egse.process
functions and classes to work with processes and sub-processes egse.protocol
base class for communicating commands with the hardware or the control server egse.proxy
base class for the Proxy objects for each device controller egse.reload
a slightly better approach to reloading modules and function egse.resource
convenience functions to use resources in your code egse.response
defines the classes to handle responses from the control servers egse.services
provides the services to the control servers egse.settings
provides functions to handle user and configuration settings egse.setup
defines the Setup, containing the complete configuration for a test egse.state
classes and functions to handle state, e.g. the GlobalState egse.system
convenience functions that provide information on system specific functionality egse.version
functions to load specific version information egse.zmq_ser
serialization function used in a ZeroMQ context"},{"location":"libs/cgse-common/settings/","title":"The Settings","text":"The Settings class contains all static information needed to configure your system, the environment you are using and the test equipment. The Settings also contain all the IP addresses and port number for all the known devices, together with other static information like the device name, default settings for the device like speed, timeout, delay time, firmware version, etc. We will go into more details about the content later, let\u2019s now first look at the format and usage of the Settings.
"},{"location":"libs/cgse-common/settings/#loading-the-settings","title":"Loading the Settings","text":"The Settings can be loaded as follows:
>>> from egse.settings import Settings\n>>> settings = Settings.load()\n
The settings
object will be a dictionary where the keys are the top-level groups that are defined in the settings for each package. For a system that has only cgse-common
and cgse-core
installed, the settings
will contain something like this:
>>> print(settings)\nSettings\n\u251c\u2500\u2500 PACKAGES\n\u2502 \u251c\u2500\u2500 CGSE_COMMON: Common classes, functions, decorators, etc. for the CGSE\n\u2502 \u2514\u2500\u2500 CGSE_CORE: The core services of the CGSE\n\u251c\u2500\u2500 SITE\n\u2502 \u251c\u2500\u2500 ID: LAB42\n\u2502 \u251c\u2500\u2500 SSH_SERVER: localhost\n\u2502 \u2514\u2500\u2500 SSH_PORT: 22\n\u251c\u2500\u2500 PROCESS\n\u2502 \u2514\u2500\u2500 METRICS_INTERVAL: 10\n\u251c\u2500\u2500 Logging Control Server\n\u2502 \u251c\u2500\u2500 PROTOCOL: tcp\n\u2502 \u251c\u2500\u2500 HOSTNAME: localhost\n\u2502 \u251c\u2500\u2500 LOGGING_PORT: 7000\n\u2502 \u251c\u2500\u2500 COMMANDING_PORT: 7001\n\u2502 \u251c\u2500\u2500 METRICS_PORT: 7003\n\u2502 \u251c\u2500\u2500 MAX_NR_LOG_FILES: 20\n\u2502 \u251c\u2500\u2500 MAX_SIZE_LOG_FILES: 20\n\u2502 \u251c\u2500\u2500 TEXTUALOG_IP_ADDRESS: 127.0.0.1\n\u2502 \u2514\u2500\u2500 TEXTUALOG_LISTENING_PORT: 19996\n\u251c\u2500\u2500 Configuration Manager Control Server\n\u2502 \u251c\u2500\u2500 PROTOCOL: tcp\n\u2502 \u251c\u2500\u2500 HOSTNAME: localhost\n\u2502 \u251c\u2500\u2500 COMMANDING_PORT: 6000\n\u2502 \u251c\u2500\u2500 MONITORING_PORT: 6001\n\u2502 \u251c\u2500\u2500 SERVICE_PORT: 6002\n\u2502 \u251c\u2500\u2500 METRICS_PORT: 6003\n\u2502 \u251c\u2500\u2500 DELAY: 1\n\u2502 \u2514\u2500\u2500 STORAGE_MNEMONIC: CM\n\u2514\u2500\u2500 Storage Control Server\n \u251c\u2500\u2500 PROTOCOL: tcp\n \u251c\u2500\u2500 HOSTNAME: localhost\n \u251c\u2500\u2500 COMMANDING_PORT: 6100\n \u251c\u2500\u2500 MONITORING_PORT: 6101\n \u251c\u2500\u2500 SERVICE_PORT: 6102\n \u251c\u2500\u2500 METRICS_PORT: 6103\n \u2514\u2500\u2500 DELAY: 1\n
If you only need the settings for a particular component, specify that group's name:
>>> storage_settings = Settings.load(\"Storage Control Server\")\n\n>>> print(storage_settings)\nStorage\nControl\nServer\n\u251c\u2500\u2500 PROTOCOL: tcp\n\u251c\u2500\u2500 HOSTNAME: localhost\n\u251c\u2500\u2500 COMMANDING_PORT: 6100\n\u251c\u2500\u2500 MONITORING_PORT: 6101\n\u251c\u2500\u2500 SERVICE_PORT: 6102\n\u251c\u2500\u2500 METRICS_PORT: 6103\n\u2514\u2500\u2500 DELAY: 1\n
The values can be accessed as usual with a dictionary, by specifying the name of the parameter as the key:
>>> print(storage_settings[\"COMMANDING_PORT\"])\n6100\n
We usually only go one level deep when defining settings, and as a convenience, that first level of variables can also be accessed with the dot-notation.
>>> print(storage_settings.COMMANDING_PORT)\n6100\n
"},{"location":"libs/cgse-common/settings/#entry-points","title":"Entry-points","text":"The Settings are collected from a set of YAML files which are provided by the packages through the entry-point cgse.settings
. The default Settings file is named settings.yaml
but this can be changed by the entry-point (see below).
Let's take a look at how the settings are provided for the cgse-core
package. First, the pyproject.toml
file of the project shall define the entry-point. In the snippet below, the entry-point cgse-core
is defined for the group cgse.settings
.
[project.entry-points.\"cgse.settings\"]\ncgse-core = \"cgse_core:settings.yaml\"\n
The entry-point itself has the following format: <name> = \"<module>.<filename>\"
, where
<name>
is the name of the entry-point given in the pyproject.toml
file, usually this is the package name,<module>
is a valid module name that can be imported and from which the location can be determined, and<filename>
is the name of the target file, e.g. a YAML file.Note
The module name for this entry point has an underscore instead of a dash, i.e. cgse_core
instead of cgse-core
. The reason is that module names with a dash will generate a SyntaxError during import.
The above example will load the settings for this package from the settings.yaml
file that is located in the cgse_core
module. That is, the package shall also provide this as follows:
cgse-core\n\u251c\u2500\u2500 pyproject.toml\n\u2514\u2500\u2500 src\n \u2514\u2500\u2500 cgse_core\n \u251c\u2500\u2500 __init__.py\n \u2514\u2500\u2500 settings.yaml\n
The settigs.yaml
file for this module looks something like this:
PACKAGES:\n CGSE_CORE: The core services of the CGSE\n\nLogging Control Server: # LOG_CS\n\n PROTOCOL: tcp\n HOSTNAME: localhost # The hostname that client shall connect to, e.g. pleiad01 @ KU Leuven\n LOGGING_PORT: 7000\n COMMANDING_PORT: 7001\n METRICS_PORT: 7003 # The HTTP port where Prometheus will connect to for retrieving metrics\n MAX_NR_LOG_FILES: 20 # The maximum number of log files that will be maintained in a roll-over\n MAX_SIZE_LOG_FILES: 20 # The maximum size one log file can become\n TEXTUALOG_IP_ADDRESS: 127.0.0.1 # The IP address of the textualog listening server\n TEXTUALOG_LISTENING_PORT: 19996 # The port number on which the textualog server is listening\n\nConfiguration Manager Control Server: # CM_CS\n\n ...\n
"},{"location":"libs/cgse-common/settings/#local-settings","title":"Local Settings","text":"You can, and you should, define local settings for your project and put those settings in a known folder on your system. The usual place is ~/cgse/local-settings.yaml
. This file will be automatically loaded by the Settings.load()
function when you define the local settings environment variable. That variable name is <PROJECT>_LOCAL_SETTINGS
where <PROJECT>
is the name of your project as defined by the PROJECT
environment variable. For a PROJECT=LAB23
the local settings would be defined as follows:
$ export LAB23_LOCAL_SETTINGS=~/cgse/local-settings-lab23.yaml\n
The local settings take higher precedence that will overwrite the global settings when loaded. You only need to define the settings that actually change for your local installation, respect the full hierarchy when specifying those settings. You are allowed to define new entries at any level in the Settings hierarchy.
The usual parameters to put into a local settings file are:
You can check the current settings from the terminal with the following command:
$ py -m egse.settings\nSettings\n\u251c\u2500\u2500 PACKAGES\n\u2502 \u251c\u2500\u2500 CGSE_COMMON: Common classes, functions, decorators, etc. for the CGSE\n\u2502 \u2514\u2500\u2500 CGSE_CORE: The core services of the CGSE\n\u251c\u2500\u2500 SITE\n\u2502 \u251c\u2500\u2500 ID: LAB42\n\u2502 \u251c\u2500\u2500 SSH_SERVER: localhost\n\u2502 \u2514\u2500\u2500 SSH_PORT: 22\n\u251c\u2500\u2500 PROCESS\n\u2502 \u2514\u2500\u2500 METRICS_INTERVAL: 10\n\u251c\u2500\u2500 Logging Control Server\n\u2502 \u251c\u2500\u2500 PROTOCOL: tcp\n\u2502 \u251c\u2500\u2500 HOSTNAME: localhost\n\u2502 \u251c\u2500\u2500 LOGGING_PORT: 7000\n\u2502 \u251c\u2500\u2500 COMMANDING_PORT: 7001\n... ...\n\u2514\u2500\u2500 Storage Control Server\n \u251c\u2500\u2500 PROTOCOL: tcp\n \u251c\u2500\u2500 HOSTNAME: localhost\n \u251c\u2500\u2500 COMMANDING_PORT: 6100\n \u251c\u2500\u2500 MONITORING_PORT: 6101\n \u251c\u2500\u2500 SERVICE_PORT: 6102\n \u251c\u2500\u2500 METRICS_PORT: 6103\n \u2514\u2500\u2500 DELAY: 1\nMemoized locations:\n['/Users/rik/github/cgse/libs/cgse-common/src/cgse_common/settings.yaml', '/Users/rik/github/cgse/libs/cgse-core/src/cgse_core/settings.yaml', '/Users/rik/cgse/local_settings_ariel.yaml']\n
The memoized locations are the settings files that have been loaded and cached. Once the application has started and the settings have been loaded, they can only be reloaded by explicitly forcing a reload as follows:
>>> settings = Settings.load(force=True)\n
Warning
The force
reload does however not guarantee that the settings will propagate properly throughout the application or to client apps. Settings can be saved in local variables or class instances that have no knowledge of a Settings reload. So, be careful when changing your Settings. If there are parameters that change often and are not as static as thought, maybe they belong in the Setup instead of the Settings. Examples are:
load()
method","text":"A word about the Settings.load()
method. Depending on the parameters provided, this method either loads all settings, a group of settings or just one single YAML file. We have already explained how to load a specific group of settings by giving the name of the group as a parameter. When you want to load just one YAML file, you need to specify its location also. When a location is given as a str or a Path, the Settings will be loaded from that file only, using the default settings.yaml
name or another name given through the filename
argument.
This can be used to e.g. load command files for a device:
>>> commands = Settings.load(location=\"~\", filename=\"DAQ5610.yaml\")\n
The mechanism behind the Settings.load()
method is shown in the following diagram. For simplicity, parameters are not shown and only the success path is presented, not any exceptions or error handling.
The projects are those packages that add functionality to the CGSE framework.
The projects live under the folder projects
, and they are organised in generic and specific projects. Generic projects do not have an implementation that is specific for one particular project, while, obviously, specific projects have. We currently have the following generic packages:
cgse-tools
symetrie-hexapod
and then there are the project specific packages:
plato-fits
plato-hdf5
plato-spw
Welcome to the CGSE user guide! An in-depth reference on how to use the CGSE.
"}]} \ No newline at end of file +{"config":{"lang":["en"],"separator":"[\\s\\-]+","pipeline":["stopWordFilter"]},"docs":[{"location":"","title":"Home","text":"Tip
See the navigation links in the header or side-bar.
Click (top left) on mobile.
"},{"location":"#welcome","title":"Welcome","text":"Welcome to the CGSE framework documentation.
Get started or go straight to the Tutorial
"},{"location":"#what-is-the-cgse","title":"What is the CGSE?","text":"The CGSE is short for Common-EGSE, a framework for managing and running test equipment in a lab. The EGSE stands for Electrical Ground Support Equipment, and this includes all equipment that is used to test or calibration an instrument.
All you need to get started using and building the CGSE.
"},{"location":"getting_started/#requirements","title":"Requirements","text":"You should always work inside a virtual environment to somehow containerize your project such that it doesn't pollute your global environment and you can run different projects next to each other. Create and activate a new virtual environment as follows. It should be Python >= 3.9
$ python -m venv venv\n$ source venv/bin/activate\n
"},{"location":"getting_started/#installation","title":"Installation","text":"The easiest way to install the CGSE is to use the pip
command. Since the CGSE is a monorepo and consists of several packages, you will need to make your choice which package you need for your project. You can however start with the cgse-common
which contains all common code that is generic and useful as a basis for other packages.
$ pip install cgse-common\n
Check the list of packages that are part of the CGSE repo and can be installed with pip
. The packages are described in more detail in the sections Libs and Projects.
To check your installation and set up your environment, here are a few tips.
The version of the core packages and any plugin packages can be verified as follows. The version you installed will probably be higher and more lines will appear when other packages are installed.
$ py -m egse.version\nCGSE version in Settings: 2025.0.5\nInstalled version for cgse-common= 2025.0.5\n
Check your environment with the command below. This will probably print out some warning since you have not defined the expected environment variables yet. There are two mandatory environment variables: PROJECT
and SITE_ID
. The former shall contain the name of your project without spaces and preferably a single word or an acronym like PLATO, ARIEL, MARVEL, MERCATOR. The latter is the name of the site or lab where the tests will be performed. Good names are KUL, ESA, LAB23.
The other environment variables follow the pattern <PROJECT>_...
, i.e. they all start with the project name as defined in the PROJECT environment variable. You should define at least <PROJECT>_DATA_STORAGE_LOCATION
. The configuration data and log file location will be derived from it unless they are explicitly set themselves.
Let's define the three expected environment variables:
$ export PROJECT=ARIEL\n$ export SITE_ID=VACUUM_LAB\n$ export ARIEL_DATA_STORAGE_LOCATION=~/data\n
Rerunning the above command now gives:
$ py -m egse.env\nEnvironment variables:\n PROJECT = ARIEL\n SITE_ID = VACUUM_LAB\n ARIEL_DATA_STORAGE_LOCATION = /Users/rik/data\n ARIEL_CONF_DATA_LOCATION = not set\n ARIEL_CONF_REPO_LOCATION = not set\n ARIEL_LOG_FILE_LOCATION = not set\n ARIEL_LOCAL_SETTINGS = not set\n\nGenerated locations and filenames\n get_data_storage_location() = '/Users/rik/data/VACUUM_LAB' \u27f6 ERROR: The data storage location doesn't exist!\n get_conf_data_location() = '/Users/rik/data/VACUUM_LAB/conf' \u27f6 ERROR: The configuration data location doesn't exist!\n get_conf_repo_location() = None \u27f6 ERROR: The configuration repository location doesn't exist!\n get_log_file_location() = '/Users/rik/data/VACUUM_LAB/log' \u27f6 ERROR: The log files location doesn't exist!\n get_local_settings() = None \u27f6 ERROR: The local settings file is not defined or doesn't exist!\n\nuse the '--full' flag to get a more detailed report, '--doc' for help on the variables.\n
Note
The folders that do not exist (and are not None) can be created by adding the option --mkdir
to the above command.
Settings contains the static configuration of your system, test setup, equipment, including the System Under Test (SUT). You can test your settings with the command below. Let's first also set the ARIEL_LOCALSETTINGS
environment variables:
$ export ARIEL_LOCAL_SETTINGS=~/cgse/local_settings_ariel_vacuum_lab.yaml\n$ py -m egse.settings\nSettings\n\u251c\u2500\u2500 PACKAGES\n\u2502 \u2514\u2500\u2500 CGSE_COMMON: Common classes, functions, decorators, etc. for the CGSE\n\u251c\u2500\u2500 SITE\n\u2502 \u251c\u2500\u2500 ID: VACUUM_LAB\n\u2502 \u251c\u2500\u2500 SSH_SERVER: localhost\n\u2502 \u2514\u2500\u2500 SSH_PORT: 22\n\u2514\u2500\u2500 PROCESS\n \u2514\u2500\u2500 METRICS_INTERVAL: 10\nMemoized locations:\n['/Users/rik/tmp/gettings-started/venv/lib/python3.9/site-packages/cgse_common/settings.yaml', \n'/Users/rik/cgse/local_settings_ariel_vacuum_lab.yaml']\n
These Settings will grow when you add more packages to your installation or when you define settings yourself in the local settings file."},{"location":"help/","title":"Help","text":"The best way to get help for something that you couldn't find in the documentation on this site, is to contact one of the authors.
"},{"location":"help/#bugs-and-feature-requests","title":"Bugs and Feature requests","text":"Report any bugs or issues through GitHub on the CGSE issues page.
"},{"location":"initialize/","title":"Initialize your project","text":"So, we have seen how to get started with some basic commands and only the cgse-common
package. It's time now to initialize your project properly with all the necessary services.
I assume you are in the same environment that we have set up in the previous section where also the cgse-common
package was installed. We will install another package that will provide us with the full functionality of the cgse
command. Install the cgse-tools
package which depends on cgse-core
and will also install that package.
$ pip install cgse-tools\n
You should now have at least the follow three packages installed in your virtual environment:
$ pip list | grep cgse\ncgse-common 2025.0.8\ncgse-core 2025.0.8\ncgse-tools 2025.0.8\n
"},{"location":"initialize/#the-cgse-command","title":"The cgse
command","text":"The two new packages that have been installed (cgse-core
and cgse-tools
) provide the cgse
command that we will use to initialize your environment, but this command is also used to inspect different parts of the system, manage core services and device drivers, etc.
When you run the cgse
command without any arguments, it will show something like this:
$ cgse\n\n Usage: cgse [OPTIONS] COMMAND [ARGS]...\n\n The main cgse command to inspect, configure, monitor the core services and device control servers.\n\n\u256d\u2500 Options \u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u256e\n\u2502 --install-completion Install completion for the current shell. \u2502\n\u2502 --show-completion Show completion for the current shell, to copy it or customize the installation. \u2502\n\u2502 --help Show this message and exit. \u2502\n\u2570\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u256f\n\u256d\u2500 Commands \u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u256e\n\u2502 version Prints the version of the cgse-core and other registered packages. \u2502\n\u2502 init Initialize your project. \u2502\n\u2502 top A top-like interface for core services and device control servers. \u2502\n\u2502 core handle core services: start, stop, status \u2502\n\u2502 show Show information about settings, environment, setup, ... \u2502\n\u2502 check Check installation, settings, required files, etc. \u2502\n\u2502 dev-x device-x is an imaginary device that serves as an example \u2502\n\u2570\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u256f\n
The cgse
command is actually an app that is the starting point for a number of commands that can be used to maintain the system, manage and inspect services and devices. For example, to check the version of the different components, use: $ cgse version\nCGSE-COMMON installed version = 2025.0.8\nCGSE-CORE installed version = 2025.0.8\nCGSE-TOOLS installed version = 2025.0.8\n
We will for now concentrate on the init
command. This command will guide us through a number of steps to define the location of our device data, configuration data, etc. We will basically define some environment variables that are used by the CGSE framework. The PROJECT is he name of the project your will be working on, the SITE_ID is the identifier for the LAB or Setup that you are using to perform the tests. As you see below, the environment variables all start with the project name allowing you to work on different projects simultaneously. If you accept all the defaults, the result of the cgse init
command will look something like this:
$ cgse init --project marvel\nPlease note default values are given between [brackets].\nWhat is the name of the project [MARVEL] ?:\nWhat is the site identifier ?: lab02\nWhere can the project data be stored [~/data/MARVEL/LAB02/] ?:\nWhere will the configuration data be located [~/data/MARVEL/LAB02/conf/] ?:\nWhere will the logging messages be stored [~/data/MARVEL/LAB02/log/] ?:\nWhere shall I create a local settings YAML file [~/data/MARVEL/LAB02/local_settings.yaml] ?:\nShall I add the environment to your ~/bash_profile ? [y/n]: n\n\n# -> Add the following lines to your bash profile or equivalent\n\nexport PROJECT=MARVEL\nexport SITE_ID=LAB02\nexport MARVEL_DATA_STORAGE_LOCATION=~/data/MARVEL/LAB02/\nexport MARVEL_CONF_DATA_LOCATION=~/data/MARVEL/LAB02/conf/\nexport MARVEL_LOG_FILE_LOCATION=~/data/MARVEL/LAB02/log/\nexport MARVEL_LOCAL_SETTINGS=~/data/MARVEL/LAB02/local_settings.yaml\n
If you answered 'Y' to the last question, you should log in to the shell again with exec bash -login
or a similar command for other shells, or you should start a new terminal to activate the environment variables. Add this point you are ready to go and start the core services and any device control servers that you need. You can explore other commands of the cgse
app in the user guide.
The CGSE is a monorepo and consists of numerous packages. Each of these packages are individually installable from PyPI. We maintain a list here with all the packages in the monorepo.
Package Descriptioncgse-common
The common code used by all other packages cgse-core
The core services cgse-coordinates
Coordinate reference Frames cgse-gui
GUI components and styles (PyQt5) cgse-tools
Plugin that adds functions to the cgse
command symetrie-hexapod
Device drivers for the Sym\u00e9trie Hexapods keithley-tempcontrol
Device driver for the Keithley temperature controller plato-fits
FITS driver with PLATO specific format plato-hdf5
HDF5 driver with PLATO specific format plato-spw
SpaceWire driver with PATO specific packets The following is a non-exhaustive list of known external packages that work well with the CGSE in terms of commanding and monitoring.
Package Descriptioncgse-dummy
Provides a dummy device driver to demonstrate plugins, commands and how to develop an external package for the CGSE."},{"location":"tutorial/","title":"Tutorial","text":"Welcome to the CGSE Tutorial!
By the end of this page you should have a solid understanding of the core features of the CGSE.
"},{"location":"dev_guide/","title":"Developer Guide","text":"Welcome to the CGSE developer guide! An in-depth reference on how to contribute to the CGSE.
First thing to know is that this repository is actually a monorepo, meaning it contains a bunch of related but self-standing packages with a minimum of interdependencies. These packages are
"},{"location":"dev_guide/coding_style/","title":"Style Guide","text":"This part of the developer guide contains instructions for coding styles that are adopted for this project.
The style guide that we use for this project is PEP8. This is the standard for Python code and all IDEs, parsers and code formatters understand and work with this standard. PEP8 leaves room for project specific styles. A good style guide that we can follow is the Google Style Guide.
The following sections will give the most used conventions with a few examples of good and bad.
"},{"location":"dev_guide/coding_style/#tldr","title":"TL;DR","text":"Type Style Example Classes CapWords ProcessManager, ImageViewer, CommandList, Observation, MetaData Methods & Functions lowercase with underscores get_value, set_mask, create_image Variables lowercase with underscores key, last_value, model, index, user_info Constants UPPERCASE with underscores MAX_LINES, BLACK, COMMANDING_PORT Modules & packages lowercase no underscores dataset, commanding, multiprocessing"},{"location":"dev_guide/coding_style/#general","title":"General","text":"user_list
might look good at first, but what if at some point you want to change the list to a set so it can not contain duplicates. Are you going to rename everything into user_set
or would user_info
be a better name?SyntaxError: invalid syntax
.Always use CamelCase (Python uses CapWords) for class names. When using acronyms, keep them all UPPER case.
Manager
or Controller
or Organizer
all mean similar things. Choose one word for the concept and stick to it.Good names are: Observation
, CalibrationFile
, MetaData
, Message
, ReferenceFrame
, URLParser
.
A function or a method does something (and should only do one thing, SRP=Single Responsibility Principle), it is an action, so the name should reflect that action.
Always use lowercase words separated with underscores.
Good names are: get_time_in_ms()
, get_commanding_port()
, is_connected()
, parse_time()
, setup_mask()
.
When working with legacy code or code from another project, names may be in camelCase (with the first letter a lower case letter). So we can in this case use also getCommandPort()
or isConnected()
as method and function names.
Use the same naming convention as functions and methods, i.e. lowercase with underscores.
Good names are: key
, value
, user_info
, model
, last_value
Bad names: NSegments
, outNoise
Take care not to use builtins: list
, type
, filter
, lambda
, map
, dict
, ...
Private variables (for classes) start with an underscore: _name
or _total_n_args
.
In the same spirit as method and function names, the variables can also be in camelCase for specific cases.
"},{"location":"dev_guide/coding_style/#constants","title":"CONSTANTS","text":"Use ALL_UPPER_CASE with underscores for constants. Use constants always within a name space, not globally.
Good names: MAX_LINES
, BLACK
, YELLOW
, ESL_LINK_MODE_DISABLED
Use simple words for modules, preferably just one word like datasets
or commanding
or storage
or extensions
. If two words are unavoidable, just concatenate them, like multiprocessing
or sampledata
or testdata
. If needed for readability, use an underscore to separate the words, e.g. image_analysis
.
from <module> import *
Be careful that you do not name any modules the same as a module in the Python standard library. This can result in strange effects and may result in an AttributeError
. Suppose you have named a module math
in the egse
directory and it is imported and used further in the code as follows:
from egse import math\n\n# in some expression further down the code you might use\n\nmath.exp(a)\n
This will result in the following runtime error:
File \"some_module.py\", line 8, in <module>\n print(math.exp(a))\nAttributeError: module 'egse.math' has no attribute 'exp'\n
Of course this is an obvious example, but it might be more obscure like e.g. in this GitHub issue: 'module' object has no attribute 'Cmd'.
"},{"location":"dev_guide/docs/","title":"Building the documentation","text":"mkdocs serve
from the project root olderdocs/*
I created a virtual environment using pyenv
and when I'm working on the documentation, I start up a shell with this environment. Currently, only mkdocs
and mkdocs-material
are needed. Of course, you need to install these only once.
$ pyenv virtualenv 3.10 cgse-doc-3.10\n$ pyenv shell cgse-doc-3.10\n$ pip install --upgrade pip setuptools wheel\n$ pip install mkdocs\n$ pip install mkdocs-material\n
From this shell, navigate to the project root folder and start the live-reload server of mkdocs
.
$ cd ~/github/cgse\n$ mkdocs serve\n
Now you can update files, create new folders in docs/*
, create new Markdown files and all changes will be reloaded live in the browser.
When you are ready with updating, you will need to build the site and publish it on GitHub pages:
$ mkdocs build\n$ mkdocs gh-deploy -r upstream -m \"documentation update on ..\"\n
"},{"location":"dev_guide/docs/#commands","title":"Commands","text":"mkdocs serve
\u2014 start the live-reloading docs servermkdocs build
\u2014 build the documentation sitemkdocs deploy
\u2014 publish your documentation on GitHub pagesmkdocs -h
\u2014 print a help message for more optionsmkdocs.yml # the mkdocs configuration file\ndocs/\n index.md # the documentation homepage\n ... # other markdown pages, image, folders, ...\n
"},{"location":"dev_guide/installation/","title":"Installation Guide for Developers","text":""},{"location":"dev_guide/plugins/","title":"Plugins","text":"The CGSE is designed to be extensible and uses a few plugin mechanisms to extend its functionally with external contributions. Also within the cgse
monorepo we use the plugin mechanism at several places. The following entry-points are currently defined:
cgse.version
: Each package that provides functionality within the CGSE or adds a device driver registers itself to provide version information.cgse.command.plugins
: Packages can add commands or sub-commands to the cgse
app to manage their functionality from within the cgse
app, e.g. to start or stop the service or to report on its status.cgse.settings
: Package can add their own settings.cgse.explore
: Package provides a set of functions to explore, e.g. if any of the processes it provides are running.cgse.resource
: Packages can register resources.Each of the entry-points knows how to load a module or object and each entry-point group is connected to a specific action or plugin hook like, e.g. add a command or command group to the cgse
app, add package specific settings to the global settings.
When you write a package that you want to integrate with the CGSE, provide a cgse.version
entry-point. The name of the entry-point shall match the package name and is used to read the version from the importlib metadata. The entry-point value is currently not used. The entry-point value can optionally provide additional information about the package, but that is currently not specified.
Add the following to your pyproject.toml
file in your project's root folder, replacing package-name with the name of your project. The entry-point value is currently not used, but you want to use a valid format, the value below is always valid.
[project.entry-points.\"cgse.version\"]\npackage-name = 'egse.version:get_version_installed'\n
"},{"location":"dev_guide/plugins/#extending-the-cgse-app","title":"Extending the cgse
app","text":""},{"location":"dev_guide/plugins/#add-a-command","title":"Add a Command","text":"If your package provides specific functionality that can be added as a command or a command group to the cgse
app, use the cgse.command
entry-point group. Since the cgse
app uses the Typer package to build its commandline interface, adding a command is as simple as writing a function. The function will be added to the cgse
app using the app.command()
function of Typer
, making the function a top-level command of the cgse
app. The function can be defined as a plain function or with Typer's @app.command
decorator.
In the pyproject.toml
file of your project, add the following lines to add the CGSE command:
[project.entry-points.\"cgse.command\"]\nname = 'module:object'\n
Where:
name
is the name of the commandmodule
is a fully qualified module name for your package, a module that can be importedobject
is the name of the function that you want to add as a commandAs an example, for the cgse-tools
package, the init
command of the cgse
app is listed in the pyproject.toml
file as follows:
[project.entry-points.\"cgse.command\"]\ninit = 'cgse_tools.cgse_commands:init'\n
The init
function is defined in the cgse_commands.py
module which is located in the cgse_tools
module in the src
folder of the package:
src\n\u251c\u2500\u2500 cgse_tools\n\u2502 \u251c\u2500\u2500 __init__.py\n\u2502 \u251c\u2500\u2500 cgse_commands.py\n...\n
"},{"location":"dev_guide/plugins/#add-a-command-group","title":"Add a Command group","text":"Some commands are more complicated and define a number of sub-commands. An example is the show
command where you currently have the sub-commands env
and settings
$ cgse show --help\n\n Usage: cgse show [OPTIONS] COMMAND [ARGS]...\n\n Show information about settings, environment, setup, ...\n\n\u256d\u2500 Options \u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u256e\n\u2502 --help Show this message and exit. \u2502\n\u2570\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u256f\n\u256d\u2500 Commands \u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u256e\n\u2502 settings Show the settings that are defined by the installed packages. \u2502\n\u2502 env Show the environment variables that are defined for the project. \u2502\n\u2570\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u256f\n
The show
command is defined as a typer.Typer()
object where env
and settings
are added using the decorator @<app>.command()
.
import typer\n\nshow = typer.Typer(help=\"Show information about settings, environment, setup, ...\")\n\n\n@show.command(name=\"settings\")\ndef show_settings():\n ...\n\n\n@show.command(name=\"env\")\ndef show_env():\n ...\n
To add this command group to the cgse
app, the following entry was used in the pyproject. toml
file of the cgse-tools
project. Notice the [group]
at the end of the entry which indicates this is a command group instead of a single command.
[project.entry-points.\"cgse.command\"]\nshow = 'cgse_tools.cgse_commands:show[group]'\n
"},{"location":"dev_guide/plugins/#add-a-service","title":"Add a Service","text":"If your package provides a device driver or a specific service, use the cgse.service
entry-point group. Service entry-points follow the same scheme as command groups, i.e. they are added to the cgse
app as a Typer()
object. Use the following entry in your pyproject.toml
file:
[project.entry-points.\"cgse.service\"]\nname = 'module:object'\n
where:
name
is the name of the service or device drivermodule
is a fully qualified module name for your package, a module that can be importedobject
is the name of the Typer()
object that you want to add as a serviceThe entry-point cgse.explore
can be used to extend functionality without adding a new command or sub-command to the cgse
app. The idea is that commands that work on different packages can use this entry-point to perform certain tasks on the package. This is currently used for the show procs
command (see below).
The entry-point has the following format:
[project.entry-points.\"cgse.explore\"]\nexplore = \"<package>.cgse_explore\"\n
So, what happens is that a command that wants to apply a functionality on an external package loads the cgse_explore.py
module for that package and checks if a function with a specific name exists in that module. It then executes that function. For the show procs
command, the function show_processes
is expected and it shall return a list of strings which currently are printed to the terminal. This entry-point is currently implemented for cgse-core
and cgse-dummy
(an external demo package) and when you run the cgse show procs
command it looks something like below (the format is from the unix ps -ef
command).
\u279c cgse show procs\n459800007 76849 1 0 11:07PM ttys003 0:03.53 /Users/rik/tmp/test_dummy/venv/bin/python3.9 -m egse.logger.log_cs start\n459800007 76850 1 0 11:07PM ttys003 2:18.60 /Users/rik/tmp/test_dummy/venv/bin/python3.9 -m egse.storage.storage_cs start\n459800007 76851 1 0 11:07PM ttys003 2:20.10 /Users/rik/tmp/test_dummy/venv/bin/python3.9 -m egse.confman.confman_cs start\n459800007 13825 1 0 4:31PM ttys003 0:02.97 /Users/rik/tmp/test_dummy/venv/bin/python3.9 -m cgse_dummy.dummy_sim start\n
"},{"location":"dev_guide/plugins/#register-resources","title":"Register resources","text":"TODO: what if two packages provide a resource icons
?
uv
","text":"uv
is an extremely fast Python package and project manager, written in Rust. We will use uv
as the single tool that replaces pip
, virtualenv
, pyenv
, and more. The main tasks for which we will use uv
are:
uv
","text":"On macOS and Linux you can install uv
using curl
:
$ curl -LsSf https://astral.sh/uv/install.sh | sh\n
If you need more specific information on installing and upgrading uv
, please refer to the official documentation.
The CGSE is guaranteed to work with Python 3.9.x. We will gradually include higher versions of Python, but currently these have not been tested. So, we will for the moment stick with Python 3.9.20. Install this version as follows:
$ uv python install 3.9.20\n
pyenv
When you are using pyenv
to manage your Python versions, make sure you also have the same Python version installed with pyenv
and uv
. Otherwise you will run into the following error. This is a known issue with uv
.
pyenv: version `3.9.20' is not installed (set by /Users/rik/github/cgse/libs/cgse-common/.python-version)\n
You can check which Python versions are installed already on your system:
CommandOutput$ uv python list --only-installed\n
cpython-3.12.8-macos-aarch64-none /Users/rik/Library/Application Support/uv/python/cpython-3.12.8-macos-aarch64-none/bin/python3.12\ncpython-3.10.16-macos-aarch64-none /Users/rik/Library/Application Support/uv/python/cpython-3.10.16-macos-aarch64-none/bin/python3.10\ncpython-3.9.21-macos-aarch64-none /Users/rik/Library/Application Support/uv/python/cpython-3.9.21-macos-aarch64-none/bin/python3.9\ncpython-3.9.20-macos-aarch64-none /Users/rik/Library/Application Support/uv/python/cpython-3.9.20-macos-aarch64-none/bin/python3.9\ncpython-3.9.6-macos-aarch64-none /Library/Developer/CommandLineTools/usr/bin/python3 -> ../../Library/Frameworks/Python3.framework/Versions/3.9/bin/python3\ncpython-3.8.17-macos-aarch64-none /Users/rik/Library/Application Support/uv/python/cpython-3.8.17-macos-aarch64-none/bin/python3.8\n
"},{"location":"dev_guide/uv/#create-a-virtual-environment","title":"Create a virtual environment","text":"Pin a Python version
You can pin a python version with the command:
$ uv python pin 3.9.20\n
uv
will search for a pinned version in the parent folders up to the root folder or your home directory.
You can create a virtual environment with uv
for the specific Python version as follows. The '--python
' is optional and uv
will use the default (pinned) Python version when creating a venv
without this option. We are working in a monorepo or what uv
calls a workspace. There will be only one virtual environment at the root of the monorepo, despite the fact that we have several individual packages in our workspace. Don't worry, uv
will always use the virtual environment at the root and keep it up-to-date with the project your are currently working in.
When creating a virtual environment make sure you are in the root folder, e.g. ~/github/cgse
.
$ cd ~/github/cgse\n$ uv venv --python 3.9.20\n
Now, navigate to the package you will be working in and update the projects' environment, assuming you are going to work in cgse-core
, this will be:
$ cd ~/github/cgse/libs/cgse-core\n$ uv sync\n
Your package(s) from the workspace should be installed as an editable install. You can check this with the command:
$ uv pip list -v\nUsing Python 3.9.20 environment at: /Users/rik/github/cgse/.venv\nPackage Version Editable project location\n----------------- ----------- ---------------------------------------\napscheduler 3.11.0\ncgse-common 0.4.0 /Users/rik/github/cgse/libs/cgse-common\ncgse-core 0.4.0 /Users/rik/github/cgse/libs/cgse-core\n...\n
To install any other project as an editable package:
$ uv pip install -e <project location>\n
Note
If you don't want to use the uv
commands, you can activate the virtual environment and use the original pip
and python
commands as you are used to, but I would recommend you try to get used to uv
for a while to experience its benefits.
$ source ~/github/cgse/.venv/bin/activate\n
Info
In a workspace, maintaining a virtual environment per package might be a hassle and most of the time that is not needed. A good approach is to always use the virtual environment at the workspace root. This venv
which will be automatically created if you run a command or if you use uv sync
in the package folder. With uv sync
you can make sure the virtual environment is up-to-date and contains only those dependencies that are required for the package you are in. So, each time you switch to another package and want to run a comand or a test for that package, use
$ uv sync\n
"},{"location":"dev_guide/uv/#building-and-publishing-all-packages","title":"Building and publishing all packages","text":"We have chosen for one and the same version number for all packages in the cgse
monorepo. That means that whenever we make a change to one of the packages and want to release that change, all packages shall be rebuild and published.
Inline
When working in a workspace, keep in mind that the commands uv run
and uv sync
by default work on the workspace root. That means that when you run the uv run pip install <package>
command, the .venv
at the workspace root will be updated or created if it didn't exist. Similar for the uv sync
command, there is only one uv.lock
file at the root of the workspace.
Fortunately, with uv
, that is done in a few commands.
When you are in the monorepo root folder, you can build all packages at once. They will be placed in the dist
folder of the root package. Before building, make sure you update the version in the pyproject.toml
of the root package and then bump the versions. Before building, clean up the dist
folder, then you can do a default uv publish
afterwards.
$ cd <monorepo root>\n$ uv run bump.py\n$ rm -r dist\n$ uv build --all-packages\n
Publish all packages in the root dist folder to PyPI. The UV_PUBLISH_TOKEN can be defined in a (read protected) ~/. setenv.bash file:
$ uv publish --token $UV_PUBLISH_TOKEN\n
The above command will publish all package to PyPI. If you don't want the token to be in a shell variable, you can omit the --token
in the command above. You will then be asked for a username, use __token__
as the username and then provide the token as a password.
The libraries are those packages that make up the CGSE framework.
The libraries are located under the libs
folder, and we currently find the following packages there:
cgse-common
cgse-core
cgse-coordinates
cgse-gui
This package cgse-common
contains modules that are used by all other packages.
egse.bits
convenience functions to work with bits, bytes and integers egse.calibration
functions to handle conversions and apply correction egse.command
classes and functions to work with commands that operate hardware devices egse.config
convenience functions to configure the system and find folders and files egse.control
defines abstract classes and convenience functions for any control server egse.decorators
a collection of useful decorator functions egse.device
defines the generic interfaces to connect devices egse.env
functionality to work with and check your environment variables egse.exceptions
common Exceptions and Errors egse.hk
functions to retrieve and convert housekeping parameter values egse.metrics
functions to define and update metrics egse.mixin
defines the mixin classes for dynamic commanding egse.monitoring
the monitoring application / function egse.observer
the classic observer and observable egse.obsid
functions to define and work with the OBSID egse.persistence
the persistence layer interface egse.plugin
functions to load plugins and settings from entry-points egse.process
functions and classes to work with processes and sub-processes egse.protocol
base class for communicating commands with the hardware or the control server egse.proxy
base class for the Proxy objects for each device controller egse.reload
a slightly better approach to reloading modules and function egse.resource
convenience functions to use resources in your code egse.response
defines the classes to handle responses from the control servers egse.services
provides the services to the control servers egse.settings
provides functions to handle user and configuration settings egse.setup
defines the Setup, containing the complete configuration for a test egse.state
classes and functions to handle state, e.g. the GlobalState egse.system
convenience functions that provide information on system specific functionality egse.version
functions to load specific version information egse.zmq_ser
serialization function used in a ZeroMQ context"},{"location":"libs/cgse-common/settings/","title":"The Settings","text":"The Settings class contains all static information needed to configure your system, the environment you are using and the test equipment. The Settings also contain all the IP addresses and port number for all the known devices, together with other static information like the device name, default settings for the device like speed, timeout, delay time, firmware version, etc. We will go into more details about the content later, let\u2019s now first look at the format and usage of the Settings.
"},{"location":"libs/cgse-common/settings/#loading-the-settings","title":"Loading the Settings","text":"The Settings can be loaded as follows:
>>> from egse.settings import Settings\n>>> settings = Settings.load()\n
The settings
object will be a dictionary where the keys are the top-level groups that are defined in the settings for each package. For a system that has only cgse-common
and cgse-core
installed, the settings
will contain something like this:
>>> print(settings)\nSettings\n\u251c\u2500\u2500 PACKAGES\n\u2502 \u251c\u2500\u2500 CGSE_COMMON: Common classes, functions, decorators, etc. for the CGSE\n\u2502 \u2514\u2500\u2500 CGSE_CORE: The core services of the CGSE\n\u251c\u2500\u2500 SITE\n\u2502 \u251c\u2500\u2500 ID: LAB42\n\u2502 \u251c\u2500\u2500 SSH_SERVER: localhost\n\u2502 \u2514\u2500\u2500 SSH_PORT: 22\n\u251c\u2500\u2500 PROCESS\n\u2502 \u2514\u2500\u2500 METRICS_INTERVAL: 10\n\u251c\u2500\u2500 Logging Control Server\n\u2502 \u251c\u2500\u2500 PROTOCOL: tcp\n\u2502 \u251c\u2500\u2500 HOSTNAME: localhost\n\u2502 \u251c\u2500\u2500 LOGGING_PORT: 7000\n\u2502 \u251c\u2500\u2500 COMMANDING_PORT: 7001\n\u2502 \u251c\u2500\u2500 METRICS_PORT: 7003\n\u2502 \u251c\u2500\u2500 MAX_NR_LOG_FILES: 20\n\u2502 \u251c\u2500\u2500 MAX_SIZE_LOG_FILES: 20\n\u2502 \u251c\u2500\u2500 TEXTUALOG_IP_ADDRESS: 127.0.0.1\n\u2502 \u2514\u2500\u2500 TEXTUALOG_LISTENING_PORT: 19996\n\u251c\u2500\u2500 Configuration Manager Control Server\n\u2502 \u251c\u2500\u2500 PROTOCOL: tcp\n\u2502 \u251c\u2500\u2500 HOSTNAME: localhost\n\u2502 \u251c\u2500\u2500 COMMANDING_PORT: 6000\n\u2502 \u251c\u2500\u2500 MONITORING_PORT: 6001\n\u2502 \u251c\u2500\u2500 SERVICE_PORT: 6002\n\u2502 \u251c\u2500\u2500 METRICS_PORT: 6003\n\u2502 \u251c\u2500\u2500 DELAY: 1\n\u2502 \u2514\u2500\u2500 STORAGE_MNEMONIC: CM\n\u2514\u2500\u2500 Storage Control Server\n \u251c\u2500\u2500 PROTOCOL: tcp\n \u251c\u2500\u2500 HOSTNAME: localhost\n \u251c\u2500\u2500 COMMANDING_PORT: 6100\n \u251c\u2500\u2500 MONITORING_PORT: 6101\n \u251c\u2500\u2500 SERVICE_PORT: 6102\n \u251c\u2500\u2500 METRICS_PORT: 6103\n \u2514\u2500\u2500 DELAY: 1\n
If you only need the settings for a particular component, specify that group's name:
>>> storage_settings = Settings.load(\"Storage Control Server\")\n\n>>> print(storage_settings)\nStorage\nControl\nServer\n\u251c\u2500\u2500 PROTOCOL: tcp\n\u251c\u2500\u2500 HOSTNAME: localhost\n\u251c\u2500\u2500 COMMANDING_PORT: 6100\n\u251c\u2500\u2500 MONITORING_PORT: 6101\n\u251c\u2500\u2500 SERVICE_PORT: 6102\n\u251c\u2500\u2500 METRICS_PORT: 6103\n\u2514\u2500\u2500 DELAY: 1\n
The values can be accessed as usual with a dictionary, by specifying the name of the parameter as the key:
>>> print(storage_settings[\"COMMANDING_PORT\"])\n6100\n
We usually only go one level deep when defining settings, and as a convenience, that first level of variables can also be accessed with the dot-notation.
>>> print(storage_settings.COMMANDING_PORT)\n6100\n
"},{"location":"libs/cgse-common/settings/#entry-points","title":"Entry-points","text":"The Settings are collected from a set of YAML files which are provided by the packages through the entry-point cgse.settings
. The default Settings file is named settings.yaml
but this can be changed by the entry-point (see below).
Let's take a look at how the settings are provided for the cgse-core
package. First, the pyproject.toml
file of the project shall define the entry-point. In the snippet below, the entry-point cgse-core
is defined for the group cgse.settings
.
[project.entry-points.\"cgse.settings\"]\ncgse-core = \"cgse_core:settings.yaml\"\n
The entry-point itself has the following format: <name> = \"<module>.<filename>\"
, where
<name>
is the name of the entry-point given in the pyproject.toml
file, usually this is the package name,<module>
is a valid module name that can be imported and from which the location can be determined, and<filename>
is the name of the target file, e.g. a YAML file.Note
The module name for this entry point has an underscore instead of a dash, i.e. cgse_core
instead of cgse-core
. The reason is that module names with a dash will generate a SyntaxError during import.
The above example will load the settings for this package from the settings.yaml
file that is located in the cgse_core
module. That is, the package shall also provide this as follows:
cgse-core\n\u251c\u2500\u2500 pyproject.toml\n\u2514\u2500\u2500 src\n \u2514\u2500\u2500 cgse_core\n \u251c\u2500\u2500 __init__.py\n \u2514\u2500\u2500 settings.yaml\n
The settigs.yaml
file for this module looks something like this:
PACKAGES:\n CGSE_CORE: The core services of the CGSE\n\nLogging Control Server: # LOG_CS\n\n PROTOCOL: tcp\n HOSTNAME: localhost # The hostname that client shall connect to, e.g. pleiad01 @ KU Leuven\n LOGGING_PORT: 7000\n COMMANDING_PORT: 7001\n METRICS_PORT: 7003 # The HTTP port where Prometheus will connect to for retrieving metrics\n MAX_NR_LOG_FILES: 20 # The maximum number of log files that will be maintained in a roll-over\n MAX_SIZE_LOG_FILES: 20 # The maximum size one log file can become\n TEXTUALOG_IP_ADDRESS: 127.0.0.1 # The IP address of the textualog listening server\n TEXTUALOG_LISTENING_PORT: 19996 # The port number on which the textualog server is listening\n\nConfiguration Manager Control Server: # CM_CS\n\n ...\n
"},{"location":"libs/cgse-common/settings/#local-settings","title":"Local Settings","text":"You can, and you should, define local settings for your project and put those settings in a known folder on your system. The usual place is ~/cgse/local-settings.yaml
. This file will be automatically loaded by the Settings.load()
function when you define the local settings environment variable. That variable name is <PROJECT>_LOCAL_SETTINGS
where <PROJECT>
is the name of your project as defined by the PROJECT
environment variable. For a PROJECT=LAB23
the local settings would be defined as follows:
$ export LAB23_LOCAL_SETTINGS=~/cgse/local-settings-lab23.yaml\n
The local settings take higher precedence that will overwrite the global settings when loaded. You only need to define the settings that actually change for your local installation, respect the full hierarchy when specifying those settings. You are allowed to define new entries at any level in the Settings hierarchy.
The usual parameters to put into a local settings file are:
You can check the current settings from the terminal with the following command:
$ py -m egse.settings\nSettings\n\u251c\u2500\u2500 PACKAGES\n\u2502 \u251c\u2500\u2500 CGSE_COMMON: Common classes, functions, decorators, etc. for the CGSE\n\u2502 \u2514\u2500\u2500 CGSE_CORE: The core services of the CGSE\n\u251c\u2500\u2500 SITE\n\u2502 \u251c\u2500\u2500 ID: LAB42\n\u2502 \u251c\u2500\u2500 SSH_SERVER: localhost\n\u2502 \u2514\u2500\u2500 SSH_PORT: 22\n\u251c\u2500\u2500 PROCESS\n\u2502 \u2514\u2500\u2500 METRICS_INTERVAL: 10\n\u251c\u2500\u2500 Logging Control Server\n\u2502 \u251c\u2500\u2500 PROTOCOL: tcp\n\u2502 \u251c\u2500\u2500 HOSTNAME: localhost\n\u2502 \u251c\u2500\u2500 LOGGING_PORT: 7000\n\u2502 \u251c\u2500\u2500 COMMANDING_PORT: 7001\n... ...\n\u2514\u2500\u2500 Storage Control Server\n \u251c\u2500\u2500 PROTOCOL: tcp\n \u251c\u2500\u2500 HOSTNAME: localhost\n \u251c\u2500\u2500 COMMANDING_PORT: 6100\n \u251c\u2500\u2500 MONITORING_PORT: 6101\n \u251c\u2500\u2500 SERVICE_PORT: 6102\n \u251c\u2500\u2500 METRICS_PORT: 6103\n \u2514\u2500\u2500 DELAY: 1\nMemoized locations:\n['/Users/rik/github/cgse/libs/cgse-common/src/cgse_common/settings.yaml', '/Users/rik/github/cgse/libs/cgse-core/src/cgse_core/settings.yaml', '/Users/rik/cgse/local_settings_ariel.yaml']\n
The memoized locations are the settings files that have been loaded and cached. Once the application has started and the settings have been loaded, they can only be reloaded by explicitly forcing a reload as follows:
>>> settings = Settings.load(force=True)\n
Warning
The force
reload does however not guarantee that the settings will propagate properly throughout the application or to client apps. Settings can be saved in local variables or class instances that have no knowledge of a Settings reload. So, be careful when changing your Settings. If there are parameters that change often and are not as static as thought, maybe they belong in the Setup instead of the Settings. Examples are:
load()
method","text":"A word about the Settings.load()
method. Depending on the parameters provided, this method either loads all settings, a group of settings or just one single YAML file. We have already explained how to load a specific group of settings by giving the name of the group as a parameter. When you want to load just one YAML file, you need to specify its location also. When a location is given as a str or a Path, the Settings will be loaded from that file only, using the default settings.yaml
name or another name given through the filename
argument.
This can be used to e.g. load command files for a device:
>>> commands = Settings.load(location=\"~\", filename=\"DAQ5610.yaml\")\n
The mechanism behind the Settings.load()
method is shown in the following diagram. For simplicity, parameters are not shown and only the success path is presented, not any exceptions or error handling.
The projects are those packages that add functionality to the CGSE framework.
The projects live under the folder projects
, and they are organised in generic and specific projects. Generic projects do not have an implementation that is specific for one particular project, while, obviously, specific projects have. We currently have the following generic packages:
cgse-tools
symetrie-hexapod
and then there are the project specific packages:
plato-fits
plato-hdf5
plato-spw
Welcome to the CGSE user guide! An in-depth reference on how to use the CGSE.
"}]} \ No newline at end of file