channel and commands in YAML files

[elmjag]

Give the following XML configuration file:

<object class="SomeClass">
  <tangoname>some/tango/device</tangoname>
  <command type="tango" name="Open">Open</command>
  <channel type="tango" name="State" polling="1000">State</channel>
</object>

How would it be expressed as YAML config file? Did we discuss it at some point?

Currently, when generating YAML files with mxcubeweb-server's --export-yaml-config flag, the channel and command tags are just ignored.

[beteva]

I do not recall discussing this and we should have a really good look, as it is not only Tango, but also Exporter, probably Epics and all the rest using the commands/channels mechanism.

[rhfogh]

I do not think it was ever discussed, either. And I am pretty sure I never looked at it. The exported yaml config files deal with properties and contained objects, but channels and commands are handled separately in the XML world and are simply not looked at in the conversion.

I think the commands and channels need their own, well-thought-out handling in the yaml world, and that they should be treated as special cases - as they already are in the XML world.

One possibility might be separate attributes 'channels' and 'commands' in parallel with 'objects' and 'configuration', and putting a list of dictionaries in with a dictionary for every channel and every command. One could write the conversion code once to re-export from XML it was decided how the end result should look. But it would have to be the people who understand commands and channels who look at this.

[elmjag]

We did some light discussions on this topic, and came up with following suggestion.

Let's add a custom section for each protocol, that define command and channel bindings. The structure of each protocol section will have following format:

class: MegaCommunicator        
<protocol>:
  <end-point>:                   # E.g. tango device / exporter address / EPICS prefix / etc
    commands:
      <command-1-name>:          # MXCuBE name
        <config-prop-1>: <val-1>
        <config-prop-2>: <val-2>        
      <command-2-name>:          # command without any config properties
    channels:
      <channel-1-name>:          # MXCuBE Name
        <config-prop-1>: <val-1>
        <config-prop-2>: <val-2>
      <channel-2-name>:          # channel without any config properties

An example how you would configure Tango commands and channels.

class: TangoShutter.TangoShutter

configuration:
  values: something

tango:
  b312a-eh/dia/detc-01:    # tango device server name
    commands:
      Open:                # MXCuBE name
        name: Open         # tango command name, optional, default is MXCuBE name
      Close:  
    channels:
      State:               # MXCuBE name
      Status:              # MXCuBE name
        attribute: Status  # tango attribute name, optional, default is MXCuBE name
        polling: 1000

An example how you would configure exporter command and channels.

class: My.MD3

configuration:
  values: something

exporter:
  "192.168.2.44:9001":      # exporter address
    commands:
      Abort:                # MXCuBE name
        name: abort         # exporter command name, optional, default is MXCuBE name
      startRasterScan:
    channels:
      AperturePosition:      
      Phase:
        name: CurrentPhase  # use exporter 'CurrentPhase' attribute
      State: 

An example of binding MXCuBE channels to EPICS Process Variables:

class: foo.bar.Epic

configuration:
  model: FooBar
  
epics:
  SOL:S:m1.:  # prefix
    channels:
      epicsActuator_val:  # MXCuBE name
        name: VAL         # will bind to 'SOL:S:m1.VAL' PV
      epicsActuator_rbv:
        name: RBV         # will bind to 'SOL:S:m1.RBV' PV
        polling: 500

This format allows to write configs where <end-point> and protocol type are not repeated. The implied attribute/command/PV names cuts down on repetition as well.

[rhfogh]

Not my expertise, so I am not qualified to judge. It does look nice, though.

[laispc]

@elmjag , thanks for sharing this discussion. I agree that the suggested structure looks cleaner, reducing a lot of repetition. :) I would just make a small change to better fit it for EPICS (which will be used here at ESS, and is already used by LNLS/Sirius and some other members).

The key does not always apply to us. It's indeed a good practice to have a common prefix for the PVs, like in this one:

https://github.com/mxcube/mxcubecore/blob/develop/mxcubecore/configuration/lnls_manaca/udiff_kappa.xml

However, it's not guaranteed in cases when a device is, for example, controlled by more than one EPICS IOC:

https://github.com/mxcube/mxcubecore/blob/develop/mxcubecore/configuration/lnls_manaca/detector.xml

So, in your example I would change the key to make it more flexible (without breaking the other cases), like this:

class: foo.bar.EPICSMotor

configuration:
  values: something
  
epics:
  end_point: SOL:S:m1.  # prefix
  channels:
    epicsActuator_val:     # MXCuBE name
      name: VAL                 # will bind to 'SOL:S:m1.VAL' PV
    epicsActuator_rbv:
      name: RBV                # will bind to 'SOL:S:m1.RBV' PV
      polling: 500

For cases where we don't have the (like in lnls_manaca/detector.xml), we could set as None (or maybe just omit it?), like this:

class: foo.bar.LNLSPilatusDet

configuration:
  values: something
  
epics:
  end_point: None  # no prefix!
  channels:
    det_threshols_energy:                           # MXCuBE name
      name: MX2:cam1:ThresholdEnergy  # will bind to 'MX2:cam1:ThresholdEnergy' PV
    user_beam_x:
      name: MNC:S:SOFT:ao:BeamX           # will bind to 'MNC:S:SOFT:ao:BeamX' PV
    ...

[fabcor-maxiv]

If I recall correctly, when we discussed this with @elmjag our assumption was that for Epics, the name prefix and the name would be simply concatenated. Which means that it should be possible to set the prefix to an empty string ''. As far as I understand, YAML allows the empty string '' as key.

So an Epics bridge could look like this:

epics:
  '':  # No common prefix for Epics names
    channels:
      det_threshols_energy:  # name in MXCuBE
        name: MX2:cam1:ThresholdEnergy  # name in epics
      user_beam_x:
        name: MNC:S:SOFT:ao:BeamX

[fabcor-maxiv]

And of course, in case it was not obvious, the idea is that it can also be done like this:

epics:
  MX2:cam1::
    channels:
      det_threshols_energy:  # name in MXCuBE
        name: ThresholdEnergy  # name in epics
  MNC:S:SOFT::
    channels:
      user_beam_x:
        name: ao:BeamX

[laispc]

@fabcor-maxiv thanks for the examples. If it works with an empty string or setting more than one end-point, then it looks good to me! :)

To be honest it wasn't obvious for me in the first look, but if we include a quick example like this in the docs, then it would be enough to understand.

[elmjag]

To be honest it wasn't obvious for me in the first look, but if we include a quick example like this in the docs, then it would be enough to understand.

I'm planning to write a bit more detailed documentation, with a lot of examples, in the future. First I want work a bit on the implementation.

[elmjag]

Seems that we have a possibly workable configuration format. Thanks everyone for looking at this.

I'll take a stab at implementing this. We'll see of this works out in practice.

[beteva]

Hi @elmjag, could you, please, give more details on how you convert the protocol name to type, which is taken into account when you choose in the HO which is your protocol.

[elmjag]

Hi @elmjag, could you, please, give more details on how you convert the protocol name to type, which is taken into account when you choose in the HO which is your protocol.

Not sure I understand the question. Each protocol gets it's own section in the config file. The section name is what defines the protocol used when creating the Command/Channel object. The XML attribute type becomes the section name.

For example:

<object class="SomeClass">
  <channel type="tango" name="Open" tangoname="some/tango/device">Open</command>
  <channel type="exporter" exporter_address="export.org" name="State">State</channel>
  <channel type="epics" name="epicsActuator_val">MNC:B:PB04:m5.VAL</channel>
</object>

Will look like this:

class: SomeClass
tango:
  some/tango/device:
    channels:
      Open:
exporter:
  export.org:
    channels:
      State:
epics:
  "MNC:B:PB04:":
    channels:
      epicsActuator_val:
        name: "m5.VAL"

I have started writing documentation on how channels and commands will be configured:

https://github.com/elmjag/mxcubecore/blob/yaml-commands-channels-doc/docs/source/dev/commands_channels.md

It's is very incomplete so far, but perhaps it clarifies the idea.

[elmjag]

This is my initial attempt at implementing this: #1034