Skip to content

Commit

Permalink
Evaluate Python expression in json_map reader
Browse files Browse the repository at this point in the history
  • Loading branch information
mmpsi committed Feb 21, 2024
1 parent 92a700d commit 0117ed2
Show file tree
Hide file tree
Showing 2 changed files with 62 additions and 1 deletion.
21 changes: 20 additions & 1 deletion pynxtools/dataconverter/readers/json_map/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ This file is designed to let you fill in the requirements of a NeXus Application
The mapping files will always be based on the Template the dataconverter generates. See above on how to generate a mapping file.
The right hand side values of the Template keys are what you can modify.

Here are the three different ways you can fill the right hand side of the Template keys:
Here are the different ways you can fill the right hand side of the Template keys:
* Write the nested path in your datafile. This is indicated by a leading `/` before the word `entry` to make `/entry/data/current_295C` below.
Example:

Expand Down Expand Up @@ -97,6 +97,25 @@ The `datetime`, `dateutil` and `timestamp` properties are mutually exclusive.
The resulting string replaces the mapped value (dictionary) in the mapping dictionary.
If date parsing is enabled, the resulting string is ISO-formatted as required by the Nexus standard.

* Python expression.
The following entry creates an axis array from scalar values in the input file.

```json
"/ENTRY[entry]/DATA[image]/angular0": {
"eval": "np.linspace(arg0[0], arg1[0], int(arg2[0]))",
"arg0": "/scan1/attrs/ScientaSliceBegin",
"arg1": "/scan1/attrs/ScientaSliceEnd",
"arg2": "/scan1/attrs/ScientaNumSlices"
},
```

The properties of the mapping declare the expression and its arguments.

"eval": (required) Python expression to be evaluated by the `eval` built-in.
The expression can use the built-in and numpy (as np) namespaces
as well as the datasets declared by the `argXxx` values.
"argXxx", where Xxx is an integer number: (optional) path of dataset to read from the input data
and to be used in the expression under the same name.

## Contact person in FAIRmat for this reader
Sherjeel Shabih
42 changes: 42 additions & 0 deletions pynxtools/dataconverter/readers/json_map/reader.py
Original file line number Diff line number Diff line change
Expand Up @@ -238,6 +238,47 @@ def parse_strings(mapping, data):
mapping[key] = value


def eval_expressions(mapping, data):
"""
Evaluate Python expressions in mapping.
If a mapping entry contains a dictionary with a `eval` key,
the `eval` expression is evaluated using the Python built-in `eval`.
The expression can use built-in functions, numpy functions in namespace `np`,
and argXxx variables that are defined in the mapping and can refer to dataset paths.
The result of the expression replaces the value of the mapping.
:param mapping: Mapping dictionary
:param data: Data dictionary
:return: None
"""

for key in mapping:
eval_args = mapping[key]

try:
expression = eval_args["eval"]
except (KeyError, TypeError):
continue

args = {}
for arg, value in eval_args.items():
if arg[0:3] == "arg":
if is_path(value):
value = get_val_nested_keystring_from_dict(value[1:], data)
else:
try:
value = float(value)
except TypeError:
pass

args[arg] = value

value = eval(expression, {"np": np}, args)
mapping[key] = value


class JsonMapReader(BaseReader):
"""A reader that takes a mapping json file and a data file/object to return a template."""

Expand Down Expand Up @@ -304,6 +345,7 @@ def read(

new_template = Template()
parse_strings(mapping, data)
eval_expressions(mapping, data)
convert_shapes_to_slice_objects(mapping)

fill_documented(new_template, mapping, template, data)
Expand Down

0 comments on commit 0117ed2

Please sign in to comment.