Skip to content

Commit

Permalink
Updated README (#10)
Browse files Browse the repository at this point in the history
  • Loading branch information
@Nikita-Smirnov-Exactpro
Nikita-Smirnov-Exactpro authored Jan 21, 2025
1 parent 6507f69 commit d3237b3
Show file tree
Hide file tree
Showing 2 changed files with 157 additions and 44 deletions.
103 changes: 97 additions & 6 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -62,13 +62,9 @@ spec:
port: 8080
```
## Requirements for Jupyter's notebooks
## Jupyter's notebooks format
* Cell tagged `parameters` (required)
* this cell should be listed parameters only.
* parameters could have typing and value, but value must be constant and have primitive type like boolean, number, string.
* required parameters:
* `output_path` - path to [JSONL](https://jsonlines.org/) file. Server considers a content of this file as run results.
* Cell tagged `parameters` (required). You can read detail about this cell below.
* Cell with dependencies (optional) - server doesn't include third-party packages by default.
You can install / uninstall packages required for your code in one of cells. All installed packages are shared between runs any notebook.
Installation example:
Expand All @@ -77,6 +73,101 @@ Installation example:
!{sys.executable} -m pip install <package_name>==<package_version>
```

### Jupyter's notebooks inputs

Notebook must have a first cell with the `parameters` tag. This cell is used to configure parametrized run.
* this cell should be listed parameters only.
* parameters could have typing and value, but value must be constant and have primitive type like boolean, number, string.

#### Required parameters:
* `output_path` - path to [JSONL](https://jsonlines.org/) file. Server considers a content of this file as run results.
`js-p` generates and passes a file path in the folder configured by `results` setting for this parameter

#### Optional parameters:
* `customization_path` - path to [JSON](https://www.json.org/) file. Server considers a content of this file as run customization.
`js-p` generates and passes a file path in the folder configured by `results` setting for this parameter

#### Special parameters suffixes

Parameter names can have special suffixes to help a viewer apples different controls for them
* `_file` - string parameter is containing path to a file on the server node. `js-p` provides paths to a files located in the folders or sub-folders of configured dirs by `notebooks` and `results` settings
* `_timestamp` - string parameter is containing datetime in the ISO format like `2024-07-01T05:06:59.664Z`
* `_pycode` - string parameter is containing a code. A viewer can apply highlighting for this parameter

### Jupyter's notebooks outputs examples

#### Content example of file configured by `output_path` parameter

Each JSON in JSONL contain can contain special fields. A viewer can have special logic for handling these fields.
* `#display-timestamp` - contain [Unix time](https://en.wikipedia.org/wiki/Unix_time) in nanoseconds. on
example:
```json
{
"#display-timestamp": 1737048910123000000,
"field": "value"
}
```
* `#display-name` - short name of JSON Node. A viewer can show this value instead or together with full node content.
example:
```json
{
"#display-name": "Root node",
"field": "value",
"sub-node": {
"#display-name": "Sub node",
"sub-field": "sub-value"
}
}
```
* `#display-table` - table view of JSON Node. This value can cover the whole content of the Node or cover only part of data.
example:
```json
{
"#display-table": [
["int", "float"],
[19700, 19.700000000000003],
[39400, 39.400000000000006]
],
"results": 2,
"array": [
{
"name": "str_100_test",
"int": 19700,
"float": 19.700000000000003
},
{
"name": "str_200_test",
"int": 39400,
"float": 39.400000000000006
}
]
}
```

```json lines
{"#display-timestamp": 1737388741801564430, "#display-name": "2 - 2025-01-20 19:59:01.801564", "#display-table": [["int", "float"], ["45200", "45.2"], ["90400", "90.4"]], "results": 2, "array": [{"#display-name": "str_100_test:2024-07-01T05:06:59.664Z:False", "name": "str_100_test", "int": 45200, "float": 45.2, "flag": false, "custom_time": "2024-07-01T05:06:59.664Z"}, {"#display-name": "str_200_test:2024-07-01T05:06:59.665Z:True", "name": "str_200_test", "int": 90400, "float": 90.4, "flag": true, "custom_time": "2024-07-01T05:06:59.665Z"}]}
{"#display-timestamp": 1737388741801596018, "#display-name": "3 - 2025-01-20 19:59:01.801596", "#display-table": [["int", "float"], ["23800", "23.8"], ["47600", "47.6"]], "results": 2, "array": [{"#display-name": "str_100_test:2024-07-01T05:06:59.664Z:False", "name": "str_100_test", "int": 23800, "float": 23.8, "flag": false, "custom_time": "2024-07-01T05:06:59.664Z"}, {"#display-name": "str_200_test:2024-07-01T05:06:59.665Z:True", "name": "str_200_test", "int": 47600, "float": 47.6, "flag": true, "custom_time": "2024-07-01T05:06:59.665Z"}]}
```

#### Content example of file configured by `customization_path` parameter

This JSON should have list of pattern. Each pattern should have the fields:
* `pattern` - text which should be highlighted
* `color` - color for highlighting

```json
[
{
"pattern": "str_100_test",
"color": "#3CD91F"
},
{
"pattern": "str_200_test",
"color": "#9356D5"
}
]
```

## local run

### run th2-rpt-viewer, th2-json-provider, jupyter-notebook are proxied by nginx
Expand Down
98 changes: 60 additions & 38 deletions example/example.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,70 +2,92 @@
"cells": [
{
"cell_type": "code",
"execution_count": 9,
"execution_count": 1,
"metadata": {
"tags": [
"parameters"
]
},
"outputs": [],
"source": [
"output_path='output.jsonl'\n",
"lambda_pycode = \"\"\"lambda a:a['weight'] < 300\"\"\""
"output_path = 'output.jsonl'\n",
"customization_path = 'customization.json'\n",
"\n",
"test_1_timestamp = \"2024-07-01T05:06:59.664Z\"\n",
"test_2_timestamp = \"2024-07-01T05:06:59.665Z\"\n",
"test_lambda_1_pycode = \"\"\"lambda item: 30000 < item['int'] < 120000 or 3.3 < item['float'] < 40.0\"\"\"\n",
"\n",
"str_1_test = \"str_100_test\"\n",
"str_2_test: str = \"str_200_test\"\n",
"int_1_test = 100\n",
"int_2_test: int = 200\n",
"float_1_test = 0.1\n",
"float_2_test: float = 0.2\n",
"bool_1_test = False\n",
"bool_2_test = True"
]
},
{
"cell_type": "code",
"execution_count": 10,
"execution_count": 2,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"0 5 50\n"
]
}
],
"outputs": [],
"source": [
"import json\n",
"import time\n",
"import random\n",
"from datetime import datetime\n",
"amount = 100\n",
"\n",
"displayTable = [\n",
" ['bid order', 'bid rate', 'bid qty', 'ask order', 'ask rate', 'ask qty'],\n",
" ['424', '0.61500000', '101', '', '', ''],\n",
" ['424', '0.61500000', '101', '', '', ''],\n",
"]\n",
"table_header = ['int', 'float']\n",
"data = []\n",
"for pos in range(1, 100):\n",
" timestamp = time.time_ns()\n",
" datetime_timestamp = datetime.fromtimestamp(timestamp / 1e9)\n",
" base = random.randrange(0, 1000)\n",
"\n",
"timestamp = 1725539650\n",
"amount = 200\n",
"pos = 0\n",
"conver = 1000000000\n",
" array = [\n",
" {'#display-name': str_1_test + ':' + test_1_timestamp +':' + str(bool_1_test), 'name': str_1_test, 'int': base * int_1_test,\n",
" 'float': base * float_1_test, 'flag': bool_1_test, 'custom_time': test_1_timestamp},\n",
" {'#display-name': str_2_test + ':' + test_2_timestamp + ':' + str(bool_2_test), 'name': str_2_test, 'int': base * int_2_test,\n",
" 'float': base * float_2_test, 'flag': bool_2_test, 'custom_time': test_2_timestamp},\n",
" ]\n",
"\n",
"testArr = []\n",
" filtered = list(filter(eval(test_lambda_1_pycode), array))\n",
"\n",
"while pos < amount:\n",
" timestamp_in_nano = timestamp*1000000000\n",
" weight = random.randrange(1, 1000)\n",
" timest_obj = datetime.fromtimestamp(timestamp)\n",
" testArr.append({ '#display-timestamp': timestamp_in_nano, '#display-name': str(pos) + ' ' + str(timest_obj), '#display-table': displayTable, 'weight': weight})\n",
" pos = pos + 1\n",
" timestamp += random.randrange(1, 100)\n",
"\n",
"filtered = list(filter(eval(lambda_pycode), testArr))\n",
" line = {\n",
" '#display-timestamp': timestamp,\n",
" '#display-name': str(pos) + ' - ' + str(datetime_timestamp),\n",
" '#display-table': [table_header, *[[item['int'], item['float']] for item in filtered]],\n",
" 'results': len(filtered),\n",
" 'array': filtered,\n",
" }\n",
" data.append(line)\n",
"\n",
"with open(output_path, \"w\") as out_file:\n",
" for orig_order in filtered:\n",
" for orig_order in data:\n",
" json.dump(orig_order, out_file)\n",
" out_file.write('\\n')"
" out_file.write('\\n')\n",
"\n",
"with (open(customization_path, \"w\") as customization_file):\n",
" r = lambda: random.randint(0,255)\n",
" customization = [\n",
" {'pattern': str_1_test, 'color': '#%02X%02X%02X' % (r(), r(), r())},\n",
" {'pattern': str_2_test, 'color': '#%02X%02X%02X' % (r(), r(), r())},\n",
" ]\n",
" json.dump(customization, customization_file, indent=4)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
Expand All @@ -79,9 +101,9 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.0"
"version": "3.10.12"
}
},
"nbformat": 4,
"nbformat_minor": 2
"nbformat_minor": 4
}

0 comments on commit d3237b3

Please sign in to comment.