Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

minor improvements #30

Merged
merged 33 commits into from
Aug 9, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
3289a65
release workflow for test pypi before merging to release branch
VigneshVSV Jul 21, 2024
4d47b67
update comment on intro level
Jul 22, 2024
0a141f0
edit project title
Jul 22, 2024
d82adbd
rename tests
VigneshVSV Jul 23, 2024
46aff3e
added discord invite
VigneshVSV Jul 23, 2024
1ef204a
add email as badge
VigneshVSV Jul 23, 2024
4c3fb3a
edit beginner info
VigneshVSV Jul 24, 2024
791b9dd
added timeout to locks on client which prevents infinite hang
VigneshVSV Jul 24, 2024
897d66b
added timeout to locks on client which prevents infinite hang
VigneshVSV Jul 24, 2024
7714be4
example for simplistic cases for action and property
VigneshVSV Jul 25, 2024
8d61e15
tab edit in action example
VigneshVSV Jul 25, 2024
e59ca5c
update control panel client github address
VigneshVSV Jul 26, 2024
f43fa0b
add zmq TCP and IPC
VigneshVSV Jul 27, 2024
5592ec0
Update README.md
VigneshVSV Jul 28, 2024
e677fe5
Update README.md
VigneshVSV Jul 28, 2024
4eef7f0
Update README.md
VigneshVSV Jul 28, 2024
65ec93a
Update README.md
VigneshVSV Jul 30, 2024
54dd28d
Update README.md
VigneshVSV Jul 30, 2024
0beb629
Update README.md
VigneshVSV Jul 30, 2024
4e897dc
Update README.md
VigneshVSV Jul 30, 2024
94616b5
Update README.md
VigneshVSV Jul 31, 2024
904c309
Update README.md
VigneshVSV Jul 31, 2024
3239458
fixed type definitions for events which was not turning up in VS code
VigneshVSV Aug 1, 2024
f877d6e
compatibility for the thing control panel viewer
VigneshVSV Aug 1, 2024
1f7c58e
added personal discord
VigneshVSV Aug 1, 2024
041cd38
update examples HEAD
VigneshVSV Aug 2, 2024
c30583f
fixes state machine non-inclusion in TD when no state machine present
VigneshVSV Aug 2, 2024
39a6032
added raspberry pi example
VigneshVSV Aug 2, 2024
7be429a
update README
VigneshVSV Aug 8, 2024
504cde2
updated client API to resemble wot operations
VigneshVSV Aug 8, 2024
b5581d4
bug fix object proxy client
VigneshVSV Aug 9, 2024
3d3c8c2
bug fix object proxy client
VigneshVSV Aug 9, 2024
f20ffc8
update changelog, doc and examples
VigneshVSV Aug 9, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
name: Python Unit Tests
name: Unit Tests For Development

on:
workflow_dispatch:
Expand Down
32 changes: 32 additions & 0 deletions .github/workflows/test-release.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
name: Unit Tests With TestPyPI

on:
workflow_dispatch:
pull_request:
branches:
- release


jobs:
test:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4

- name: Set up Python
uses: actions/setup-python@v3
with:
python-version: 3.11

- name: Install dependencies
run: |
pip install jsonschema
pip install -i https://test.pypi.org/simple/ hololinked

- name: Run unit tests to verify if the release to TestPyPI is working
run: |
python -m unittest discover -s tests -p 'test_*.py'



7 changes: 7 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,13 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Security
- cookie auth & its specification in TD

## [v0.2.2] - 2024-08-09

- thing control panel works better with the server side and support observable properties
- `ObjectProxy` client API has been improved to resemble WoT operations better, for examplem `get_property` is now
called `read_property`, `set_properties` is now called `write_multiple_properties`.
- `ObjectProxy` client reliability for poorly written server side actions improved

## [v0.2.1] - 2024-07-21

### Added
Expand Down
2 changes: 1 addition & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ Otherwise, I will then take care of the issue as soon as possible.
Developers are always welcome to contribute to the code base. If you want to tackle any issues, un-existing features, let me know (at my email), I can create some open issues and features which I was never able to solve or did not have the time. You can also suggest what else can be contributed functionally or conceptually or also simply code-refactoring. The lack of issues or features in the [Issues](https://github.com/VigneshVSV/hololinked/issues) section of github does not mean the project is considered feature complete or I dont have ideas what to do next. On the contrary, there is tons of work to do.

There are also repositories which can use your skills:
- An [admin client](https://github.com/VigneshVSV/hololinked-portal) in react
- An [admin client](https://github.com/VigneshVSV/thing-control-panel) in react
- [Documentation](https://github.com/VigneshVSV/hololinked-docs) in sphinx which needs significant improvement in How-To's, beginner level docs which may teach people concepts of data acquisition or IoT, Docstring or API documentation of this repository itself
- [Examples](https://github.com/VigneshVSV/hololinked-examples) in nodeJS, Dashboard/PyQt GUIs or server implementations using this package. Hardware implementations of unexisting examples are also welcome, I can open a directory where people can search for code based on hardware and just download your code.

Expand Down
57 changes: 33 additions & 24 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
# hololinked - Pythonic Supervisory Control & Data Acquisition / Internet of Things
# hololinked - Pythonic Object-Oriented Supervisory Control & Data Acquisition / Internet of Things

### Description

For beginners - `hololinked` is a server side pythonic package suited for instrumentation control and data acquisition over network, especially with HTTP. If you have a requirement to control and capture data from your hardware/instrumentation, show the data in a browser/dashboard, provide a GUI or run automated scripts, `hololinked` can help. Even for isolated applications or a small lab setup without networking concepts, one can still separate the concerns of the tools that interact with the hardware & the hardware itself.
<br/> <br/>
For those familiar with RPC & web development - This package is an implementation of a ZeroMQ-based Object Oriented RPC with customizable HTTP end-points. A dual transport in both ZMQ and HTTP is provided to maximize flexibility in data type, serialization and speed, although HTTP is preferred for networked applications. If one is looking for an object oriented approach towards creating components within a control or data acquisition system, or an IoT device, one may consider this package.
`hololinked` is a server side pythonic tool suited for instrumentation control and data acquisition over network, especially with HTTP. If you have a requirement to control and capture data from your hardware/instrumentation, show the data in a browser/dashboard, provide a GUI or run automated scripts, `hololinked` can help. Even for isolated applications or a small lab setup without networking concepts, one can still separate the concerns of the tools that interact with the hardware & the hardware itself.

[![Documentation Status](https://readthedocs.org/projects/hololinked/badge/?version=latest)](https://hololinked.readthedocs.io/en/latest/?badge=latest) [![PyPI](https://img.shields.io/pypi/v/hololinked?label=pypi%20package)](https://pypi.org/project/hololinked/) [![PyPI - Downloads](https://img.shields.io/pypi/dm/hololinked)](https://pypistats.org/packages/hololinked) [![codecov](https://codecov.io/gh/VigneshVSV/hololinked/graph/badge.svg?token=JF1928KTFE)](https://codecov.io/gh/VigneshVSV/hololinked)
[![Documentation Status](https://readthedocs.org/projects/hololinked/badge/?version=latest)](https://hololinked.readthedocs.io/en/latest/?badge=latest) [![PyPI](https://img.shields.io/pypi/v/hololinked?label=pypi%20package)](https://pypi.org/project/hololinked/) [![PyPI - Downloads](https://img.shields.io/pypi/dm/hololinked)](https://pypistats.org/packages/hololinked) [![codecov](https://codecov.io/gh/VigneshVSV/hololinked/graph/badge.svg?token=JF1928KTFE)](https://codecov.io/gh/VigneshVSV/hololinked)
<br>
[![email](https://img.shields.io/badge/email%20me-brown)](mailto:[email protected]) [![find me on discord](https://img.shields.io/badge/find_me_on_discord-brown)](https://discord.com/users/1178428338746966066)

### To Install

Expand All @@ -19,16 +19,14 @@ Or, clone the repository (develop branch for latest codebase) and install `pip i
`hololinked` is compatible with the [Web of Things](https://www.w3.org/WoT/) recommended pattern for developing hardware/instrumentation control software.
Each device or thing can be controlled systematically when their design in software is segregated into properties, actions and events. In object oriented terms:
- the hardware is (generally) represented by a class
- properties are validated get-set attributes of the class which may be used to model hardware settings, hold captured/computed data or generic network accessible quantities
- properties are validated get-set attributes of the class which may be used to model settings, hold captured/computed data or generic network accessible quantities
- actions are methods which issue commands like connect/disconnect, execute a control routine, start/stop measurement, or run arbitray python logic
- events can asynchronously communicate/push (arbitrary) data to a client (say, a GUI), like alarm messages, streaming measured quantities etc.

It does not even matter whether you are controlling your hardware locally or remotely, what protocol you use, what is the nature of the client etc.,
one has to provide these three interactions with the hardware. In this package, the base class which enables this classification is the `Thing` class. Any class that inherits the `Thing` class
can instantiate properties, actions and events which
become visible to a client in this segragated manner. For example, consider an optical spectrometer, the following code is possible:
In this package, the base class which enables this classification is the `Thing` class. Any class that inherits the `Thing` class
can instantiate properties, actions and events which become visible to a client in this segragated manner. For example, consider an optical spectrometer, the following code is possible:

> This is a fairly mid-level intro, if you are beginner, for another variant check [How-To](https://hololinked.readthedocs.io/en/latest/howto/index.html)
> This is a fairly mid-level intro focussed on HTTP. If you are beginner or looking for ZMQ which can be used with no networking knowledge, check [How-To](https://hololinked.readthedocs.io/en/latest/howto/index.html)

#### Import Statements

Expand Down Expand Up @@ -64,8 +62,8 @@ class OceanOpticsSpectrometer(Thing):
serial_number = String(default=None, allow_None=True, URL_path='/serial-number',
doc="serial number of the spectrometer to connect/or connected",
http_method=("GET", "PUT"))
# GET and PUT is default for reading and writing the property respectively.
# Use other HTTP methods if necessary.
# GET and PUT is default for reading and writing the property respectively.
# So you can leave it out, especially if you are going to use ZMQ and dont understand HTTP

integration_time = Number(default=1000, bounds=(0.001, None), crop_to_bounds=True,
URL_path='/integration-time',
Expand All @@ -81,10 +79,9 @@ class OceanOpticsSpectrometer(Thing):
```

In non-expert terms, properties look like class attributes however their data containers are instantiated at object instance level by default.
For example, the `integratime_time` property defined above as `Number`, whenever set/written, will be validated as a float or int, cropped to bounds and assigned as an attribute to each instance of the `OceanOpticsSpectrometer` class with an internally generated name. It is not necessary to know this internally generated name as the property value can be accessed again in any python logic, say, `print(self.integration_time)`.
For example, the `integration_time` property defined above as `Number`, whenever set/written, will be validated as a float or int, cropped to bounds and assigned as an attribute to each instance of the `OceanOpticsSpectrometer` class with an internally generated name. It is not necessary to know this internally generated name as the property value can be accessed again in any python logic, say, `print(self.integration_time)`.

To overload the get-set (or read-write) of properties, one may do the following:

```python
class OceanOpticsSpectrometer(Thing):

Expand Down Expand Up @@ -132,8 +129,7 @@ Those familiar with Web of Things (WoT) terminology may note that these properti
```
If you are not familiar with Web of Things or the term "property affordance", consider the above JSON as a description of
what the property represents and how to interact with it from somewhere else. Such a JSON is both human-readable, yet consumable
by a client provider to create a client object to interact with the property in the way the property demands. You, as the developer,
only need to use the client.
by a client provider to create a client object to interact with the property.

The URL path segment `../spectrometer/..` in href field is taken from the `instance_name` which was specified in the `__init__`.
This is a mandatory key word argument to the parent class `Thing` to generate a unique name/id for the instance. One should use URI compatible strings.
Expand All @@ -153,6 +149,13 @@ class OceanOpticsSpectrometer(Thing):
self.serial_number = serial_number
self.device = Spectrometer.from_serial_number(self.serial_number)
self._wavelengths = self.device.wavelengths().tolist()

# So you can leave it out, especially if you are going to use ZMQ and dont understand HTTP
@action()
def disconnect(self):
"""disconnect from the spectrometer"""
self.device.close()

```

Methods that are neither decorated with action decorator nor acting as getters-setters of properties remain as plain python methods and are **not** accessible on the network.
Expand Down Expand Up @@ -266,6 +269,8 @@ what the event represents and how to subscribe to it) with subprotocol SSE (HTTP
```
> data schema ("data" field above which describes the event payload) are optional and discussed later

Events follow a pub-sub model with '1 publisher to N subscribers' per `Event` object, both through ZMQ and HTTP SSE.

Although the code is the very familiar & age-old RPC server style, one can directly specify HTTP methods and URL path for each property, action and event. A configurable HTTP Server is already available (from `hololinked.server.HTTPServer`) which redirects HTTP requests to the object according to the specified HTTP API on the properties, actions and events. To plug in a HTTP server:

```python
Expand All @@ -284,14 +289,17 @@ if __name__ == '__main__':
log_level=logging.DEBUG
)
O.run_with_http_server(ssl_context=ssl_context)
# or O.run(zmq_protocols='IPC') - interprocess communication and no HTTP
# or O.run(zmq_protocols=['IPC', 'TCP'], tcp_socket_address='tcp://*:9999')
# both interprocess communication & TCP, no HTTP
```

Here one can see the use of `instance_name` and why it turns up in the URL path. See the detailed example of the above code [here](https://gitlab.com/hololinked-examples/oceanoptics-spectrometer/-/blob/simple/oceanoptics_spectrometer/device.py?ref_type=heads).

##### NOTE - The package is under active development. Contributors welcome, please check CONTRIBUTING.md.
##### NOTE - The package is under active development. Contributors welcome, please check CONTRIBUTING.md and the open issues. Some issues can also be independently dealt without much knowledge of this package.

- [example repository](https://github.com/VigneshVSV/hololinked-examples) - detailed examples for both clients and servers
- [helper GUI](https://github.com/VigneshVSV/hololinked-portal) - view & interact with your object's methods, properties and events.
- [helper GUI](https://github.com/VigneshVSV/thing-control-panel) - view & interact with your object's actions, properties and events.

See a list of currently supported possibilities while using this package [below](#currently-supported).

Expand All @@ -306,24 +314,25 @@ One may use the HTTP API according to one's beliefs (including letting the packa
- auto-generate Thing Description for Web of Things applications.
- use serializer of your choice (except for HTTP) - MessagePack, JSON, pickle etc. & extend serialization to suit your requirement. HTTP Server will support only JSON serializer to maintain compatibility with node-wot. Default is JSON serializer based on msgspec.
- asyncio compatible - async RPC server event-loop and async HTTP Server - write methods in async
- choose from multiple ZeroMQ transport methods. Some of the possibilities one can achieve by choosing ZMQ transport methods:
- choose from multiple ZeroMQ transport methods which offers some possibilities like the following without changing the code:
- run HTTP Server & python object in separate processes or the same process
- serve multiple objects with the same HTTP server
- run direct ZMQ-TCP server without HTTP details
- expose only a dashboard or web page on the network without exposing the hardware itself

Again, please check examples or the code for explanations. Documentation is being activety improved.

### Currently being worked

- improving accuracy of Thing Descriptions
- cookie credentials for authentication - as a workaround until credentials are supported, use `allowed_clients` argument on HTTP server which restricts access based on remote IP supplied with the HTTP headers.

### Internals

This package is an implementation of a ZeroMQ-based Object Oriented RPC with customizable HTTP end-points. A dual transport in both ZMQ and HTTP is provided to maximize flexibility in data type, serialization and speed, although HTTP is preferred for networked applications. If one is looking for an object oriented approach towards creating components within a control or data acquisition system, or an IoT device, one may consider this package.

### Some Day In Future

- mongo DB support for DB operations
- HTTP 2.0

### Contact

Contributors welcome for all my projects related to hololinked including web apps. Please write to my contact email available at my [website](https://hololinked.dev).
2 changes: 1 addition & 1 deletion doc
Submodule doc updated from 68e1be to a06486
2 changes: 1 addition & 1 deletion examples
2 changes: 1 addition & 1 deletion hololinked/__init__.py
Original file line number Diff line number Diff line change
@@ -1 +1 @@
__version__ = "0.2.1"
__version__ = "0.2.2"
Loading
Loading