bring latest released features from other branches to protocol binding cleanup branch

.github/FUNDING.yml

@@ -1,3 +1,6 @@
 # FUNDING.yml
 
-github: VigneshVSV
+github: VigneshVSV
+open_collective: hololinked-dev
+buy_me_a_coffee: vigneshvsv
+thanks_dev: gh/vigneshvsv

.github/workflows/python-publish-pypi.yml

@@ -30,7 +30,7 @@ jobs:
         python -m pip install --upgrade pip
         pip install build    
     - name: Build package
-      run: python -m build --wheel
+      run: python -m build
     - name: Publish package
       uses: pypa/gh-action-pypi-publish@release/v1
       with:

.github/workflows/python-publish-testpypi.yml

@@ -30,7 +30,7 @@ jobs:
         python -m pip install --upgrade pip
         pip install build    
     - name: Build package
-      run: python -m build --wheel
+      run: python -m build
     - name: Publish package
       uses: pypa/gh-action-pypi-publish@release/v1
       with:

.github/workflows/test-dev.yml

@@ -12,15 +12,29 @@ on:
 
 jobs:
   test:
-    runs-on: ubuntu-latest
+    strategy:
+      matrix:
+        include:
+          # Define specific Python versions for each OS
+          - os: ubuntu-latest
+            python-version: 3.11
+          # - os: windows-latest
+          #   python-version: 3.11
+          # - os: macos-latest
+          #   python-version: 3.11
+          - os: ubuntu-latest
+            python-version: 3.12
+
+    runs-on: ${{ matrix.os }}
+
     steps:
       - name: Checkout code
         uses: actions/checkout@v4
 
-      - name: Set up Python
+      - name: Set up Python ${{ matrix.python-version }}
         uses: actions/setup-python@v3
         with:
-          python-version: 3.11
+          python-version: ${{ matrix.python-version }}
 
       - name: Install dependencies
         run: pip install -r tests/requirements.txt

.github/workflows/test-release.yml

@@ -9,15 +9,30 @@ on:
 
 jobs:
   test:
-    runs-on: ubuntu-latest
+
+    strategy:
+      matrix:
+        include:
+          # Define specific Python versions for each OS
+          - os: ubuntu-latest
+            python-version: 3.11
+          # - os: windows-latest
+          #   python-version: 3.11
+          # - os: macos-latest
+          #   python-version: 3.11
+          - os: ubuntu-latest
+            python-version: 3.12
+
+    runs-on: ${{ matrix.os }}
+
     steps:
       - name: Checkout code
         uses: actions/checkout@v4
 
-      - name: Set up Python
+      - name: Set up Python ${{ matrix.python-version }}
         uses: actions/setup-python@v3
         with:
-          python-version: 3.11
+          python-version: ${{ matrix.python-version }}
 
       - name: Install dependencies
         run: | 

CHANGELOG.md

@@ -7,13 +7,45 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
-### Security
-- cookie auth & its specification in TD
+✓ means ready to try
+
+New:
+- cookie auth & its specification in TD (cookie auth branch)
+- adding custom handlers for each property, action and event to override default behaviour
+
+## [v0.2.8] - 2024-12-07
+
+- pydantic & JSON schema support for property models 
+- composed sub`Thing`s exposed with correct URL path 
+
+## [v0.2.7] - 2024-10-22
+
+- HTTP SSE would previously remain unclosed when client abruptly disconnected (like closing a browser tab), but now it would close correctly
+- retrieve unserialized data from events with `ObjectProxy` (like JPEG images) by setting `deserialize=False` in `subscribe_event()` 
+
+## [v0.2.6] - 2024-09-09
+
+- bug fix events when multiple serializers are used
+- events support custom HTTP handlers (not polished yet, use as last resort, not auto-added to TD)
+- image event handlers for streaming live video as JPEG and PNG (not polished yet, not auto-added to TD)
+
+## [v0.2.5] - 2024-09-09
+
+- released to anaconda, it can take a while to turn up. A badge will be added in README when successful.  
+
+## [v0.2.4] - 2024-09-09
+
+- added multiple versions of python for testing
+- unlike claimed in previous versions, this package runs only on python 3.11 or higher
+
+## [v0.2.3] - 2024-08-11
+
+- HTTP SSE minor bug-fix/optimization - no difference to the user 
 
 ## [v0.2.2] - 2024-08-09
 
 - thing control panel works better with the server side and support observable properties
-- `ObjectProxy` client API has been improved to resemble WoT operations better, for examplem `get_property` is now 
+- `ObjectProxy` client API has been improved to resemble WoT operations better, for example `get_property` is now 
 called `read_property`, `set_properties` is now called `write_multiple_properties`. 
 - `ObjectProxy` client reliability for poorly written server side actions improved
 

README.md

@@ -2,17 +2,24 @@
 
 ### Description
 
-`hololinked` is a server side pythonic tool suited for instrumentation control and data acquisition over network, especially with HTTP. If you have a requirement to control and capture data from your hardware/instrumentation, show the data in a browser/dashboard, provide a GUI or run automated scripts, `hololinked` can help. Even for isolated applications or a small lab setup without networking concepts, one can still separate the concerns of the tools that interact with the hardware & the hardware itself.
+`hololinked` is a beginner-friendly server side pythonic tool suited for instrumentation control and data acquisition over network, especially with HTTP. If you have a requirement to control and capture data from your hardware/instrumentation, show the data in a browser/dashboard, provide a GUI or run automated scripts, `hololinked` can help. Even for isolated applications or a small lab setup without networking concepts, one can still separate the concerns of the tools that interact with the hardware & the hardware itself.
+
+For those that understand, this package is a ZMQ/HTTP-RPC.
 
-[![Documentation Status](https://readthedocs.org/projects/hololinked/badge/?version=latest)](https://hololinked.readthedocs.io/en/latest/?badge=latest) [![PyPI](https://img.shields.io/pypi/v/hololinked?label=pypi%20package)](https://pypi.org/project/hololinked/) [![PyPI - Downloads](https://img.shields.io/pypi/dm/hololinked)](https://pypistats.org/packages/hololinked) [![codecov](https://codecov.io/gh/VigneshVSV/hololinked/graph/badge.svg?token=JF1928KTFE)](https://codecov.io/gh/VigneshVSV/hololinked) 
+[![Documentation Status](https://readthedocs.org/projects/hololinked/badge/?version=latest)](https://hololinked.readthedocs.io/en/latest/?badge=latest) [![PyPI](https://img.shields.io/pypi/v/hololinked?label=pypi%20package)](https://pypi.org/project/hololinked/) [![Anaconda](https://anaconda.org/conda-forge/hololinked/badges/version.svg)](https://anaconda.org/conda-forge/hololinked)
+[![codecov](https://codecov.io/gh/VigneshVSV/hololinked/graph/badge.svg?token=JF1928KTFE)](https://codecov.io/gh/VigneshVSV/hololinked) 
+<br>
+[![email](https://img.shields.io/badge/email%20me-brown)](mailto:[email protected]) [![ways to contact me](https://img.shields.io/badge/ways_to_contact_me-brown)](https://hololinked.dev/contact)
 <br>
-[![email](https://img.shields.io/badge/email%20me-brown)](mailto:[email protected]) [![find me on discord](https://img.shields.io/badge/find_me_on_discord-brown)](https://discord.com/users/1178428338746966066)
+[![PyPI - Downloads](https://img.shields.io/pypi/dm/hololinked?label=pypi%20downloads)](https://pypistats.org/packages/hololinked)
+[![Conda Downloads](https://img.shields.io/conda/d/conda-forge/hololinked)](https://anaconda.org/conda-forge/hololinked)
 
 ### To Install
 
-From pip - ``pip install hololinked``
+From pip - ``pip install hololinked`` <br>
+From conda -  `conda install -c conda-forge hololinked`
 
-Or, clone the repository (develop branch for latest codebase) and install `pip install .` / `pip install -e .`. The conda env ``hololinked.yml`` can also help to setup all dependencies. 
+Or, clone the repository (main branch for latest codebase) and install `pip install .` / `pip install -e .`. The conda env ``hololinked.yml`` can also help to setup all dependencies. 
 
 ### Usage/Quickstart
 
@@ -21,7 +28,7 @@ Each device or thing can be controlled systematically when their design in softw
 - the hardware is (generally) represented by a class 
 - properties are validated get-set attributes of the class which may be used to model settings, hold captured/computed data or generic network accessible quantities
 - actions are methods which issue commands like connect/disconnect, execute a control routine, start/stop measurement, or run arbitray python logic
-- events can asynchronously communicate/push (arbitrary) data to a client (say, a GUI), like alarm messages, streaming measured quantities etc.
+- events can asynchronously communicate/push arbitrary data to a client, like alarm messages, streaming measured quantities etc.
 
 In this package, the base class which enables this classification is the `Thing` class. Any class that inherits the `Thing` class 
 can instantiate properties, actions and events which become visible to a client in this segragated manner. For example, consider an optical spectrometer, the following code is possible:
@@ -77,6 +84,7 @@ class OceanOpticsSpectrometer(Thing):
         super().__init__(instance_name=instance_name, serial_number=serial_number, **kwargs)
 
 ```
+> There is an ongoing work to remove HTTP API from the property API and completely move them to the HTTP server
 
 In non-expert terms, properties look like class attributes however their data containers are instantiated at object instance level by default.
 For example, the `integration_time` property defined above as `Number`, whenever set/written, will be validated as a float or int, cropped to bounds and assigned as an attribute to each instance of the `OceanOpticsSpectrometer` class with an internally generated name. It is not necessary to know this internally generated name as the property value can be accessed again in any python logic, say, `print(self.integration_time)`. 
@@ -103,9 +111,9 @@ class OceanOpticsSpectrometer(Thing):
 
 ```
 
-In this case, instead of generating a data container with an internal name, the setter method is called when `integration_time` property is set/written. One might add the hardware device driver (say, supplied by the manufacturer) logic here to apply the property onto the device. In the above example, there is not a way provided by lower level library to read the value from the device, so we store it in a variable after applying it and supply the variable back to the getter method. Normally, one would also want the getter to read from the device directly.
+In this case, instead of generating a data container with an internal name, the setter method is called when `integration_time` property is set/written. One might add the hardware device driver logic here (say, supplied by the manufacturer) or a protocol that talks directly to the device to apply the property onto the device. In the above example, there is not a way provided by the device driver library to read the value from the device, so we store it in a variable after applying it and supply the variable back to the getter method. Normally, one would also want the getter to read from the device directly.
 
-Those familiar with Web of Things (WoT) terminology may note that these properties generate the property affordance schema to become accessible by the [node-wot](https://github.com/eclipse-thingweb/node-wot) HTTP(s) client. An example of autogenerated property affordance for `integration_time` is as follows:
+Those familiar with Web of Things (WoT) terminology may note that these properties generate the property affordance. An example for `integration_time` is as follows:
 
 ```JSON
 "integration_time": {
@@ -127,9 +135,9 @@ Those familiar with Web of Things (WoT) terminology may note that these properti
     "minimum": 0.001
 },
 ```
-If you are not familiar with Web of Things or the term "property affordance", consider the above JSON as a description of 
-what the property represents and how to interact with it from somewhere else. Such a JSON is both human-readable, yet consumable 
-by a client provider to create a client object to interact with the property. 
+If you are <span style="text-decoration: underline">not familiar</span> with Web of Things or the term "property affordance", consider the above JSON as a description of 
+what the property represents and how to interact with it from somewhere else. Such a JSON is both human-readable, yet consumable by any application that may use the property - say, a client provider to create a client object to interact with the property or a GUI application to autogenerate a suitable input field for this property. 
+For example, the Eclipse ThingWeb [node-wot](https://github.com/eclipse-thingweb/node-wot) supports this feature to produce a HTTP(s) client that can issue `readProperty("integration_time")` and `writeProperty("integration_time", 1000)` to read and write this property.
 
 The URL path segment `../spectrometer/..` in href field is taken from the `instance_name` which was specified in the `__init__`. 
 This is a mandatory key word argument to the parent class `Thing` to generate a unique name/id for the instance. One should use URI compatible strings.
@@ -271,7 +279,7 @@ what the event represents and how to subscribe to it) with subprotocol SSE (HTTP
 
 Events follow a pub-sub model with '1 publisher to N subscribers' per `Event` object, both through ZMQ and HTTP SSE. 
 
-Although the code is the very familiar & age-old RPC server style, one can directly specify HTTP methods and URL path for each property, action and event. A configurable HTTP Server is already available (from `hololinked.server.HTTPServer`) which redirects HTTP requests to the object according to the specified HTTP API on the properties, actions and events. To plug in a HTTP server: 
+To start the Thing, a configurable HTTP Server is already available (from `hololinked.server.HTTPServer`) which redirects HTTP requests to the object:
 
 ```python
 import ssl, os, logging
@@ -293,6 +301,7 @@ if __name__ == '__main__':
     # or O.run(zmq_protocols=['IPC', 'TCP'], tcp_socket_address='tcp://*:9999')
     # both interprocess communication & TCP, no HTTP 
 ```
+> There is an ongoing work to remove HTTP API from the API of all of properties, actions and events and completely move them to the HTTP server for a more accurate syntax. The functionality will not change though.
 
 Here one can see the use of `instance_name` and why it turns up in the URL path. See the detailed example of the above code [here](https://gitlab.com/hololinked-examples/oceanoptics-spectrometer/-/blob/simple/oceanoptics_spectrometer/device.py?ref_type=heads). 
 
@@ -305,34 +314,36 @@ See a list of currently supported possibilities while using this package [below]
 
 > You may use a script deployment/automation tool to remote stop and start servers, in an attempt to remotely control your hardware scripts. 
 
-One may use the HTTP API according to one's beliefs (including letting the package auto-generate it), but it is mainly intended for web development and cross platform clients like the [node-wot](https://github.com/eclipse-thingweb/node-wot) HTTP(s) client. If your plan is to develop a truly networked system, it is recommended to learn more and use [Thing Descriptions](https://www.w3.org/TR/wot-thing-description11) to describe your hardware. A Thing Description will be automatically generated if absent as shown in JSON examples above or can be supplied manually. The node-wot HTTP(s) client will be able to consume such a description, validate it and abstract away the protocol level details so that one can invoke actions, read & write properties or subscribe to events in a technology agnostic manner. In this way, one can plugin code developed from this package to the rest of the IoT/data-acquisition tools, protocols & standardizations. To know more about client side scripting with node-wot, please look into the documentation [How-To](https://hololinked.readthedocs.io/en/latest/howto/clients.html#using-node-wot-http-s-client) section.
+### A little more about Usage
+
+The HTTP API may be autogenerated or adjusted by the user. If your plan is to develop a truly networked system, it is recommended to learn more and 
+use [Thing Descriptions](https://www.w3.org/TR/wot-thing-description11) to describe your hardware (This is optional and one can still use a classic HTTP client). A Thing Description will be automatically generated if absent as shown in JSON examples above or can be supplied manually. The default end point to fetch thing descriptions are: <br> `http(s)://<host name>/<instance name of the thing>/resources/wot-td` <br>
+If there are errors in generation of Thing Description
+(mostly due to JSON non-complaint types), one could use: <br> `http(s)://<host name>/<instance name of the thing>/resources/wot-td?ignore_errors=true`
+
+(client docs will be updated here next, also check official docs)
 
 ### Currently Supported
 
 - control method execution and property write with a custom finite state machine.
 - database (Postgres, MySQL, SQLite - based on SQLAlchemy) support for storing and loading properties when the object dies and restarts. 
 - auto-generate Thing Description for Web of Things applications. 
-- use serializer of your choice (except for HTTP) - MessagePack, JSON, pickle etc. & extend serialization to suit your requirement. HTTP Server will support only JSON serializer to maintain compatibility with node-wot. Default is JSON serializer based on msgspec.
+- use serializer of your choice (except for HTTP) - MessagePack, JSON, pickle etc. & extend serialization to suit your requirement. HTTP Server will support only JSON serializer to maintain comptibility with Javascript (MessagePack may be added later). Default is JSON serializer based on msgspec.
 - asyncio compatible - async RPC server event-loop and async HTTP Server - write methods in async 
 - choose from multiple ZeroMQ transport methods which offers some possibilities like the following without changing the code:
-  - run HTTP Server & python object in separate processes or the same process
-  - serve multiple objects with the same HTTP server
-  - run direct ZMQ-TCP server without HTTP details
   - expose only a dashboard or web page on the network without exposing the hardware itself
+  - run direct ZMQ-TCP server without HTTP details
+  - serve multiple objects with the same HTTP server, run HTTP Server & python object in separate processes or the same process
 
 Again, please check examples or the code for explanations. Documentation is being activety improved. 
 
 ### Currently being worked
 
+- unit tests coverage
+- separation of HTTP protocol specification like URL path and HTTP verbs from the API of properties, actions and events and move their customization completely to the HTTP server 
+- serve multiple things with the same server (unfortunately due to a small oversight it is currently somewhat difficult for end user to serve multiple things with the same server, although its possible. This will be fixed.)
 - improving accuracy of Thing Descriptions 
-- cookie credentials for authentication - as a workaround until credentials are supported, use `allowed_clients` argument on HTTP server which restricts access based on remote IP supplied with the HTTP headers.
-
-### Internals
-
-This package is an implementation of a ZeroMQ-based Object Oriented RPC with customizable HTTP end-points. A dual transport in both ZMQ and HTTP is provided to maximize flexibility in data type, serialization and speed, although HTTP is preferred for networked applications. If one is looking for an object oriented approach towards creating components within a control or data acquisition system, or an IoT device, one may consider this package. 
+- cookie credentials for authentication - as a workaround until credentials are supported, use `allowed_clients` argument on HTTP server which restricts access based on remote IP supplied with the HTTP headers. This wont still help you in public networks or modified/non-standard HTTP clients.
 
-### Some Day In Future
 
-- mongo DB support for DB operations
-- HTTP 2.0 
 

codecov.yml

@@ -0,0 +1,5 @@
+coverage:
+  status:
+    patch:
+      default:
+        enabled: false