Minor Changes and Bug Fixes

Minor Changes and Bug Fixes

.github/FUNDING.yml

@@ -2,5 +2,5 @@
 
 github: VigneshVSV
 buy_me_a_coffee: vigneshvsv
-thanks.dev: gh/vigneshvsv
+thanks_dev: gh/vigneshvsv
 

CHANGELOG.md

@@ -9,10 +9,15 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ✓ means ready to try
 
+New:
 - cookie auth & its specification in TD (cookie auth branch)
 - adding custom handlers for each property, action and event to override default behaviour
 - pydantic support for property models 
 
+Bug Fixes:
+- retrieve unserialized data from events with `ObjectProxy` (like JPEG images) by setting `deserialize=False` in `subscribe_event()` ✓ 
+- composed sub`Thing`s exposed with correct URL path ✓ 
+
 ## [v0.2.6] - 2024-09-09
 
 - bug fix events when multiple serializers are used

README.md

@@ -8,7 +8,9 @@
 [![codecov](https://codecov.io/gh/VigneshVSV/hololinked/graph/badge.svg?token=JF1928KTFE)](https://codecov.io/gh/VigneshVSV/hololinked) 
 <br>
 [![email](https://img.shields.io/badge/email%20me-brown)](mailto:[email protected]) [![ways to contact me](https://img.shields.io/badge/ways_to_contact_me-brown)](https://hololinked.dev/contact)
-
+<br>
+[![PyPI - Downloads](https://img.shields.io/pypi/dm/hololinked?label=pypi%20downloads)](https://pypistats.org/packages/hololinked)
+[![Conda Downloads](https://img.shields.io/conda/d/conda-forge/hololinked)](https://anaconda.org/conda-forge/hololinked)
 
 <!-- [![PyPI - Downloads](https://img.shields.io/pypi/dm/hololinked)](https://pypistats.org/packages/hololinked)  -->
 ### To Install
@@ -274,7 +276,7 @@ what the event represents and how to subscribe to it) with subprotocol SSE (HTTP
 
 Events follow a pub-sub model with '1 publisher to N subscribers' per `Event` object, both through ZMQ and HTTP SSE. 
 
-Although the code is the very familiar & age-old RPC server style, one can directly specify HTTP methods and URL path for each property, action and event. A configurable HTTP Server is already available (from `hololinked.server.HTTPServer`) which redirects HTTP requests to the object according to the specified HTTP API on the properties, actions and events. To plug in a HTTP server: 
+To start the Thing, a configurable HTTP Server is already available (from `hololinked.server.HTTPServer`) which redirects HTTP requests to the object:
 
 ```python
 import ssl, os, logging
@@ -318,22 +320,26 @@ One may use the HTTP API according to one's beliefs (including letting the packa
 - use serializer of your choice (except for HTTP) - MessagePack, JSON, pickle etc. & extend serialization to suit your requirement. HTTP Server will support only JSON serializer to maintain comptibility with Javascript (MessagePack may be added later). Default is JSON serializer based on msgspec.
 - asyncio compatible - async RPC server event-loop and async HTTP Server - write methods in async 
 - choose from multiple ZeroMQ transport methods which offers some possibilities like the following without changing the code:
-  - run HTTP Server & python object in separate processes or the same process
+  - expose only a dashboard or web page on the network without exposing the hardware itself
   - serve multiple objects with the same HTTP server
+  - run HTTP Server & python object in separate processes or the same process
   - run direct ZMQ-TCP server without HTTP details
-  - expose only a dashboard or web page on the network without exposing the hardware itself
 
 Again, please check examples or the code for explanations. Documentation is being activety improved. 
 
 ### Currently being worked
 
-- improving accuracy of Thing Descriptions 
-- separation of HTTP protocol specification like URL path and HTTP verbs from the API of properties, actions and events and move their customization completely to the HTTP server
 - unit tests coverage
+- improving accuracy of Thing Descriptions 
+- separation of HTTP protocol specification like URL path and HTTP verbs from the API of properties, actions and events and move their customization completely to the HTTP server 
 - cookie credentials for authentication - as a workaround until credentials are supported, use `allowed_clients` argument on HTTP server which restricts access based on remote IP supplied with the HTTP headers. This wont still help you in public networks or modified/non-standard HTTP clients.
 
 ### Internals
 
 This package is an implementation of a ZeroMQ-based Object Oriented RPC with customizable HTTP end-points. A dual transport in both ZMQ and HTTP is provided to maximize flexibility in data type, serialization and speed, although HTTP is preferred for networked applications. If one is looking for an object oriented approach towards creating components within a control or data acquisition system, or an IoT device, one may consider this package. 
 
+### Sponsorships
+
+If you are interesting in donating, please consider doing through thanks.dev as dependencies looking for sponsorships are also sponsored automatically. Its also possible to sponsor for services like bug fixes (informally - i.e. no legal claims), please look at open collective or github sponsors profile. 
+
 

hololinked/client/proxy.py

@@ -463,7 +463,7 @@ async def async_write_multiple_properties(self, **properties) -> None:
 
 
     def subscribe_event(self, name : str, callbacks : typing.Union[typing.List[typing.Callable], typing.Callable],
-                        thread_callbacks : bool = False) -> None:
+                        thread_callbacks : bool = False, deserialize : bool = True) -> None:
         """
         Subscribe to event specified by name. Events are listened in separate threads and supplied callbacks are
         are also called in those threads. 
@@ -489,7 +489,7 @@ def subscribe_event(self, name : str, callbacks : typing.Union[typing.List[typin
         if event._subscribed:
             event.add_callbacks(callbacks)
         else: 
-            event.subscribe(callbacks, thread_callbacks)
+            event.subscribe(callbacks, thread_callbacks, deserialize)
 
 
     def unsubscribe_event(self, name : str):
@@ -756,7 +756,7 @@ def oneway_set(self, value : typing.Any) -> None:
 class _Event:
 
     __slots__ = ['_zmq_client', '_name', '_obj_name', '_unique_identifier', '_socket_address', '_callbacks', '_serialization_specific',
-                    '_serializer', '_subscribed', '_thread', '_thread_callbacks', '_event_consumer', '_logger']
+                    '_serializer', '_subscribed', '_thread', '_thread_callbacks', '_event_consumer', '_logger', '_deserialize']
     # event subscription
     # Dont add class doc otherwise __doc__ in slots will conflict with class variable
 
@@ -772,6 +772,7 @@ def __init__(self, client : SyncZMQClient, name : str, obj_name : str, unique_id
         self._serializer = serializer
         self._logger = logger 
         self._subscribed = False
+        self._deserialize = True
 
     def add_callbacks(self, callbacks : typing.Union[typing.List[typing.Callable], typing.Callable]) -> None:
         if not self._callbacks:
@@ -782,22 +783,23 @@ def add_callbacks(self, callbacks : typing.Union[typing.List[typing.Callable], t
             self._callbacks.append(callbacks)
 
     def subscribe(self, callbacks : typing.Union[typing.List[typing.Callable], typing.Callable], 
-                    thread_callbacks : bool = False):
+                    thread_callbacks : bool = False, deserialize : bool = True):
         self._event_consumer = EventConsumer(
                                     'zmq-' + self._unique_identifier if self._serialization_specific else self._unique_identifier, 
                                     self._socket_address, f"{self._name}|RPCEvent|{uuid.uuid4()}", b'PROXY',
                                     zmq_serializer=self._serializer, logger=self._logger
                                 )
         self.add_callbacks(callbacks) 
         self._subscribed = True
+        self._deserialize = deserialize
         self._thread_callbacks = thread_callbacks
         self._thread = threading.Thread(target=self.listen)
         self._thread.start()
 
     def listen(self):
         while self._subscribed:
             try:
-                data = self._event_consumer.receive()
+                data = self._event_consumer.receive(deserialize=self._deserialize)
                 if data == 'INTERRUPT':
                     break
                 for cb in self._callbacks: 

hololinked/server/dataklasses.py

@@ -550,9 +550,7 @@ def get_organised_resources(instance):
     # Events
     for name, resource in inspect._getmembers(instance, lambda o : isinstance(o, Event), getattr_without_descriptor_read):
         assert isinstance(resource, Event), ("thing event query from inspect.ismethod is not an Event",
-                                    "logic error - visit https://github.com/VigneshVSV/hololinked/issues to report")
-        if getattr(instance, name, None):
-           continue 
+                                    "logic error - visit https://github.com/VigneshVSV/hololinked/issues to report")    
         # above assertion is only a typing convenience
         fullpath = f"{instance._full_URL_path_prefix}{resource.URL_path}"
         # resource._remote_info.unique_identifier = fullpath

hololinked/server/events.py

@@ -70,14 +70,13 @@ def __get__(self, obj : ParameterizedMetaclass, objtype : typing.Optional[type]
 
     def __set__(self, obj : Parameterized, value : typing.Any) -> None:
         if isinstance(value, EventDispatcher):
-            if not obj.__dict__.get(self._internal_name, None):
-                value._remote_info.name = self.friendly_name
-                value._remote_info.obj_name = self._obj_name
-                value._owner_inst = obj
-                obj.__dict__[self._internal_name] = value 
-            else:
-                raise AttributeError(f"Event object already assigned for {self._obj_name}. Cannot reassign.") 
-                # may be allowing to reassign is not a bad idea 
+            value._remote_info.name = self.friendly_name
+            value._remote_info.obj_name = self._obj_name
+            value._owner_inst = obj
+            current_obj = obj.__dict__.get(self._internal_name, None) # type: typing.Optional[EventDispatcher]
+            if current_obj and current_obj._publisher:
+                current_obj._publisher.unregister(current_obj)
+            obj.__dict__[self._internal_name] = value 
         else:
             raise TypeError(f"Supply EventDispatcher object to event {self._obj_name}, not type {type(value)}.")
 

hololinked/server/thing.py

@@ -55,6 +55,7 @@ def __new__(cls, __name, __bases, __dict : TypedKeyMappingsConstrainedDict):
 
     def __call__(mcls, *args, **kwargs):
         instance = super().__call__(*args, **kwargs)
+
         instance.__post_init__()
         return instance
 
@@ -127,16 +128,6 @@ class Thing(Parameterized, metaclass=ThingMeta):
                         URL_path='/object-info') # type: ThingInformation
 
 
-    def __new__(cls, *args, **kwargs):
-        obj = super().__new__(cls)
-        # defines some internal fixed attributes. attributes created by us that require no validation but 
-        # cannot be modified are called _internal_fixed_attributes
-        obj._internal_fixed_attributes = ['_internal_fixed_attributes', 'instance_resources',
-                                        '_httpserver_resources', '_zmq_resources', '_owner', 'rpc_server', 'message_broker',
-                                        '_event_publisher']        
-        return obj
-
-
     def __init__(self, *, instance_name : str, logger : typing.Optional[logging.Logger] = None, 
                 serializer : typing.Optional[JSONSerializer] = None, **kwargs) -> None:
         """
@@ -183,10 +174,10 @@ class attribute, see docs.
         self._owner : typing.Optional[Thing] = None 
         self._internal_fixed_attributes : typing.List[str]
         self._full_URL_path_prefix : str
+        self._gui = None # filler for a future feature
+        self._event_publisher = None # type : typing.Optional[EventPublisher]
         self.rpc_server  = None # type: typing.Optional[RPCServer]
         self.message_broker = None # type : typing.Optional[AsyncPollingZMQServer]
-        self._event_publisher = None # type : typing.Optional[EventPublisher]
-        self._gui = None # filler for a future feature
         # serializer
         if not isinstance(serializer, JSONSerializer) and serializer != 'json' and serializer is not None:
             raise TypeError("serializer key word argument must be JSONSerializer. If one wishes to use separate serializers " +
@@ -211,24 +202,11 @@ class attribute, see docs.
 
 
     def __post_init__(self):
-        # self._prepare_resources()
+        self._prepare_resources()
         self.load_properties_from_DB()
         self.logger.info(f"initialialised Thing class {self.__class__.__name__} with instance name {self.instance_name}")
 
 
-    def __setattr__(self, __name: str, __value: typing.Any) -> None:
-        if  __name == '_internal_fixed_attributes' or __name in self._internal_fixed_attributes: 
-            # order of 'or' operation for above 'if' matters
-            if not hasattr(self, __name) or getattr(self, __name, None) is None:
-                # allow setting of fixed attributes once
-                super().__setattr__(__name, __value)
-            else:
-                raise AttributeError(f"Attempted to set {__name} more than once. " +
-                                     "Cannot assign a value to this variable after creation.")
-        else:
-            super().__setattr__(__name, __value)
-
-
     def _prepare_resources(self):
         """
         this method analyses the members of the class which have '_remote_info' variable declared
@@ -423,8 +401,9 @@ def event_publisher(self) -> EventPublisher:
     @event_publisher.setter
     def event_publisher(self, value : EventPublisher) -> None:
         if self._event_publisher is not None:
-            raise AttributeError("Can set event publisher only once")
-
+            if value is not self._event_publisher:
+                raise AttributeError("Can set event publisher only once")
+
         def recusively_set_event_publisher(obj : Thing, publisher : EventPublisher) -> None:
             for name, evt in inspect._getmembers(obj, lambda o: isinstance(o, Event), getattr_without_descriptor_read):
                 assert isinstance(evt, Event), "object is not an event"
@@ -658,4 +637,3 @@ def run_with_http_server(self, port : int = 8080, address : str = '0.0.0.0',
 
 
 
-

hololinked/server/utils.py

@@ -109,6 +109,7 @@ def run_coro_sync(coro : typing.Coroutine):
         eventloop = asyncio.get_event_loop()
     except RuntimeError:
         eventloop = asyncio.new_event_loop()
+        asyncio.set_event_loop(eventloop)
     if eventloop.is_running():
         raise RuntimeError(f"asyncio event loop is already running, cannot setup coroutine {coro.__name__} to run sync, please await it.")
         # not the same as RuntimeError catch above.  
@@ -126,6 +127,7 @@ def run_callable_somehow(method : typing.Union[typing.Callable, typing.Coroutine
         eventloop = asyncio.get_event_loop()
     except RuntimeError:
         eventloop = asyncio.new_event_loop()
+        asyncio.set_event_loop(eventloop)
     if asyncio.iscoroutinefunction(method):
         coro = method()
     else:

hololinked/server/zmq_message_brokers.py

@@ -2,6 +2,7 @@
 import os
 import threading
 import time
+import warnings
 import zmq
 import zmq.asyncio
 import asyncio
@@ -2188,7 +2189,21 @@ def register(self, event : "EventDispatcher") -> None:
             raise AttributeError(f"event {event._name} already found in list of events, please use another name.")
         self.event_ids.add(event._unique_identifier)
         self.events.add(event) 
-
+
+    def unregister(self, event : "EventDispatcher") -> None:
+        """
+        unregister event with a specific (unique) name
+
+        Parameters
+        ----------
+        event: ``Event``
+            ``Event`` object that needs to be unregistered. 
+        """
+        if event in self.events:
+            self.events.remove(event)
+            self.event_ids.remove(event._unique_identifier)
+        else:
+            warnings.warn(f"event {event._name} not found in list of events, please use another name.", UserWarning)
 
     def publish(self, unique_identifier : bytes, data : typing.Any, *, zmq_clients : bool = True, 
                         http_clients : bool = True, serialize : bool = True) -> None: 

tests/test_events.py

@@ -27,7 +27,7 @@ def push_events(self):
     def _push_worker(self):
         for i in range(100):
             self.test_event.push('test data')
-            time.sleep(0.01)
+            time.sleep(0.01) # 10ms
 
 
 

tests/test_thing_init.py

@@ -162,6 +162,7 @@ def test_7_servers_init(self):
     def test_8_resource_generation(self):
         # basic test only to make sure nothing is fundamentally wrong
         thing = self.thing_cls(instance_name="test_servers_init", log_level=logging.WARN)
+        # thing._prepare_resources()
         self.assertIsInstance(thing.get_thing_description(), dict)
         self.assertIsInstance(thing.httpserver_resources, dict)
         self.assertIsInstance(thing.zmq_resources, dict)