Showcase of Components.

[BenediktBurger]

This is a showcase how some Components could be implemented (I use them in my lab). Merging will take place in smaller PRs.

This is a working version of Coordinator and some utils according to pymeasure/leco-protocol#38

As the message Layer is still under development, I used json with a list of commands, each command is again a list with a command type and optionally more information. For example a set command and an acknowledgement (I have a StrEnum for the command type):

[["S", {'property1': 7}]]
[["A"]]
[["S", {'property1': 7}], ["G", ["property1"]]  # this sets a value and gets it afterwards.

I can also upload an Actor and more, but they are not as polished as the Coordinator (but they do their job in the lab).

[bilderbuchi]

Very nice first step, thank you! It will take sometime until I can take a look through (I know you're not looking for a detailed review at this point).

In the meantime I will try to get CI set up so that we can run some checks against this.

[BenediktBurger]

I was not even looking for a rough review, just for a help, how an implementation might look like.
This can help Form the protocol and to understand, how I understand the protocol.

Obviously I'm glad for feedback, when you look at the code (especially those general things like filenames and structure).

[bilderbuchi]

Alles klar!
One thing that might be useful to catch more shortcomings of the spec is to implement pyleco "by the book", i.e. following the specification as written (not as we know it in our head).
I know such detachment is probably gonna be difficult as we also wrote the spec, but during review reviewers should probably compare to the spec regularly.

[BenediktBurger]

In fact, yesterday I added tests according to the leco specifications (in the leco pr) and found a mistake (which is still in this PR).

I reworked the Coordinator according to the specs (including tests) before creating this PR.

[BenediktBurger]

I added an Actor (still called "Controller" in my implementation) and the Data Protocol Proxy Server and a Data Protocol Publisher.

I consider this PR as a showcase of my current code and do not intend to merge this PR at all.

For serious review processes (intended for merging), I will use individual PRs.

[BenediktBurger]

Now the communication is done via jsonrpc messages.

[bilderbuchi]

I think, effectively, you have become the primary dev of pyleco. I don't know when I would find the time to review a 7k lines beast of a PR meaningfully. @bklebel how about you?
Considering that, do you want to just merge this into main unreviewed, and do any further changes as more PRs (or however best). I'm prepared to take this one on trust in your ability. :-)

[bklebel]

I agree. I would very much like to work together on this code, and contribute a significant part, but quite frankly, my responsibilities in our research group have changed a bit, so that I cannot implement pyleco in our lab right now and supplant the backend of CryostatGUI with it right now, even if I would much prefer to do so over some other responsibilities. However, only doing this would really enable me (right now) to contribute meaningfully, since then I would really have the time to look into it.

I really like what you built here @bmoneke, so for now, also from my side, feel free to merge, and and I very much hope I can contribute more in the future.

[BenediktBurger]

Ok. My idea is, to merge it in small parts.
In the PRs I'm going to ask you questions regarding structure (i.e. separation of folders / modules, naming schemes etc.) instead of a full code review.
Until now, you have to import every class from its module, there are no shortcuts like pymeasure.instrument yet (code will continue to work, if a shortcut is added, but not if it is removed).

[bklebel]

Okay, that sounds like a good way to proceed, I'm up for it.

[github-actions]

Coverage Report

File	Stmts	Miss	Cover	Missing
pyleco
test.py	61	35	39%	36, 39, 42, 53–60, 63, 66–67, 70, 73, 76, 84, 88, 93–97, 100–101, 106–110, 113, 116, 119, 122–124, 127–128
pyleco/actors
actor.py	134	100	23%	64–81, 84–90, 94–95, 98, 101–102, 109–132, 135, 139, 143, 151–152, 159, 166–174, 178, 181, 184, 189, 193, 196, 199, 204–206, 210–219, 223–230, 234–238, 243–250
motorController.py	102	102	0%	2–170
pyleco/coordinators
coordinator.py	224	170	21%	89–117, 121–138, 141–144, 147, 150, 154–160, 164, 177, 188–193, 200–202, 205–212, 223–234, 238–245, 252–274, 277–288, 291–295, 307–325, 328–338, 347, 351, 354–356, 362–367, 370–375, 378–382, 385–391, 401–407, 411–412, 415, 419, 423–425, 430, 435–438
proxy_server.py	71	59	15%	58, 72–96, 126–131, 135–177
pyleco/core
internal_protocols.py	12	2	83%	60, 70
message.py	62	23	66%	68, 74, 92–95, 99–101, 104, 109, 113, 125, 128–144, 148
rpc_generator.py	13	5	53%	41–45, 50, 54
serialization.py	50	11	75%	76, 80, 83–84, 90–92, 97–99, 126
pyleco/directors
coordinator_director.py	15	5	67%	38, 42, 46, 50, 54
director.py	92	64	25%	56–69, 72–73, 77, 80, 83, 88–97, 101–103, 106–109, 115–120, 126–131, 136, 147–148, 151–152, 155, 159, 164–167, 172–175, 182, 195–196, 202–203
motor_director.py	94	94	0%	25–197
starter_director.py	30	17	32%	42, 50–52, 60–62, 70–72, 80–82, 91–93, 97
transparent_director.py	27	27	0%	25–95
pyleco/management
data_logger.py	210	204	2%	33–355
starter.py	185	145	16%	50–53, 66–74, 114–131, 134–141, 148–164, 168–169, 172–173, 177–199, 202–203, 207–223, 226–228, 231–232, 236–237, 240–241, 245, 252–259, 263–279, 283–287, 291–311
pyleco/management/test_tasks
test_task.py	24	9	66%	15, 18, 22, 26, 30, 33, 39–41
pyleco/utils
communicator.py	142	111	17%	73–86, 90–96, 100–105, 109–110, 113, 117–120, 124, 128–131, 134–142, 145–149, 153, 156, 160–181, 185–195, 199–210, 213–215, 220–233, 237–240, 243, 247
coordinator_utils.py	308	211	25%	71–73, 76, 80, 83, 86, 89, 92–93, 98–100, 103, 112, 118, 133–136, 139, 145, 161–162, 166–168, 172–176, 179–182, 186, 189, 192, 197–199, 202–203, 206, 209, 212, 215, 218, 225–232, 235–241, 244–249, 253–273, 277–284, 287–294, 297–309, 313–319, 338–343, 346–350, 353–361, 364–366, 369, 372–387, 392–405, 411–418, 422–423, 426–430, 433–435, 444–450, 459, 462, 465–468, 471–474, 477–480, 483, 486–489, 492, 495–500, 503–515, 518–521
events.py	9	2	78%	41, 44
extended_message_handler.py	77	58	20%	39–41, 44–47, 51–56, 60–64, 67–68, 71–87, 90, 94–98, 101–108, 112–116, 119–124, 128–129
extended_publisher.py	21	21	0%	25–59
listener.py	141	94	29%	71–84, 88, 92, 96, 101–111, 118–120, 123, 127–128, 134–136, 145, 150, 155–160, 163, 166–168, 173–177, 181–185, 189, 193, 196, 199, 208–234, 242–243, 274–276, 281–284, 288–291, 295–305
message_handler.py	193	146	20%	68–102, 106–108, 111, 114, 118, 122, 126–127, 130, 134–138, 143–150, 154–158, 164, 167, 176–187, 194–202, 210–217, 221–223, 230–249, 252–253, 256–264, 267–270, 273–275, 278–280, 283–285, 289–298, 302, 305, 312–315, 318–323, 326–327, 336–340
parser.py	20	11	32%	54–65
pipe_handler.py	133	99	19%	52–58, 62–64, 71–78, 82–87, 91–99, 109–114, 120–122, 126–128, 131, 134–135, 139–141, 145–149, 152–168, 173–174, 179–187, 191, 196–199, 203, 206–208, 212, 216–218, 222–224, 227–229, 232, 235–237
publisher.py	63	43	28%	60–78, 81, 85, 90, 94–100, 105–111, 118–124, 128–130, 136–139
qt_listener.py	25	25	0%	25–79
timers.py	16	8	50%	38–39, 42–43, 53–54, 59–60
zmq_log_handler.py	40	27	25%	48–52, 56–74, 79–82, 85
TOTAL	2607	1928	22%

Coverage Summary

Tests	Skipped	Failures	Errors	Time
2	1 💤	0 ❌	1 🔥	1.526s ⏱️

[BenediktBurger]

Superseded by #10 due do branch name change.