-
Notifications
You must be signed in to change notification settings - Fork 6
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add a way to quantify plan progress #605
Comments
Obviously there is no universal metric for the progress of a plan, it depends on what the plan is trying to accomplish. Some plans cannot quantify their progress very well at all. For example an adaptive scan may have no idea of its total number of points upfront. For this reason I think it is best to have an optional way to include a self-reporting mechanism in a plan itself, maybe something similar to TQDM. For example: def my_scan(points: Spec[Movable]) -> MsgGenerator:
motors = points.axes()
yield from bps.mv(motors, starting_point)
for point in plan_progress(points.frames()):
yield from bps.mv(motors)
It is then the responsibility of a plan author to define how to report progress. How does that sound to you @stan-dot? Also, would this then become an issue on bluesky itself rather than blueapi? |
more than half of plans might have some metric that goes up, and for those that have an uncertain length the denominator of the progress bar could be just the previous run - and it would go beyond 100%. and TQDM seems most suited to a CLI use case. I don't think leaving this up to the plan author is the best idea. and still in the context of the RunEngine and a web UI the issue for blueapi remains to report it. unless it's again like with the data streaming case before, the question whether blueapi is one way street for data or not. |
How would you propose we do it then? |
to have a default, overridable way which for all scans in the form |
the length of the plan would be estimated from the params. |
So you want to inspect the params for |
that's an implementation thing, the issue is about a user need. One implementation could be handling the start an stop in the client side and listen for 'step' events, and increment |
This discussion is focused on implementation, as the requirements have already been agreed upon. I believe this calculation should be handled server-side, without relying on a default that may or may not suit the specific plan. Implementing it as suggested could lead to the following issues:
|
of course the adaptive scans won't have this re: missing a step - I thought that the RMQ message bus has a delivery guarantee.
I guess the server-side implementation would likely use some |
Fair enough, if we agree on server-side I think the decision I want out of this discussion is whether the implementation lives in blueapi or actual core bluesky (in which case I will close this issue and raise one there). I think @coretl will advocate for the latter since he is keen on the workflow of developing plans outside of blueapi and running them for users inside it. |
Depends on implementation. One idea is to use the existing document streams to work out progress.
If we go with this then I don't mind where the implementation goes, as I don't care too much about overall scan progress in iPython, only the progress of the current |
@coretl 's description is a more verbose version of what I had in mind when writing
This aligns with the websockets stuff, and the flow of information the other way, or with the bluesky-stomp library as used in blueapi. we could add an endpoint def get_plan_progress(job_id: string): Tuple[number, number]:
```
|
StartDocument does not guarantee scan shape or length. I think offering "progress reports if you provide scan shape information" is a reasonable give-and-take, and it's useful for analysis and reshaping down the line anyway. But if scan shape has any unknown axes, we can't report on the progress: [-1, 5, 4] (20 point inner scan an unknown number of times) is unknowable and so is [20, -1] (20 repetitions of an unknown number of points, as we can't tie seq_num to outer scan progress). If we're relying on the scan shape to figure out scan progress something something data reshaping service something something: maybe the progress in the UI should come from the data API, rather than adding more to the scan engine? |
Rather than trying to work out a general way to measure progress: the author of the plan should be able to publish the relative progress even if it is not linear, so could we provide a method (like There are other progress bar implementations which allow periodic updates, which could also be used as a model. This does not need to be perfect, and for adaptive scans it could also reasonably go retrograde if the plan suddenly decides that it wants to re-scan a region at a higher resolution. I assume that this is really a courtesy to the end user to see how long it will probably be until this scan is complete right? |
As a user I would like to see the progress of the plan I defined, be it in points scanned out of total number, or some other metric.
The text was updated successfully, but these errors were encountered: