Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix delayed data frame updates #55

Merged
merged 1 commit into from
Aug 31, 2023

Conversation

ajoubertza
Copy link
Contributor

During data streaming, the "pending" stage is removed, so that clients gets FrameData as soon as possible. A disadvantage is that the client may receive some corrupt data. The user is able to detect this condition, so the benefit is thought to outweigh the disadvantage.

API changes:

  • Writing HDF5 files via write_hdf_files, an exception will now be raised if there's an overrun.
  • The command line tool pandablocks hdf will have exit code 1 (instead of 0) if there's an overrun.

Documentation and tests updated accordingly.

Closes #54

During data streaming, the pending stage is removed, so
that clients gets `FrameData` as soon as possible. A
disadvantage is that the client may receive some corrupt
data. The user is able to detect this condition, so
the benefit is thought to outweigh the disadvantage.

When writing HDF5 files via `write_hdf_files`, an
exception will now be raised if an overrun occurs.

Documentation and tests updated accordingly.

Closes PandABlocks#54
@codecov
Copy link

codecov bot commented Aug 31, 2023

Codecov Report

Patch coverage: 100.00% and no project coverage change.

Comparison is base (1581223) 96.37% compared to head (34c09eb) 96.37%.

Additional details and impacted files
@@           Coverage Diff           @@
##           master      #55   +/-   ##
=======================================
  Coverage   96.37%   96.37%           
=======================================
  Files          12       12           
  Lines        1159     1160    +1     
=======================================
+ Hits         1117     1118    +1     
  Misses         42       42           
Files Changed Coverage Δ
pandablocks/responses.py 100.00% <ø> (ø)
pandablocks/connections.py 98.98% <100.00%> (-0.03%) ⬇️
pandablocks/hdf.py 97.39% <100.00%> (+0.11%) ⬆️

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@coretl coretl merged commit e24fdfc into PandABlocks:master Aug 31, 2023
6 checks passed
@coretl
Copy link
Contributor

coretl commented Aug 31, 2023

Excellent, thank you!

@ajoubertza ajoubertza deleted the remove-pending-data-step branch August 31, 2023 16:45
@ajoubertza
Copy link
Contributor Author

Thanks for the review @coretl. Any chance we can get a new release soon? There are a few useful features for us that have been merged.

@coretl
Copy link
Contributor

coretl commented Sep 1, 2023

Sure, I'm just getting #56 ready (which may interest you if you are using the asyncio client with flush_period set) then will make a release

@coretl
Copy link
Contributor

coretl commented Sep 18, 2023

0.5.1 has been released (thanks @evalott100!)

@ajoubertza
Copy link
Contributor Author

Thanks @coretl and @evalott100 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

How to get data frames immediately, instead of one frame late?
2 participants