Skip to content
This repository was archived by the owner on Mar 25, 2024. It is now read-only.

Feature/post processing #21

Merged
merged 30 commits into from
Sep 30, 2021
Merged

Feature/post processing #21

merged 30 commits into from
Sep 30, 2021

Conversation

troehling
Copy link
Contributor

Implementation of new processing pipeline:

  • Using the same code for online and post processing
  • better performances due to better sizing of the packages
  • test case for post processing based on the results of previously used RawConverter
  • Using git lfs for storage of large files

Tobias Röhling added 23 commits September 6, 2021 13:30
- h5py is newly required for post-processing pipeline
- Logging lead to a resource leak due to the logging queue not being closed
- this could not be resolved with the implemented logging functionality
- logging was reduced to very basic python logging, which should be fine
- the previous logging was not really useful and overcomplicated things heavily
- massive refactoring of current pipeline implementation
- enable use of package processof and centroid calculator in different use cases more flexible
- enhance performance of centroiding by running in chunks of specific size, which gives ideal runtime of DBSCAN and Centroiding
- Implementation of test cases for new CentroidCalculator class and post processing pipeline
- Parameters: epsilon, min_samples, threshold, skip
- Introduced shared memory to enable changes from pymepix-viewer
 - Write TOA data to queue
 - Fix post processing to ignore included toa data
@troehling troehling requested a review from BenMoon September 27, 2021 11:58
Copy link
Contributor

@BenMoon BenMoon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall, this looks quite good.
In general, maybe we should employ a linter like flake8 or such.
Pls check the header in each file.

@troehling troehling dismissed BenMoon’s stale review September 30, 2021 14:11

Implemented all suggested changes and discussed with @BenMoon in person.

@troehling troehling merged commit f016ab5 into develop Sep 30, 2021
@troehling troehling deleted the feature/post_processing branch September 30, 2021 14:13
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants