Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

On-the-fly Reconstruction #393

Draft
wants to merge 196 commits into
base: live-recon-compatibility
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from 195 commits
Commits
Show all changes
196 commits
Select commit Hold shift + click to select a range
646b09b
pin README links
talonchandler Apr 19, 2023
2d26387
initial cli skeleton
talonchandler Apr 22, 2023
51fff6e
first-pass docs
talonchandler Apr 22, 2023
5e72fb9
first-pass pydantic models
talonchandler Apr 23, 2023
d59a7fc
improve typing and testing of settings.py
talonchandler Apr 23, 2023
bb8dbed
full prototype of `recorder calculate_transfer_function`
talonchandler Apr 23, 2023
d36f6f6
complete cli draft
talonchandler Apr 24, 2023
2a3dc7f
buttons work via cli
talonchandler Apr 24, 2023
f3a90db
Merge pull request #347 from mehta-lab/cli-dev
talonchandler Apr 25, 2023
f71bc4e
connect background corrections
talonchandler Apr 25, 2023
1b07d72
update LC_DEVICE_NAME
talonchandler Apr 25, 2023
8791f87
20220920 -> 20230418
talonchandler Apr 25, 2023
c728cad
document check for "PC software version"
talonchandler Apr 25, 2023
8530849
three -> two
talonchandler Apr 25, 2023
5013c4f
mention changed name
talonchandler Apr 25, 2023
1d67e7b
remove remaining converter util function
ziw-liu Apr 26, 2023
23d8aa5
Merge pull request #353 from mehta-lab/connect-bg-correction
talonchandler Apr 26, 2023
cd1d589
fix syntax error
talonchandler Apr 26, 2023
4579350
Merge pull request #355 from mehta-lab/converter-cleanup
talonchandler Apr 26, 2023
746e83e
Merge branch '0.4.0dev' into upgrade-mm2
talonchandler Apr 26, 2023
29e152f
improve software+firmware instructions
talonchandler Apr 26, 2023
3c7a17e
device adapter -> driver
talonchandler Apr 26, 2023
3725be1
direct download link
talonchandler Apr 26, 2023
76bf59a
Merge pull request #354 from mehta-lab/upgrade-mm2
talonchandler Apr 26, 2023
d3a4370
add regularization settings
talonchandler Apr 26, 2023
4144310
connect recon params to cli
talonchandler Apr 26, 2023
f98dbe3
no kwargs
talonchandler Apr 26, 2023
00b2e11
invert 2d phase to match
talonchandler Apr 26, 2023
3d5b987
more background integration
talonchandler Apr 26, 2023
3011258
Merge pull request #356 from mehta-lab/connect-reg-params
talonchandler Apr 27, 2023
7367a84
style cleanup
ziw-liu Apr 26, 2023
0d7d93d
WIP draw overlay with a hook
ziw-liu Apr 27, 2023
7d5f3b8
eager mode for now
ziw-liu Apr 27, 2023
081a472
fix typo
ziw-liu Apr 27, 2023
52a2d49
move large overlay info banner
ziw-liu Apr 27, 2023
3f576e6
reduce info banner size limit
ziw-liu Apr 27, 2023
803ba28
avoid repeated updates
ziw-liu Apr 27, 2023
88d7c98
only use event information
ziw-liu Apr 27, 2023
df703e1
bump mm version
talonchandler Apr 27, 2023
7f78f47
update instructions
talonchandler Apr 27, 2023
0e202e9
Merge pull request #358 from mehta-lab/bump-mm
talonchandler Apr 27, 2023
7911a6b
remove acquisition mode suffix from birefringence
ziw-liu Apr 27, 2023
ec49eb8
detect orientation layer and set colormap
ziw-liu Apr 27, 2023
c2b07b5
log
ziw-liu Apr 27, 2023
c4780fc
increase log level
ziw-liu Apr 27, 2023
c042e63
remove copy and test that input is not modified
ziw-liu Apr 27, 2023
5518350
wait until the birefringence layers are spawn
ziw-liu Apr 27, 2023
31c3e9b
"changed" -> "moved" hook
talonchandler Apr 27, 2023
9b1906f
Merge pull request #357 from mehta-lab/overlay-hook
talonchandler Apr 27, 2023
1fb2113
Hot fix for flip + reflection convention
talonchandler Apr 28, 2023
1e1bc4b
reuse variable
ziw-liu Apr 28, 2023
33eeaed
match repetitive birefringence layers
ziw-liu Apr 28, 2023
d8c1870
pass overlay name
ziw-liu Apr 28, 2023
84441cc
use names with suffices
ziw-liu Apr 28, 2023
c50815c
set colormap for multiple orientation layers
ziw-liu Apr 28, 2023
e15e7d0
fix key
ziw-liu Apr 28, 2023
72fabf4
remove unused function
ziw-liu Apr 28, 2023
bd0f8f3
sort once
ziw-liu Apr 28, 2023
9d84348
pin upgraded pm
talonchandler May 12, 2023
92802de
update to version compatible with multithreading
talonchandler May 12, 2023
beb79f5
Merge pull request #373 from mehta-lab/upgrade-pm
talonchandler Jun 1, 2023
aadfec0
update waveorder dependency
talonchandler Jun 7, 2023
6a8509c
Merge pull request #359 from mehta-lab/multi-overlay
talonchandler Jun 7, 2023
4ccf725
`illumination_wavelength` -> `wavelength_illumination`
talonchandler Jun 8, 2023
595eddf
Merge pull request #376 from mehta-lab/rename-wavelength
talonchandler Jun 9, 2023
d844477
added tests for compute-transfer-function and separated function logic
Jun 13, 2023
971e226
added documentation for compute-transfer-function
Jun 13, 2023
c1ff28d
test
Jun 13, 2023
6aeba57
`anisotropic_thin` -> `oriented_thick`
talonchandler Jun 13, 2023
bbee1d1
changed function names and removed returns
Jun 13, 2023
b08b82f
rename output test and got output-path option working
Jun 13, 2023
178522c
added a possible template for testing different configurations
Jun 14, 2023
3ae0a88
Merge pull request #378 from mehta-lab/rename-anisotropic
talonchandler Jun 14, 2023
16decfd
typo
talonchandler Jun 14, 2023
a786b56
removed test file
Jun 14, 2023
74f1fd6
matched function name and fixed typo
Jun 14, 2023
dcc295c
remove unused reconstruction settings class
talonchandler Jun 14, 2023
ee50fcb
added unit tests, fixed typos
clinton-huynh Jun 14, 2023
c2c17f2
added tests, fixed docstring, added fixtures
clinton-huynh Jun 15, 2023
7353be9
cleaned up tests
clinton-huynh Jun 15, 2023
e46a4e1
revised settings with examples
talonchandler Jun 15, 2023
a91da6a
initial revision of cli
talonchandler Jun 15, 2023
e775e69
document and test `model_to_yaml` and `yaml_to_model`
talonchandler Jun 15, 2023
a2c96ae
update compute_transfer_function
talonchandler Jun 16, 2023
656506a
update apply_inverse_transfer_function
talonchandler Jun 16, 2023
9db02f4
doc strings
talonchandler Jun 16, 2023
0280edc
improved names
talonchandler Jun 16, 2023
0daadf7
`-h` for help
talonchandler Jun 16, 2023
5dd673d
first pass `reconstruct` cli
talonchandler Jun 16, 2023
f431d89
rename to prepare for merge
talonchandler Jun 16, 2023
6f8f05f
Merge branch '0.4.0dev' into testing_compute_transfer
talonchandler Jun 16, 2023
a9c2a3d
remove `or "None"`
talonchandler Jun 16, 2023
9e11dee
fixed docstring parameters
clinton-huynh Jun 16, 2023
03f024c
cleaned imports and fixtures
clinton-huynh Jun 16, 2023
9158a51
Merge branch 'testing_compute_transfer' of github.com:mehta-lab/recOr…
clinton-huynh Jun 16, 2023
6ce020d
complete reconstruct draft
talonchandler Jun 16, 2023
020f2e5
add 3d fluorescence compute-tf
talonchandler Jun 16, 2023
1d6b7e7
fluorescence apply_inverse_transfer_function
talonchandler Jun 16, 2023
ace833e
add `axial_flip` parameter to phase
talonchandler Jun 16, 2023
f96e519
abstracted fixture
clinton-huynh Jun 16, 2023
b65591b
Merge pull request #377 from mehta-lab/testing_compute_transfer
clinton-huynh Jun 16, 2023
853b2b8
first-pass acquisition_workers.py
talonchandler Jun 16, 2023
a7691eb
Merge branch '0.4.0dev' into fluor-cli
talonchandler Jun 16, 2023
7f7852e
fix bugs and tests
talonchandler Jun 17, 2023
f205fdb
simplify tests
talonchandler Jun 20, 2023
acbbcb9
typo
talonchandler Jun 20, 2023
dacac8b
test fluorescence and refactor compute_tf tests
talonchandler Jun 20, 2023
b62d9f7
Merge branch '0.4.0dev' of github.com:mehta-lab/recOrder into 0.4.0dev
clinton-huynh Jun 20, 2023
30584f0
update default settings
talonchandler Jun 21, 2023
e95eb05
typo
talonchandler Jun 21, 2023
67ad0c4
apply_inverse_transfer_function tests
talonchandler Jun 21, 2023
4cfd8ac
pass birefringence tests
talonchandler Jun 21, 2023
7ed06f1
whoops
talonchandler Jun 21, 2023
c0b618b
found image data
clinton-huynh Jun 23, 2023
0dbf1cb
clean imports
talonchandler Jun 23, 2023
f0742f2
fix tests
talonchandler Jun 23, 2023
ece2ac3
don't accept empty config
talonchandler Jun 23, 2023
933ae85
`options` -> `option`
talonchandler Jun 23, 2023
1aa7f6a
validate fov path
talonchandler Jun 23, 2023
ea9d4c9
unused import
talonchandler Jun 23, 2023
5d98463
removed iterator, updated to run diff acq orders
clinton-huynh Jun 27, 2023
fc9a4d5
iterates through all dim for every acq order
clinton-huynh Jun 28, 2023
3d60248
grouped together z-stack data
clinton-huynh Jun 30, 2023
ee88181
simplify
talonchandler Jul 5, 2023
dbab887
remove redundant "scheme" parameter
talonchandler Jul 5, 2023
2a48d56
fix docs
talonchandler Jul 5, 2023
99125b5
basic test of reconstruct
talonchandler Jul 5, 2023
b0e592a
check input channel names
talonchandler Jul 5, 2023
bfbdd5c
index into input dataset
talonchandler Jul 5, 2023
2a222e4
document automatic configuration file generation
talonchandler Jul 5, 2023
d4341b2
improved instructions
talonchandler Jul 5, 2023
a77c61f
better default to match example
talonchandler Jul 5, 2023
0e9b554
default 4 states
talonchandler Jul 5, 2023
a6d2fee
revamped examples folder
talonchandler Jul 5, 2023
123ad08
remove `compute` module
talonchandler Jul 5, 2023
7f70c72
write entire acq into zarr for ptcz, ptzc order
clinton-huynh Jul 5, 2023
d00ea88
simplify examples folder
talonchandler Jul 5, 2023
cbc4411
remove unused imports
talonchandler Jul 5, 2023
0ce1b84
remove old reconstructors
talonchandler Jul 5, 2023
d375c9a
remove listener vestiges
talonchandler Jul 5, 2023
8f328c9
bf fixes
talonchandler Jul 6, 2023
14e9f53
integration fixes for phase recon
talonchandler Jul 6, 2023
f687b05
ndtiff to zarr on all acq orders
clinton-huynh Jul 6, 2023
0d21512
birefringence integration fixes
talonchandler Jul 6, 2023
cd2fae5
changed acq_mode to str
clinton-huynh Jul 6, 2023
af6d180
background acquisition integration
talonchandler Jul 7, 2023
20a673f
tidy background recons
talonchandler Jul 7, 2023
2d0895c
bkg correction integration fixes
talonchandler Jul 7, 2023
9c96e54
Enable phase w/o calibration
talonchandler Jul 7, 2023
f98a2a8
Add `cli_structure` documentation
talonchandler Jul 7, 2023
ebd6b1d
fix doc....specifiy input.zarr/0/0/0
talonchandler Jul 11, 2023
4582bf5
Merge pull request #381 from mehta-lab/fluor-cli
talonchandler Jul 11, 2023
9d519c6
convert to zarr from ndtiff
clinton-huynh Jul 14, 2023
c4d05f8
listener works on automaton
clinton-huynh Jul 19, 2023
3b2e185
working threaded napari opener
clinton-huynh Jul 21, 2023
4e93fd8
to test on hummingbird
clinton-huynh Jul 21, 2023
78cc673
listen and open working on hummingbird
talonchandler Jul 22, 2023
9a9ee16
Merge branch 'mda-listener' of github.com:mehta-lab/recOrder into mda…
clinton-huynh Jul 24, 2023
8a6d561
clean up
clinton-huynh Jul 24, 2023
39b58fa
works on hummingbird
clinton-huynh Jul 24, 2023
47d16b6
changed initialize zarr store
clinton-huynh Jul 25, 2023
d030bab
Merge branch '0.4.0dev' of github.com:mehta-lab/recOrder into 0.4.0dev
clinton-huynh Jul 25, 2023
f23fc38
found image data
clinton-huynh Jun 23, 2023
0b8ec44
removed iterator, updated to run diff acq orders
clinton-huynh Jun 27, 2023
f2616a1
iterates through all dim for every acq order
clinton-huynh Jun 28, 2023
a51f715
grouped together z-stack data
clinton-huynh Jun 30, 2023
5363a8e
write entire acq into zarr for ptcz, ptzc order
clinton-huynh Jul 5, 2023
6175547
ndtiff to zarr on all acq orders
clinton-huynh Jul 6, 2023
f03d59a
changed acq_mode to str
clinton-huynh Jul 6, 2023
7ed87de
convert to zarr from ndtiff
clinton-huynh Jul 14, 2023
b97baf6
listener works on automaton
clinton-huynh Jul 19, 2023
356c1d7
working threaded napari opener
clinton-huynh Jul 21, 2023
8d4e5a6
to test on hummingbird
clinton-huynh Jul 21, 2023
80f5dfc
listen and open working on hummingbird
talonchandler Jul 22, 2023
22e901b
clean up
clinton-huynh Jul 24, 2023
ef5507e
works on hummingbird
clinton-huynh Jul 24, 2023
991161a
changed initialize zarr store
clinton-huynh Jul 25, 2023
de5fe65
Merge branch 'mda-listener' of github.com:mehta-lab/recOrder into mda…
clinton-huynh Jul 25, 2023
a0db0ad
z-stack done logic
clinton-huynh Jul 25, 2023
3ba377b
progress towards reconstruction
clinton-huynh Jul 26, 2023
2754c8b
Merge 'new-mda-listener' into main-mda-listener
clinton-huynh Jul 26, 2023
b53f7c0
applying reconstruction but not to all positions
clinton-huynh Jul 26, 2023
2506fcc
reconstruction working
clinton-huynh Jul 26, 2023
9c162ef
documentation
clinton-huynh Jul 27, 2023
e25765d
fixed offset bug
clinton-huynh Jul 27, 2023
6a5efcd
Merge recon-compatibility to main-mda-listener
clinton-huynh Jul 28, 2023
ddaca6c
working reconstruction with ReconstructionSettings
clinton-huynh Jul 28, 2023
572590f
working reconstruction w ReconstructionSettings
clinton-huynh Jul 28, 2023
84dcb74
benchmark
clinton-huynh Jul 29, 2023
e954e40
recon works for birefringence
clinton-huynh Aug 2, 2023
61d81d3
clean up
clinton-huynh Aug 2, 2023
b2cedbf
documentation and comments
clinton-huynh Aug 3, 2023
e830928
untracked unnecessary files
clinton-huynh Aug 3, 2023
f0f08e0
untracked another file
clinton-huynh Aug 3, 2023
41afe4d
untracked pluginsnap
clinton-huynh Aug 3, 2023
80234e2
merge artifacts
clinton-huynh Aug 3, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 15 additions & 12 deletions recOrder/cli/apply_inverse_transfer_function.py
Original file line number Diff line number Diff line change
Expand Up @@ -144,26 +144,29 @@ def apply_inverse_transfer_function_cli(
output_z_shape = input_dataset.data.shape[2]

output_shape = (
t_shape,
input_dataset.data.shape[0],
len(channel_names),
output_z_shape,
) + input_dataset.data.shape[3:]

# Create output dataset
output_dataset = open_ome_zarr(
output_path, layout="fov", mode="w", channel_names=channel_names
output_path, layout="fov", mode="a", channel_names=channel_names
)
output_array = output_dataset.create_zeros(
name="0",
shape=output_shape,
dtype=np.float32,
chunks=(
1,
1,
1,
if 0 in time_indices:
output_array = output_dataset.create_zeros(
name="0",
shape=output_shape,
dtype=np.float32,
chunks=(
1,
1,
1,
)
+ input_dataset.data.shape[3:], # chunk by YX
)
+ input_dataset.data.shape[3:], # chunk by YX
)
else:
output_array = output_dataset[0]

# Load data
tczyx_uint16_numpy = input_dataset.data.oindex[:, channel_indices]
Expand Down
7 changes: 7 additions & 0 deletions recOrder/cli/reconstruct.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,10 +23,17 @@ def reconstruct(input_data_path, config_path, output_path):
convenience function for a `compute-tf` call followed by a `apply-inv-tf`
call.

<<<<<<< HEAD
ziw-liu marked this conversation as resolved.
Show resolved Hide resolved
See /examples for example configuration files.

Example usage:\n
$ recorder reconstruct input.zarr/0/0/0 -c /examples/birefringence.yml -o output.zarr
=======
See /examples/settings/ for example configuration files.

Example usage:\n
$ recorder reconstruct input.zarr/0/0/0 -c /examples/settings/birefringence.yml -o output.zarr
>>>>>>> new-mda-listener
"""

# Handle transfer function path
Expand Down
Loading
Loading