Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Close all HandleTie servants created during import #394

Merged
merged 14 commits into from
Sep 7, 2023

Conversation

jburel
Copy link
Member

@jburel jburel commented Aug 25, 2023

This PR replaces #354

Run an import of one or multiple filesets and watch the creation/deletion of HandleTie servants server-side e.g. using tail -F Blitz-0.log | grep HandleTie.
All the handles should now be closed and the service closed.
Thumbnails are no longer loaded

To test:

  • regular import of single files
  • regular import of directories (1-10 & >50 to test the lightfileimport)
  • import of broken files (failing server-side)
  • import of HCS data
  • import of some images which take longer to process on the server side and closing insight before the server side import is finished.

Copy link
Member

@sbesson sbesson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested against a remote OMERO server in a separate geographical region using a local build of OMERO.insight from the HEAD of this PR across the scenarios described in the description, specifically:

  • two folders containing each 10 & 30 fake files
  • a plate.fake file with an accompanying plate.fake.ini with plateRows=4, plateCols=4 and fields=2
  • a slow.fake file with an accompanying slow.fake.ini with sleepOpenBytes=30000
  • an OME-XML file with an incorrect detector settings reference, currently failing import side
    wrong-detector-settings-mk1.ome.xml.zip

All scenarios were tested individually as well as using combinations and the logs of the server were monitored using the command mentioned in the description.

regular import of directories (1-10 & >50 to test the lightfileimport)

50 files was the threshold I remembered for the switch to the light file import UI but after verification the current limit is 100. This threshold is configurable using the importer.xml

<entry name="/options/DetailedImportFileLimit" type="integer">100</entry>
and this value was lowered to 20 for the sake of my testing.

Confirmed that in all scenarios above, all the HandleTie servants are appropriately closed when the import completes and no leftover servant ends up being unregistered when closing OMERO.insight and terminating the session.
The only exception to this rule was the testing of the early closure of the importer/insight client after uploading a sample file with a long-processing time (slow.fake). In that case, the servant is not cleaned until the end of the server-side import steps as expected.

A few inline comments/clarifications related to code review. Possibly the biggest impact outside the scope of the original fix is that the View button and the checkbox in the standard importer have disappeared.

With OMERO.insight 5.8.1,
Screenshot 2023-08-28 at 11 53 04
Screenshot 2023-08-28 at 11 53 35

With this PR
Screenshot 2023-08-28 at 11 54 08
Screenshot 2023-08-28 at 11 54 21

For import failures, the cross icon is still visible and the Failed button gives access to the import logs

Screenshot 2023-08-28 at 12 08 24

I will leave @dominikl to perform his own testing and add his comments. In addition to the above, I would like to add a scenario testing the light UI import with a mixed case of files with successful imports and failures to confirm the statuses are as expected. But that can be scheduled for another round of testing especially if additional commits are added.

Copy link
Member

@sbesson sbesson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Retested with the last commits using all scenarios above as well as a directory using the light UI import with a mixture of successes and failures. Everything works as expected. The completion icon/button are displayed in the importer UI and all servants are unregistered server-side.

Copy link
Member

@dominikl dominikl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. Tested different scenarios: Few images (with detailed info), many images (with just the summary, set limit to 20 like @sbesson mentioned). Plates. Multiple import tabs. Closing Insight after "File upload" step, etc. UI still behaves like expected. And there's one "Unregistered servant" for each "Added servant" in the Blitz log.
👍
For testing I used fresh installed server on rocky9 VM in locally running Virtualbox.

@jburel jburel merged commit 7839c80 into ome:master Sep 7, 2023
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants