v1.0rc2 testing (on OSX Catalina / Chrome 90.0.4430.93) #153

craig-willis · 2021-05-12T18:54:56Z

Test Plan

https://github.com/whole-tale/wt-design-docs/issues/new?template=TEST_PLAN.md

Note: For all tests, repeat for supported browser/OS combinations.

Preconditions:

Clear browsing history or open incognito/private session
Disconnect all third party integrations

Splash page

Splash page

Goto https://dashboard.stage.wholetale.org
Confirm Terms of Service link
Confirm acknowledgements (NSF, XSEDE/Jetstream, Stata)
Confirm partner logos

Authentication

Basic login flow

Goto https://dashboard.stage.wholetale.org
Select "Access Whole Tale" button. You should be prompted to login via Globus Auth.
Select your organization and login
You should be taken to the "My Tales" page
Your Gravatar should display in the upper right

NOTE: Post auth re-display of splash page is unexpected.

Basic logout flow

Select the "User" menu
Select the "Log out" menu item. You should be taken to the login page.
Select "Access Whole Tale" button.
Note that you are not prompted to login.
Restart your incognito session
Select "Access Whole Tale" button. You should be prompted to login via Globus Auth.

Return-route for non-logged in users

Logout from Whole Tale
Attempt to access https://dashboard.stage.wholetale.org/public
You should be redirected to https://dashboard.stage.wholetale.org/login?rd=%252Fpublic
Select "Access Whole Tale".
You should be redirected to the "Public Tales" page.

Navigation

General navigation

Select "Tale Dashboard" link. "My Tales" page should display
Click "Create New Tale" > "Create New Tale". The Create Tale modal should display. Click "Cancel" to close the modal
Select the "i" icon at the upper right corner; it should open the User Guide in new tab
Select the "Notification" button next to the "i" icon; it should display the notification panel
Select the "Report a problem" button; it should open https://github.com/whole-tale/whole-tale/issues in new tab
Mouse over a Tale and select "View". The Run page should display
Click the "user" icon at the top right

You should see a dropdown expand
The dropdown should contain the user's first/last name and gravatar image
The dropdown should provide a link to the Settings page
The dropdown should provide an option to Logout
Select the "Settings" option; it should bring you to the Settings page

TEST: Minor test case inconsistency re: "report a problem" should be "bug" icon

Tale Dashboard

Preconditions:

Assumes production Tales present (e.g., LIGO, materials, etc).
No running instances
General

Select "My Tales" link. Confirm "My Tales" page displays
Confirm that no Tales are running
Select "Shared with Me" link. Confirm "Shared with Me" page displays

Search

Select "Public Tales" link. Confirm "Public Tales" page displays
Enter "Ligo" as search term. Confirm only LIGO tale displays (by K. Kowalik)
Clear search term. Confirm all tales display

TEST: Minor test inconsistency with search for "Ligo" instead of "LIGO"

View tale

Select "View" button on LIGO Tale
Confirm that Metadata page displays by default
Confirm that you cannot delete or edit Tale properties
Select Back button. Confirm that you are taken back to My Tales page
Select "View" button on Tale you own
1. Confirm that the Run and Close buttons are present
2. Confirm that you can edit Tale properties
3. Select "Close"
4. Confirm that you are taken back to "Public Tales"

FAIL: Interact displays by default (known)
TEST: Test case inconsistencies. Step 4 should be Public Tales and step 5.4 should be My Tales

Launch instance

Select "Run Tale" button on "Ligo" tale
1. Confirm you are prompted to Copy and Run
2. Select "Confirm"
3. Confirm that you are redirected to the Tale's "Metadata" page
4. Confirm that the "Edit" button is present
5. Select "Run Tale"
6. Confirm that the "Interact" page is displayed with spinner
7. Confirm that the notification dialog appears
8. Confirm that the step progression is correct
9. Confirm that the "View Logs" button shows the build logs in a modal
10. Navigate to the "My Tales" page
11. Confirm that the Tale is shown under "Currently Running"
12. Select "View" on the Tale card
13. Confirm that "Interact" page is displayed
Select "Stop" to end the Tale instance
1. Confirm that the Tale has stopped
2. Confirm that the button has changed to Run
Navigate to Browse
1. Confirm that there are no Tales under "Currently Running"
2. Select the "Run Tale" button on a Tale you own
3. After the instance is launched, navigate back to the My Tales page
4. Try to delete the Tale (not instance)
5. Confirm you cannot select the delete button on the My Tales page
6. Confirm that the "Run Tale" button has been replaced with "Stop Tale"
7. Confirm that the Tale is shown at the top of the page under "Currently running"

FAIL: Redirected to interact, not metadata on copy on launch
FAIL: Redirected to metadata on view after run (expected interact)

Managing Data

Preconditions

Empty home directory
No registered data
Register General

Open an owned Tale
Open the data registration panel from the 'External Data' tab
Confirm list of providers matches list of supported providers (Globus, DataONE, Dataverse, Zenodo)
TEST: Need to correct step for opening search modal

Register DataONE data

Open an owned Tale
Open the data registration panel from the 'External Data' tab
Confirm "Search for Data Sources" modal displays
Search for DOI 10.5065/D6862DM8
Confirm "Humans and Hydrology" dataset is found on DataONE.
Select Register
Confirm dataset appears in Manage Data list
Confirm additional URL formats:

TEST: Need to correct step for opening search modal and titles for additional URLs would be helpful
NOTE: Updated DASHBOARD_URL to point to prod_cn on stage for this to work.

Dataverse

Register dataset doi:10.7910/DVN/SZ9YXZ
Confirm "Replication Data for: Direct Democracy and Women's Political Engagement" from Dataverse succeeds

Globus/MDF

Confirm DOI resolution for http://dx.doi.org/doi:10.18126/M2301J results in https://petreldata.net/mdf/detail/pub_30_shahani_twinmediated_v1.2
Register dataset https://petreldata.net/mdf/detail/pub_30_shahani_twinmediated_v1.2

FAIL: whole-tale/girder_wholetale#481

Run Tale

Preconditions:

No running Tale instances
General

Launch (Copy and Run) the Ligo Tale and wait for the instance to start
Confirm you are redirected to the Run page
1. Confirm that the Run panel displays the Tale icon, title, and author
2. Confirm that the Run page displays the Interact, Files, Metadata tabs
Select the "..." menu.
1. Confirm that menu options exist for "Rebuild Tale", "Restart Tale", "Save Tale Version", "Duplicate Tale", "Publish Tale", "Export Tale", "Connect to Git Repository..."
2. Confirm that the "Read the docs" link displays https://wholetale.readthedocs.io/en/stable/users_guide/run.html in a new tab or window
3. Confirm that the "View Fullscren" link displays the Run panel in full screen

TEST: Test case inconsistencies with release

Interact tab

Select the "Interact" tab
Confirm Jupyter environment is displayed in iframe
Select "popout" icon. Confirm Jupyter is opened in new Tab

Files tab

Select "Files" tab
You should see a vertical navigation menu with three tabs: "Home", "External Data", and "Tale Workspace".

[ x] Metadata tab - Owned Tale

Open a Tale that you own
Navigate to the metadata page
Add multiple datasets that have DOIs to the Tale
Navigate to Run > Metadata
Valid environments should be listed in the Environment dropdown menu
Valid licenses should be listed in the License dropdown
Confirm that you see Created by <your_name> under Authors
Confirm that your Tale has the CC4 license assigned to it
Confirm that the Tale's Published Location reads This Tale has not been published
Confirm that there are citations for the external datasets
Click the "Edit" button to edit the metadata fields
Change the name of the Tale, Category, Description, Illustration, Public, License, and add multiple authors
Select Save. Confirm that you see a notification telling you that the Tale was saved.
Refresh the page. Your changes should be preserved between page refreshes, confirming that they are being properly persisted.
Confirm that the new Tale authors are displayed on the Tale card

Metadata tab - Non-Owned Tale

Open a Tale that you don't own
Navigate to the metadata page
Note that there isn't an 'Edit' button present

Home

Select Files > Home. Confirm that your home directory displays
Create a folder and upload a file to the folder
Confirm file operations:
1. Copy, rename, remove, download, Move To...

NOTE: Home folder is not present on public tale? Test case should state select owned tale since previous was public
FAIL: Creating folder with duplicate name fails with 500 (whole-tale/virtual_resources#14?)
TEST: File operations test could use some details

External data

Precondition: Register one or more Datasets (e.g. doi:10.18739/A29G5GD0V)
1. Confirm notification progress and success
Confirm "External Data" list is initially empty
Confirm (+) button ("Select Data modal") behavior
1. Confirm that "WT Data Catalog" shows all registered datasets
2. Confirm that "My External Data" filters to only show your registered datasets
3. Select a dataset
4. Confirm that clicking "Add Selected" adds the dataset to the "Selected Data" column
5. Confirm that "Remove Selected" data to dissociate it with a Tale
6. Using the dataset above, add the dataset to the Tale, by clicking "Save"
7. Confirm that the Tale's "External Data" list matches what was saved
8. Remove the dataset from the Tale
9. Confirm "External Data" list matches what was saved
10. Select a dataset but then click out of the modal or click "Cancel"
11. Confirm that the changes weren't saved

Workspaces

Confirm create folder
Confirm upload file
Select "+" button has options "New folder", "Upload file", Import Tale Data..."
Folder operations
1. Confirm context menu Copy, Rename, Download, Remove
2. Confirm rename folder
3. Confirm download folder (this should result in a zip file)
4. Confirm remove folder
File operations
1. Upload a file
2. Confirm context menu Copy, Rename, Download, Remove
3. Confirm rename folder
4. Confirm download file
5. Confirm remove file
Select "Select Data..."
1. Note vertical navigation with Tale Workspaces
2. Copy to Workspace from another workspace
3. Move to workspace from another workspace

FAIL: 500 on create duplicate folder name (whole-tale/virtual_resources#14)
NOTE: Downloaded folder contains files but not folder (expected Girder behavior, but possibly note in test case)
NOTE: Select Data workspace sorted by ? Not Tale Title
FAIL: Copy from tale workspace requires manual refresh

Files - Non-Owned Tale

Open a Tale that you don't have write permissions to
Attempt to add data to the workspace and external data
Note that the "+" icons are greyed out, preventing you from doing so

Export Tale

View a Tale that you own
Select "Export Tale" from Tale menu
Confirm that a download starts for the ZIP archive
Open the archive
Confirm bagit format and contents
Run the exported tale and confirm that it builds and runs as expected

Settings

Default State

Navigate to the Settings page
Confirm there are empty settings for DataONE, Dataverse, and Zenodo

Connect to Zenodo

Click "Connect Account" in the Zenodo section
Confirm that the connection modal opens
Confirm that sandbox.zenodo.org and zenodo.org are listed
Enter your API key for one of the repositories
Click "Connect"
Confirm that you see "Authorized on "
Select disconnect, confirmation window should appear
Repeat test with invalid key, confirm error message appears
Repeat for each Zenodo server

NOTE: API Key link does not provide documentation. Consider adding docs describing what to do to. I think a general RTD link for all repos would suffice.
FAIL: Sandox.zenodo.org repo "Get from links to zenodo.org

Connect to Dataverse

Click "Connect Account" in the Dataverse section
Confirm that the connection modal opens
Enter your API key for one of the repositories
Click "Connect"
Confirm that you see "Authorized on "
Select disconnect, confirmation window should appear
Repeat test with invalid key, confirm error message appears
Repeat for each Dataverse server

NOTE: Stage has dev2.dataverse.org listed

Connect to DataONE

Click "Connect Account" under the DataONE repository
Confirm that you're redirected to the DataONE login
Log in with ORCID
Confirm that you're redirected back to Whole Tale
Confirm that you see "Authorized" under the DataONE server
Confirm that the "Connect Account" Button is disabled
Disconnect your account
Confirm that you can re-connect it

FAIL: DataONE auth fails, results in empty key

Confirm tokens retained across logins
#370

Connect one or more accounts as outlined above
Logout and login again
Using Swagger UI, hit the /user/me endpoint to confirm tokens are still present

Tale Creation

Create RStudio Tale

Select the Create New Tale button on the 'Browse' page
Enter title "Test Tale"
Select either RStudio or RStudio 4.0.2
Select "Create New Tale"
Confirm that you're brought to the 'Run' page and that the Tale isn't building
Click 'Run Tale'
Confirm notification dialog opens
Confirm that the build/launch steps are correct
Wait for the building to complete
Navigate to the Browse page
Confirm the Tale is shown under "Currently running"

TODO: Needs rebuild test case
NOTE: Need to decide what to do about different RStudio versions

Compose Jupyter Notebook Tale

TODO: Needs test case

Compose JupyterLab Tale

TODO: Needs test case

Compose JupyterLab with Spark

TODO: Needs test case

Compose MATLAB Desktop Tale

Create tale from Git repo https://github.com/whole-tale/matlab-example/
Run multiplicative_arima_example.m, confirm outputs
Confirm CSP settings

Compose MATLAB Jupyter Kernel Tale

Create tale from Git repo https://github.com/whole-tale/matlab-example/
Run multiplicative_arima_example.ipynb, confirm outputs match multiplicative_arima_example.html

Compose STATA Desktop Tale

Create tale from Git repo https://github.com/whole-tale/stata-example/
Run example.do, confirm outputs

Compose STATA Jupyter Kernel Tale

Create tale from Git repo https://github.com/whole-tale/stata-example/
Run example.ipynb, confirm outputs match example.html

Compose OpenRefine Tale

TODO: Needs test case

Too many instances

Make sure you have two launched Tales
Attempt to create a Tale
Confirm that you receive an error that you have the max limit
Confirm that the Tale is created, but not the instance

TEST: Revise test case

Analyze in Whole Tale

Task 1: Importing a Dataset from Dataverse

These test cases cover potential situations that can occur when importing datasets from Dataverse.

Case 1: Ideal Scenario:

Testing Steps:

Remove all running Tale instances
Navigate to https://girder.stage.wholetale.org/api/v1/integration/dataverse?datasetPid=doi%3A10.7910%2FDVN%2F3MJ7IR&siteUrl=https%3A%2F%2Fdataverse.harvard.edu
Confirm that the Tale title reads as Replication Data for: "Agricultural Fires and Health at Birth"
Confirm that the only item in the Input data section matches the uri with Data Source appended
Confirm that no environment is selected
Confirm that the Create New Tale button is disabled
Select an environment
Confirm that READ/WRITE is selected
Click Create New Tale
Confirm notification progress and success
Confirm that you are redirected to the run page
Confirm that the Tale name matches the Tale Name in the Create Tale Modal
Confirm that the data exists under Tale Workspace
Confirm that the Tale's category is science

Task 2: Importing a Dataset from DataONE

These test cases cover different scenarios that may happen when importing datasets from DataONE.

Case 1: Ideal Scenario:

This test covers the ideal case where dataset importing has all of the needed information
to properly run. This includes the dataset location, title, and selected environment.

Testing Steps:

Remove all running Tale instances
Navigate to https://girder.stage.wholetale.org/api/v1/integration/dataone?uri=https%3A%2F%2Fsearch.dataone.org%2Fview%2Fdoi%3A10.18739%2FA2VQ2S94D&title=Fire%20influences%20on%20forest%20recovery%20and%20associated%20climate%20feedbacks%20in%20Siberian%20Larch%20Forests%2C%20Russia&environment=RStudio
Confirm that the Tale title reads as Fire influences on forest recovery and associated climate feedbacks in Siberian Larch Forests, Russia
Confirm that the only item in the Selected data section matches the uri with Data Source appended
Confirm that Rstudio is selected in the Environments widget
Confirm that the Create New Tale button is enabled
Confirm READ ONLY is selected
Click Create New Tale
Confirm that you are redirected to the run page
Confirm that the Tale name matches the Tale Name in the Create Tale Modal
Confirm that the data exists in the Tale under External Data
Confirm that the Tale's category is science

Case 2: Testing Different DataONE Deployments
The Analyze in Whole Tale feature is deployed across a few DataONE test sites. This test ensures Whole Tale has the ability to import
data from each of them.

Links:

Testing Steps:

Remove all running Tale instances
Navigate to each of the two links above
Select an environment
Click Create New Tale
Confirm that the correct data exists in the Tale under External Data

Task 3: Shared Behavior

These cases show errors that are common between Dataverse and DataONE. Although they are tested using DataONE URIs, the errors should be the same when replacing it with a Dataverse URI.

Case 1: Missing Dataset Title Query Param:

This test covers the case where the name query parameter is missing. The expected behavior is
that the uri query parameter is used instead.

Testing Steps:

Remove all running Tale instances
Navigate to https://dashboard.stage.wholetale.org/browse?uri=https%3A%2F%2Fdataverse.harvard.edu%2Fapi%2Faccess%2Fdatafile%2F3323458
Confirm that the Tale title reads as https://dataverse.harvard.edu/api/access/datafile/3323458
Confirm that the only item in the Selected data section matches the uri with Data Source appended
Confirm that no Environment is selected
Confirm that the Create New Tale button is disabled
Select an environment
Click Create New Tale
Confirm that the notification bar appears & properly updates
Confirm that you are redirected to the Metadata page
Confirm that the Tale name matches the Tale Name in the Create Tale Modal
Confirm that the data exists in the Tale under External Data

FAIL: Tale title is blank

Case 2: Importing Data with Read/Write:

This test covers the case where a user imports data from an external source and selects READ/WRITE in the Create Tale Modal.

Testing Steps (Variant 1):

Remove all running Tale instances
Navigate to https://dashboard.stage.wholetale.org/browse?uri=https%3A%2F%2Fdataverse.harvard.edu%2Fapi%2Faccess%2Fdatafile%2F3323458
Confirm that the Tale title matches the URI above
Confirm that the only item in the Selected data section matches the uri with Data Source appended
Confirm that the Create New Tale button is disabled
Select an environment
Select READ/WRITE
Click Create New Tale
Confirm that the notification bar appears & properly updates
Confirm that you are redirected to the run page
Confirm that the Tale name matches the Tale Name in the Create Tale Modal
Confirm that the data exists in the Tale Workspace

FAIL: Tale title is blank

Testing Steps (Variant 2):

Remove all running Tale instances
Navigate to https://dashboard.stage.wholetale.org/browse?api=https%3A%2F%2Fdev.nceas.ucsb.edu%2Fknb%2Fd1%2Fmn%2Fv2&environment=JupyterLab&name=proveit&uri=doi%3A10.5072%2FFK27P92Z55
Confirm that the Tale title matches the dataset
Confirm that the only item in the Selected data section matches the uri with Data Source appended
Confirm that the Create New Tale button is disabled
Select READ/WRITE
Click Create New Tale
Confirm that the notification bar appears & properly updates
Confirm that you are redirected to the run page
Confirm that the Tale name matches the Tale Name in the Create Tale Modal
Confirm that the data exists in the Tale Workspace

Tale metadata tests

The purpose of these tests are to confirm that the metadata files (manifest.json, environment.json, LICENSE) we generate are correct.

manifest.json

Create or view a Tale that you own
Add a dataset to the Tale
Add at least one author
Add at least one file to the workspace
Export the Tale (as BagIt)
Open the manifest.json file under metadata/
Confirm that the author is listed under schema:author
Confirm that you are listed under createdBy
Confirm that the file(s) in the workspace are properly listed in aggregates
Confirm that the external dataset is referenced in aggregates
Confirm that the dataset is also listed under Datasets

environment.json

Open the archive from the manifest.json test
Navigate to metadata/environment.json
Open the girder web API
Locate and GET the Image that the Tale used
Confirm that the config section from girder matches the config section in environment.json

LICENSE

Open the archive from the manifest.json test
Open the LICENSE file
Navigate to Run > metadata
Confirm that the opened LICENSE file matches the selected license on the metadata page
Change the Tale's license
Export the Tale
Confirm that the LICENSE changed appropriately

Tale Sharing Tests

These tests are centered around the feature for sharing Tales with other Whole Tale
users. To run these tests, register at least one other account, referred to as User B.

Case 1: Sharing with 'Can View' permissions
1. Create a new Tale
2. Add data to the workspace and add at least one dataset
3. Navigate to the 'Share' tab
4. Share the Tale with User B
5. Confirm that the default permission is 'Can View'
6. Switch the account to User B
7. Navigate to the 'Shared with Me' tab on the Browse page
8. Confirm that you see the Tale from step 1
9. Confirm that you can't edit the Workspace, External Data, or metadata
Case 2: Sharing with 'Can Edit' permissions
1. Create a new Tale
2. Add data to the workspace and add at least one dataset
3. Navigate to the 'Share' tab
4. Share the Tale with User B
5. Set the permissions to 'Can Edit'
6. Switch the account to User B
7. Navigate to the 'Shared with Me' tab on the Browse page
8. Confirm that you see the Tale from step 1
9. Remove one of the files from the Workspace
10. Add a dataset to External Data
11. Change a field in the metadata (title, description, etc)
12. Switch to User A
13. Confirm that the changes made to the Tale persisted

Publishing tests

Case 1: No Accounts Configured
1. Disconnect all of your external accounts
2. Launch a Tale that you own
3. Click the ellipsis dropdown on the Run page
4. Click Publish tale...
5. Confirm that your are notified to link accounts on the settings page
6. Click Go to Settings
7. Confirm you are re-directed to settings
[] Case 2: Publishing to a Third Party
1. Connect to a third party provider (make sure it's a dev server)
2. Open a Tale that you own
3. Add some data to it
4. Click Publish tale...
5. Confirm that the publishing modal opens
6. Confirm that the third party that you are connected to appears in the dropdown
7. Select the third party
8. Click Publish
9. Once complete, navigate to the metadata page
10. Confirm that the published location is present
11. Visit the published location
12. Confirm that you see the following files plus yours
  1. metadata.xml
  2. LICENSE
  3. README.md
  4. manifest.json
  5. environment.json
13. Repeat for each third party provider
  a. After publishing to DataONE, use girder to query the Tale
  b. Ensure that the 'pid' field in the 'publishInfo' object has an identifier in a doi format

FAIL: Can't publish to DataONE

Case 3: Re-published Tale

Launch a Tale that has been published before
Take note of the Published Location in Run > metadata
Re-publish the Tale to the third party
Confirm that the Published Location has changed to the correct package landing page

NOTE: Tested for Zenodo

Zenodo integration tests

Register Zenodo data

Select Manage > Data tab
Select "+". Confirm "Search for Data Sources" modal displays
Search for Concept DOI doi:10.5281/zenodo.16384
Confirm "Belmont Forum..." dataset is found on Zenodo with suffix ver_1
Search for URL https://zenodo.org/record/1172960#.XidIvlNKh24
Confirm "Belmont Forum..." dataset is found on Zenodo with suffix ver_2
Search for DOI doi:10.5281/zenodo.1172960
Confirm "Belmont Forum..." dataset is found on Zenodo with suffix ver_2
Search for Concept DOI doi:10.5281/zenodo.608984
Confirm "Belmont Forum..." dataset is found on Zenodo with suffix ver_2
Select Register
Confirm dataset appears in Manage Data list
Add to tale, run tale and list data dir to confirm contents match Zenodo version

The register tests the following cases.

Datasets with multiple files
Datasets with multiple versions
Concept DOI
Zenodo dataset via AiWT

Open https://dashboard.stage.wholetale.org/browse?uri=https%3A%2F%2Fdoi.org%2F10.5281%2Fzenodo.820575&name=Automotive%20Sensor%20Data
Confirm Source Data URL and Title match above URL and "READ ONLY" is selected
Select environment
Select 'Create New Tale'
Confirm dataset is mounted read-only, citation and relate is as expected, zipfile is not extracted (must run IDE)

The register tests the following cases.

Dataset with single file (zipfile)
Zenodo Binder via AiWT

Open https://dashboard.stage.wholetale.org/browse?uri=https%3A%2F%2Fdoi.org%2F10.5281%2Fzenodo.3242073&environment=JupyterLab&name=Simple%20requirements.txt%20based%20example%20for%20repo2docker&asTale=true
COnfirm Source Data and Title match URL and "READ WRITE" is selected
Select 'Create Tale'
Confirm image builds and notebook runs without error

Zenodo Tale via AiWT

Open https://girder.stage.wholetale.org/api/v1/integration/zenodo?doi=10.5072%2Fzenodo.490450&resource_server=sandbox.zenodo.org
Confirm Water Tale is created.
Confirm citation on Metadata view
Confirm DOI matches above
Use the Swagger UI to confirm the publishInfo field on the tale object contains pid, repository, repository_id, uri
Run the tale and and notebook, confirm it works as expected (produces glorious map of Texas)

FAILS: Bokeh

Publish/import to/from Zenodo Sandbox

Follow instructions above to configure your API key for Zenodo sandbox
Create your own Tale and publish to sandbox.zenodo.org
Confirm publishInfo contains Sandbox entry
Delete tale
Import tale from Zenodo
Confirm publishInfo
Modify the tale
Re-publish to sandbox.zenodo.org
Delete tale
Import tale from Zenodo
Confirm publishInfo
Download and extract the exported Tale
Execute the run-local.sh command to run the Tale locally

Versioning Tests

Tale Versioning

View a Tale that you own
Expand Tale Version panel (on the right)
Create a New Version: init
Modify metadata, upload a file, and attach a dataset to the Tale
Create another new version: modified
While viewing Run > Files > External Data, restore from init - you should see no datasets
Rename the modified Version to something else
In the same view, restore back to something else - you should see the dataset you added
Run the tale. Confirm the versions are available under ../versions and that the contents including metadata are as expected
View Info for a version
Rename a version. Confirm rename in container.
Export a version. Confirm exported version matches selected version.
Delete a version. Confirm deletion in container.

FAIL: "This Tale was updated by another user." popup on unshared tale

Copy versions

Login as a user with view-only permissions of a tale with versions.
Run the tale and confirm versions are copied

Publish/export version

Create a new tale
Upload files but do not create a version
Publish the tale
Note that there is a new version in the version panel
Confirm that the data files in the published artifact are correct
Repeat for export
Confirm manifest contents including vocabulary

Git Integration Tests

Compose Tale from Git Repository

Select the Create New Tale button on the 'Browse' page
Select the Create Tale from Git Repository option
Paste a link to a Git repository in the correct field
Add a name and pick a compute environment
Create the Tale
Confirm notification progress and success
Confirm that the git repository was cloned to the workspace/ root
Run the tale
Confirm interaction via git tools (may require environment customization)

Connect to Git

Create a new empty tale
Select "Connect to Git Repository" menu item from Tale menu
Confirm notification progress and success
Once complete, navigate to the Tale workspace and confirm that the repository has been cloned

Other tests

Citations

Add files from multiple external datasets (e.g.: doi:10.5281/zenodo.3381658, doi:10.5065/D6862DM8, doi:10.5063/F10K26VR)
Confirm Datasets Used citations and related identifiers are updated

Event Notification Tests

Tale sharing/unsharing

Login as user1 and user2 in separate browser sessions
As user2, select "Shared with me"
As user1, share a tale with user2; as user2 confirm the tale appears
As user2, select "View" on the shared tale
As user1 and 2 edit the tale metadata
As user1 save changes; as user2 confirm Tale edited modal
As user1, unshare the tale; as user2 confirm Tale unshared modal
As user1, re-share with user2
As user2, select "Shared with Me"
As user1, delete the shared tale; as user2 confirm that the tale disappears

FAIL: Fields do not update for user2 (only displayed title, not field)

Import tale

Import a tale from Github, Zenodo, and as Binder
Confirm metadata updates accordingly after import completes

Vocabulary Tests

Vocabularies

Go to https://vocabularies.stage.wholetale.org/wt/1.0/index.html
Confirm that it matches https://whole-tale.github.io/serialization-format/wt/1.0/

Forward Authentication Test

Forward authentication

Start a tale
Confirm that the tale is accessible via iframe and popout
Open an incognito window and access the tale URL
Confirm that you are prompted for authentication and can access the tale once authenticated
Login as a user who does not have permission to access the tale
Confirm that they receive an access denied error

Changing the Environment

Changing the environment

Create a new JupyterLab tale
Run the Tale, confirm JupyterLab IDE displays
Edit metadata, change environment to "RStudio"
Confirm icon changes in title bar and Environment section
Stop/run tale or rebuild/restart and confirm RStudio IDE displays

FAIL: Restart after rebuild fails with:

[2021-05-14 17:31:48,211: ERROR/ForkPoolWorker-5] Task gwvolman.tasks.update_container[f3e4857c-d2db-4905-8afd-26b342ae829c] raised unexpected: RuntimeError('Restarting the Tale failed with "update paused due to failure or early termination of task 3r2t2ngowx0u9ovshd0b5afee"')
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/celery/app/trace.py", line 412, in trace_task
    R = retval = fun(*args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/girder_worker/task.py", line 148, in __call__
    results = super(Task, self).__call__(*_t_args, **_t_kwargs)
  File "/usr/local/lib/python3.8/dist-packages/celery/app/trace.py", line 704, in __protected_call__
    return self.run(*args, **kwargs)
  File "/gwvolman/gwvolman/tasks.py", line 269, in update_container
    raise RuntimeError(
RuntimeError: Restarting the Tale failed with "update paused due to failure or early termination of task 3r2t2ngowx0u9ovshd0b5afee"

docker service ps tmp-ixzecwnwcme4 --no-trunc
m0wy6hhav6y65np77dcouwqxn    \_ tmp-ixzecwnwcme4.1   registry.stage.wholetale.org/5f71f410a4f22e103392e760/1620934770@sha256:055d72e9feeaef67084798e5cc27e6569ecaf2f10aa04778c5bf000541751f02   wt-stage-02         Shutdown            Failed 1 second ago     "starting container failed: OCI runtime create failed: container_linux.go:367: starting container process caused: exec: "jupyter": executable file not found in $PATH: unknown"

The text was updated successfully, but these errors were encountered:

craig-willis · 2021-05-13T18:48:38Z

Summary of issues found/encountered. Most of these are minor.

View tale when tale is not running should display metadata page, not interact
~~View tale when tale is running should display interact page, not metadata~~ fix: view for a running tale should display interact ngx-dashboard#179
Copy on launch (without run) should redirect to metadata page, not interact
MDF registration error Loosen mdf regex girder_wholetale#481
Creating folder with duplicate name silently fails with 500 from backend (Return validation error on POST if resource already exists virtual_resources#15)
Copying a file from another tale workspace requires manual refresh
~~Settings "Get link" links do not change based on selected repo~~ (fix: make 'Get token from url' for API keys dynamical ngx-dashboard#175)
Settings DataONE connect fails with empty key (DataONE authentication no longer works in Chrome ngx-dashboard#172)
Tale listed as running after stop (Stopped tale displays under "currently running" until refresh ngx-dashboard#148)
404 when running Jupyter/Spark test (404 when launch_container exceeds timeout gwvolman#144)
AinWT with empty title does not use URL
Sharing with use with Edit permissions sets View permissions (Adding shared user always sets view permission ngx-dashboard#170)
UI doesn't update when a new version is created (i.e., via export/publish)
"This Tale was updated by another user" popup displays on an unshared tale. ("Tale was updated" modal not behaving as expected ngx-dashboard#181)
Sync on shared tale metadata edit not working (fields are not updated for second user) Changes to shared tale metadata are not updated for shared user in edit mode ngx-dashboard#182
Changing env from Jupyter to RStudio then rebuild/restart results in error because of incorrect command update_container only works if base image hasn't changed gwvolman#143
Datasets Used links on Metadata page go to Tale Workspace Fix dataset and DOI links ngx-dashboard#183
Related identifier URLs are all relative to dashboard Fix dataset and DOI links ngx-dashboard#183

Other observations (not necessarily issues)

Settings page lists dev2.dataverse.org
Confusing to see running tales listed under Public Tales
Are we going to keep multiple RStudio environments active?
Need to cull MATLAB instances to prevent holding licenses Implement idle instance handling girder_wholetale#477
~~We should have a set of test repos for each IDE similar to matlab and stata to ease testing~~
Why can't I copy/paste text from External Data?
Since I can't sort, My Tales should be listed by last updated?
Uploaded file not displayed for shared user without refresh
Version created on export could have name indicating it was created during export (e.g., EXPORT-DATE)
Sometimes I can't get the version menu to appear, no errors in console
Staging Water Tale fails due to unpinned Bokeh -- fix or replace in test
I shouldn't be able to restart a stopped tale
Need to think about how to clarify the rebuild/restart requirement. Maybe a notification when a change is detected (i.e., you've changed your image or changed the r2d config, these changes will not be reflected until you rebuild/restart).
Author names are separated by " ," instead of ", "
One I dismiss notifications, I can't get back to the status or log of a building tale. Would be nice to be able to get that info
Current OpenRefine version is 3.4.1; Apache Spark 3.1.1; JupyterLab 3.0.15; R 4.0.5; Just to say, we might want to go out with more recent versions of IDEs? Our openrefine is using xenial base?
Nightly image builds (waiting for OpenRefine...

craig-willis closed this as completed May 21, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

v1.0rc2 testing (on OSX Catalina / Chrome 90.0.4430.93) #153

v1.0rc2 testing (on OSX Catalina / Chrome 90.0.4430.93) #153

craig-willis commented May 12, 2021 •

edited

Loading

craig-willis commented May 13, 2021 •

edited

Loading

v1.0rc2 testing (on OSX Catalina / Chrome 90.0.4430.93) #153

v1.0rc2 testing (on OSX Catalina / Chrome 90.0.4430.93) #153

Comments

craig-willis commented May 12, 2021 • edited Loading

Test Plan

Splash page

Authentication

Navigation

Tale Dashboard

Managing Data

Run Tale

Settings

Tale Creation

Analyze in Whole Tale

Task 1: Importing a Dataset from Dataverse

Task 2: Importing a Dataset from DataONE

Task 3: Shared Behavior

Tale metadata tests

Tale Sharing Tests

Publishing tests

Zenodo integration tests

Versioning Tests

Git Integration Tests

Other tests

Event Notification Tests

Vocabulary Tests

Forward Authentication Test

Changing the Environment

craig-willis commented May 13, 2021 • edited Loading

craig-willis commented May 12, 2021 •

edited

Loading

craig-willis commented May 13, 2021 •

edited

Loading