December 25, 2018, Tuesday

• Improvements
  • For job API, can we add job name, also to the interface?
    • Conclusion is that we will only add 'job id' search to simplify locating the failed job
      • We can find the failed job from the History column (e.g. missing outputs)
      • We can get 'job id' from the 'i' icon
      • We can search with the 'job id' to locate the failed job and delete it
        • Make sure to relaunch another one before deletion
        • Think about deleting the failed job directory before deleting the job to save storage
    • For the interface, replace email with user provided job name
    • When not provided, use first input (without extensions, trim .XX.YY) + App name
    • If no input, use App name
    • For the diagram and History, still use App name only?
    • Change "App name" to "Job name" in the 'My jobs' table
    • Add job name to API
  • Push API to the master branch
    • For processing large scale of data locally
    • For analyses that not necessary to be released
  • Redesign 'load workflow' to allow load more than one workflow
    • Use case 1: DE analysis
    • Use case 2: QC of multiple workflows
    • Shall we rename 'load' as 'relaunch' in 'My workflows'?
      • 'Visualization' actually can be used to load multiple workflows one by one to the History column
  • For MaizeCODE RNAseq, if one replicate QC failed
    • We will need to automate the following QC
    • Construct a new workflow after rerunning the failed step
    • Save the workflow
    • Replace the "SciApps id" of the read1 of rep1
    • Possible improvements
      • Improve MCrna to quit early when an error occurs
      • Turn off verbose outputs of all tools if possible otherwise hard to debug from the error file
      • Automate deletion of failed job through a combination of calling API and icommands
        • Retrieve the job archive folder through Agave
        • Delete the folder with icommands
        • Delete the workflow and job from SciApps throgh SciApps API
  • Add data page for MaizeCode, NAM, etc
    • For MaizeCode, just list the workflows owned(or created) by the maizecode user?
    • We can add a piechart for statistics of the datasets and load as default?
    • Change 'Visualization' to 'Load' and 'Load' to 'Relaunch' (Apply to 'My workflows' as well)
      • Also remove the code to display diagram (Apply to 'My workflows as well)
      • So we can search and load several workflows to the History panel for DE!
      • Also add a 'View Diagram' button to the top of the workflow forms?
    • Think about adding more metadata (e.g. full tissue name, SRA, etc) to the workflow description in the automation script
      • How about more complicated search?
    • For processing large scale data, we may need to customize a jetstream image