updated machine spec recommendations #23

nainathangaraj · 2018-08-21T05:06:20Z

No description provided.

…ctions for Ubuntu and Red Hat

alphabdiallo · 2018-10-01T18:03:49Z

README.md

+
+Minimal Ansible version: 2.0.
+
+This program is intended for Ubuntu 14.04 and 16.04, and has been tested on Red Hat 7.4/7.5 and OLE (Oracle Linux Enterprise) 7. It has not been tested on any other versions. 


Let's add something like: It has not been tested on any other versions but it should work with most of the Linux OS releases.

alphabdiallo · 2018-10-01T18:07:42Z

README.md

+
+More information and tutorials about the DNAnexus platform can be found at the [DNAnexus wiki page](https://wiki.dnanexus.com/Home).
+
+The remote-user that the role is run against must possess READ access to monitored_folder and WRITE access to disk for logging and temporary storage of tar files. These are typically stored under the remote-user's home directory, and is specified in the file monitor_run_config.template or as given explicitly by the variables local_tar_directory and local_log_directory.


Remote-user is a little bit confusing, let's define it and find a better name

I have updated this phrase to local user

alphabdiallo

LGTM overall, you can add those changes and ask for a last review by Samantha, thanks

…hrasing around remote user

slzarate

My computer is about to die, so I didn't get to everything. But I think you have enough to work with here. :)

slzarate · 2018-10-02T23:22:31Z

README.md

+dx-streaming-upload
+===================
+
+The dx-streaming-upload Ansible role packages the streaming upload module for increamentally uploading a RUN directory from an Illumina sequencer onto the DNAnexus platform.


--> "The dx-streaming-upload Ansible role packages the streaming upload module for incrementally uploading a RUN directory from an Illumina sequencer onto the DNAnexus platform."

slzarate · 2018-10-02T23:23:12Z

README.md

+
+The dx-streaming-upload Ansible role packages the streaming upload module for increamentally uploading a RUN directory from an Illumina sequencer onto the DNAnexus platform.
+
+Instruments that this module support include the Illumina MiSeq, NextSeq, HiSeq-2500, HiSeq-4000, HiSeq-X and NovaSeq.


--> "Instruments that this module supports include the Illumina MiSeq, NextSeq, HiSeq-2500, HiSeq-4000, HiSeq-X, and NovaSeq."

slzarate · 2018-10-02T23:23:38Z

README.md

+
+## Requirements
+
+Users of this module needs a DNAnexus account and its accompanying authentication. To register for a trial account, visit the [DNAnexus homepage](https://platform.dnanexus.com/register).


"needs" --> "need"

slzarate · 2018-10-02T23:49:26Z

README.md

+
+The local user utilizing this package should possess READ access to monitored_folder and WRITE access to disk for logging and temporary storage of tar files. These are typically stored under the local user's home directory, and is specified in the file monitor_run_config.template or as given explicitly by the variables local_tar_directory and local_log_directory.
+
+The machine that this role is deployed to should have sufficient free memory depending on the throughput of the sequencing instrument. For Novaseq and HiSeqs we recommend a machine with atleast 8 cores, 32 GB of RAM, and 500GB - 1TB of storage.


--> "The machine to which this role is deployed should have sufficient memory available depending on the throughput of the sequencing instrument. For Novaseq and HiSeqs, we recommend a machine with at least 8 cores, 32 GB of RAM, and 500GB - 1TB of storage."

slzarate · 2018-10-02T23:50:27Z

README.md

+#### Role Variables
+- `mode`: `{deploy, debug}` In the *debug* mode, monitoring cron job is triggered every minute; in *deploy mode*, monitoring cron job is triggered every hour.
+- `upload_project`: ID of the DNAnexus project that the RUN folders should be uploaded to. The ID is of the form `project-BpyQyjj0Y7V0Gbg7g52Pqf8q`
+- `dx_token`: API token for the DNAnexus user to be used for data upload. The API token should give minimally UPLOAD access to the `{{ upload project }}`, or CONTRIBUTE access if `downstream_applet` is specified. Instructions for generating a API token can be found at [DNAnexus wiki](https://wiki.dnanexus.com/UI/API-Tokens). This value is overriden by `dx_user_token` in `monitored_users`.


--> - dx_token: API token for the DNAnexus user to be used for data upload. The API token should give at a minimum UPLOAD access to the {{ upload project }}, or CONTRIBUTE access if downstream_applet is specified. Instructions for generating a API token can be found at DNAnexus wiki. This value is overridden by dx_user_token in monitored_users.

slzarate · 2018-10-03T00:18:31Z

README.md

+For an applet, the `executable_input` hash to the `run` command will be prepopulated with the key-value pair {"`upload_sentinel_record`": `$record_id`} where `$record_id` is the DNAnexus file-id of the sentinel record generated for the uploaded RUN directory (see section titled **Files generated**).
+For a workflow the `executable_input` hash will be prepoluated with the key-value pair {"`0.upload_sentinel_record`": `$record_id`} where `$record_id` is the DNAnexus file-id of the sentinel record generated for the uploaded RUN directory (see section titled **Files generated**).
+**It is the user's responsibility to ensure that the specified applet/workflow has an appropriate input contract which accepts a DNAnexus record with the input name of `upload_sentinel_record`**
+Additional input/options can be specified, statically using the Ansible variable `downstream_input`. This should be provided as a JSON string, parsable, at the top level, as a Python dict of `str` to `str`. 


"specified, statically" --> "specified statically"

slzarate · 2018-10-03T00:18:46Z

README.md

+For a workflow the `executable_input` hash will be prepoluated with the key-value pair {"`0.upload_sentinel_record`": `$record_id`} where `$record_id` is the DNAnexus file-id of the sentinel record generated for the uploaded RUN directory (see section titled **Files generated**).
+**It is the user's responsibility to ensure that the specified applet/workflow has an appropriate input contract which accepts a DNAnexus record with the input name of `upload_sentinel_record`**
+Additional input/options can be specified, statically using the Ansible variable `downstream_input`. This should be provided as a JSON string, parsable, at the top level, as a Python dict of `str` to `str`. 
+Example of a properly formatted `downstream_input` for an `applet`


Add colon to end of this and next

slzarate · 2018-10-03T00:19:13Z

README.md

+- ```{"input_name1": "value1", "input_name2": "value2"}```
+Example of a properly formatted `downstream_input` for a `workflow`
+- ```{"0.step0_input": "value1", "1.step2_input": "value2"})```
+*Note the numerical index prefix necessary when specifying input for an `workflow`, which disambiguates which step in the workflow an input is targeted to*


--> "Note that the numerical index prefix necessary when specifying input for a workflow, which disambiguates to which step in the workflow an input is targeted. "

slzarate · 2018-10-03T00:19:36Z

README.md

+(specified in monitor_run_config.template file)
+- no persistent files (tar files stored transiently, deleted upon successful upload to DNAnexus)
+```
+**Files Streamed to DNAnexus project**


--> either make "streamed" lowercase or "project" title case

slzarate · 2018-10-03T00:19:57Z

README.md

+- ```{"0.step0_input": "value1", "1.step2_input": "value2"})```
+*Note the numerical index prefix necessary when specifying input for an `workflow`, which disambiguates which step in the workflow an input is targeted to*
+#### Files generated
+We use a hypothetical example of a local RUN folder named `20160101_M000001_0001_000000000-ABCDE`, that was placed into the `monitored_directory`, after the `dx-streaming-upload` role has been set up.


--> " We use a hypothetical example of a local RUN folder named 20160101_M000001_0001_000000000-ABCDE, which was placed into the monitored_directory after the dx-streaming-upload role has been set up. "

nainathangaraj added 9 commits August 15, 2018 17:27

updated machine spec recommendations

db7531c

modified the structure of the ReadMe, and updated installation instru…

42d3f64

…ctions for Ubuntu and Red Hat

better formatting of code blocks

1655ab5

better formatting of code blocks part 2

766699b

included a link for auth token

104e2e2

cleaned up some steps for the installation

ecbf67e

added troubleshooting steps

7a8fa2c

better formatting

fbcd446

better formatting part 2

cef8296

nainathangaraj requested a review from alphabdiallo September 26, 2018 03:07

alphabdiallo reviewed Oct 1, 2018

View reviewed changes

incorporating Alpha's feedback on linux distributions supported and p…

de9ae29

…hrasing around remote user

nainathangaraj requested a review from slzarate October 2, 2018 22:30

slzarate suggested changes Oct 3, 2018

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

updated machine spec recommendations #23

updated machine spec recommendations #23

nainathangaraj commented Aug 21, 2018

alphabdiallo Oct 1, 2018

nainathangaraj Oct 2, 2018

alphabdiallo Oct 1, 2018

nainathangaraj Oct 2, 2018

alphabdiallo left a comment

slzarate left a comment

slzarate Oct 2, 2018

slzarate Oct 2, 2018

slzarate Oct 2, 2018

slzarate Oct 2, 2018

slzarate Oct 2, 2018

slzarate Oct 3, 2018

slzarate Oct 3, 2018

slzarate Oct 3, 2018

slzarate Oct 3, 2018

slzarate Oct 3, 2018


		Minimal Ansible version: 2.0.

		This program is intended for Ubuntu 14.04 and 16.04, and has been tested on Red Hat 7.4/7.5 and OLE (Oracle Linux Enterprise) 7. It has not been tested on any other versions.


		More information and tutorials about the DNAnexus platform can be found at the [DNAnexus wiki page](https://wiki.dnanexus.com/Home).

		The remote-user that the role is run against must possess READ access to monitored_folder and WRITE access to disk for logging and temporary storage of tar files. These are typically stored under the remote-user's home directory, and is specified in the file monitor_run_config.template or as given explicitly by the variables local_tar_directory and local_log_directory.


		The dx-streaming-upload Ansible role packages the streaming upload module for increamentally uploading a RUN directory from an Illumina sequencer onto the DNAnexus platform.

		Instruments that this module support include the Illumina MiSeq, NextSeq, HiSeq-2500, HiSeq-4000, HiSeq-X and NovaSeq.


		## Requirements

		Users of this module needs a DNAnexus account and its accompanying authentication. To register for a trial account, visit the [DNAnexus homepage](https://platform.dnanexus.com/register).


		The local user utilizing this package should possess READ access to monitored_folder and WRITE access to disk for logging and temporary storage of tar files. These are typically stored under the local user's home directory, and is specified in the file monitor_run_config.template or as given explicitly by the variables local_tar_directory and local_log_directory.

		The machine that this role is deployed to should have sufficient free memory depending on the throughput of the sequencing instrument. For Novaseq and HiSeqs we recommend a machine with atleast 8 cores, 32 GB of RAM, and 500GB - 1TB of storage.

updated machine spec recommendations #23

Are you sure you want to change the base?

updated machine spec recommendations #23

Conversation

nainathangaraj commented Aug 21, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

alphabdiallo left a comment

Choose a reason for hiding this comment

slzarate left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment