notebooks/Notebook02-1.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Trying out Unicycler on our charged instance"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This notebook is created on the AWS instance that we created in Notebook02; all the other notebooks were created from my personal computer."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To test AWS out, maybe I should try out doing bacterial genome assemblies. I have installed Anaconda. Then I created an environment where I installed Unicycler from bioconda channel. With Unicycler, we can choose to assemble a bacterial genome by either using short reads only, long reads only, or a combination of both, also known as hybrid assembly.\n",
    "\n",
    "Now, we need data - traditionally people would start with the genome of <em>E. coli</em> K12 MG1655, so we'll do that. Download Illumina files and Nanopore files."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "--2019-04-21 10:52:17--  http://gigadb.org/dataset/view/id/100102/token/S30Hp9ZurcARyhov\n",
      "Resolving gigadb.org (gigadb.org)... 218.189.16.137\n",
      "Connecting to gigadb.org (gigadb.org)|218.189.16.137|:80... connected.\n",
      "HTTP request sent, awaiting response... 200 OK\n",
      "Length: unspecified [text/html]\n",
      "Saving to: 'S30Hp9ZurcARyhov'\n",
      "\n",
      "S30Hp9ZurcARyhov        [ <=>                ]  48.10K  --.-KB/s    in 0.08s   \n",
      "\n",
      "2019-04-21 10:52:20 (602 KB/s) - 'S30Hp9ZurcARyhov' saved [49256]\n",
      "\n",
      "--2019-04-21 10:52:20--  http://spades.bioinf.spbau.ru/spades_test_datasets/ecoli_mc/s_6_1.fastq.gz\n",
      "Resolving spades.bioinf.spbau.ru (spades.bioinf.spbau.ru)... 194.85.238.21\n",
      "Connecting to spades.bioinf.spbau.ru (spades.bioinf.spbau.ru)|194.85.238.21|:80... connected.\n",
      "HTTP request sent, awaiting response... 200 OK\n",
      "Length: 1210172526 (1.1G) [application/x-gzip]\n",
      "Saving to: 's_6_1.fastq.gz'\n",
      "\n",
      "s_6_1.fastq.gz      100%[===================>]   1.13G  1.84MB/s    in 5m 48s  \n",
      "\n",
      "2019-04-21 10:58:09 (3.31 MB/s) - 's_6_1.fastq.gz' saved [1210172526/1210172526]\n",
      "\n",
      "--2019-04-21 10:58:09--  http://spades.bioinf.spbau.ru/spades_test_datasets/ecoli_mc/s_6_2.fastq.gz\n",
      "Resolving spades.bioinf.spbau.ru (spades.bioinf.spbau.ru)... 194.85.238.21\n",
      "Connecting to spades.bioinf.spbau.ru (spades.bioinf.spbau.ru)|194.85.238.21|:80... connected.\n",
      "HTTP request sent, awaiting response... 200 OK\n",
      "Length: 1246700196 (1.2G) [application/x-gzip]\n",
      "Saving to: 's_6_2.fastq.gz'\n",
      "\n",
      "s_6_2.fastq.gz      100%[===================>]   1.16G  5.47MB/s    in 5m 25s  \n",
      "\n",
      "2019-04-21 11:03:35 (3.65 MB/s) - 's_6_2.fastq.gz' saved [1246700196/1246700196]\n",
      "\n"
     ]
    }
   ],
   "source": [
    "wget http://s3.climb.ac.uk/nanopore/ecoli_allreads.fasta\n",
    "wget http://spades.bioinf.spbau.ru/spades_test_datasets/ecoli_mc/s_6_1.fastq.gz\n",
    "wget http://spades.bioinf.spbau.ru/spades_test_datasets/ecoli_mc/s_6_2.fastq.gz"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Check out unicycler."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "usage: unicycler [-h] [--help_all] [--version] [-1 SHORT1] [-2 SHORT2]\n",
      "                 [-s UNPAIRED] [-l LONG] -o OUT [--verbosity VERBOSITY]\n",
      "                 [--min_fasta_length MIN_FASTA_LENGTH] [--keep KEEP]\n",
      "                 [-t THREADS] [--mode {conservative,normal,bold}]\n",
      "                 [--linear_seqs LINEAR_SEQS] [--vcf]\n",
      "\n",
      "\u001b[31m\u001b[1m       __\n",
      "\u001b[0m\u001b[31m\u001b[1m       \\ \\___\n",
      "\u001b[0m\u001b[31m\u001b[1m        \\ ___\\\n",
      "\u001b[0m\u001b[31m\u001b[1m        //\n",
      "\u001b[0m\u001b[31m\u001b[1m   ____//      \u001b[0m\u001b[93m\u001b[1m_    _         _                     _\n",
      "\u001b[0m\u001b[31m\u001b[1m //_  //\\\\    \u001b[0m\u001b[93m\u001b[1m| |  | |       |_|                   | |\n",
      "\u001b[0m\u001b[31m\u001b[1m//  \\//  \\\\   \u001b[0m\u001b[93m\u001b[1m| |  | | _ __   _   ___  _   _   ___ | |  ___  _ __\n",
      "\u001b[0m\u001b[31m\u001b[1m||  (O)  ||   \u001b[0m\u001b[93m\u001b[1m| |  | || '_ \\ | | / __|| | | | / __|| | / _ \\| '__|\n",
      "\u001b[0m\u001b[31m\u001b[1m\\\\    \\_ //   \u001b[0m\u001b[93m\u001b[1m| |__| || | | || || (__ | |_| || (__ | ||  __/| |\n",
      "\u001b[0m\u001b[31m\u001b[1m \\\\_____//     \u001b[0m\u001b[93m\u001b[1m\\____/ |_| |_||_| \\___| \\__, | \\___||_| \\___||_|\n",
      "\u001b[0m\u001b[93m\u001b[1m                                        __/ |\n",
      "\u001b[0m\u001b[93m\u001b[1m                                       |___/\u001b[0m\n",
      "\n",
      "\u001b[1mUnicycler: an assembly pipeline for bacterial genomes\u001b[0m\n",
      "\n",
      "\u001b[1mHelp\u001b[0m:\n",
      "  -h, --help              \u001b[2mShow this help message and exit\u001b[0m\n",
      "  --help_all              \u001b[2mShow a help message with all program options\u001b[0m\n",
      "  --version               \u001b[2mShow Unicycler's version number\u001b[0m\n",
      "\n",
      "\u001b[1mInput\u001b[0m:\n",
      "  -1 SHORT1, --short1 SHORT1\n",
      "                          \u001b[2mFASTQ file of first short reads in each pair\u001b[0m\n",
      "                          \u001b[2m(required)\u001b[0m\n",
      "  -2 SHORT2, --short2 SHORT2\n",
      "                          \u001b[2mFASTQ file of second short reads in each pair\u001b[0m\n",
      "                          \u001b[2m(required)\u001b[0m\n",
      "  -s UNPAIRED, --unpaired UNPAIRED\n",
      "                          \u001b[2mFASTQ file of unpaired short reads (optional)\u001b[0m\n",
      "  -l LONG, --long LONG    \u001b[2mFASTQ or FASTA file of long reads (optional)\u001b[0m\n",
      "\n",
      "\u001b[1mOutput\u001b[0m:\n",
      "  -o OUT, --out OUT       \u001b[2mOutput directory (required)\u001b[0m\n",
      "  --verbosity VERBOSITY   \u001b[2mLevel of stdout and log file information (default: 1)\u001b[0m\n",
      "                          \u001b[2m  0 = no stdout, 1 = basic progress indicators,\u001b[0m\n",
      "                          \u001b[2m  2 = extra info, 3 = debugging info\u001b[0m\n",
      "  --min_fasta_length MIN_FASTA_LENGTH\n",
      "                          \u001b[2mExclude contigs from the FASTA file which are\u001b[0m\n",
      "                          \u001b[2mshorter than this length (default: 100)\u001b[0m\n",
      "  --keep KEEP             \u001b[2mLevel of file retention (default: 1)\u001b[0m\n",
      "                          \u001b[2m  0 = only keep final files: assembly (FASTA,\u001b[0m\n",
      "                          \u001b[2m  GFA and log),\u001b[0m\n",
      "                          \u001b[2m  1 = also save graphs at main checkpoints,\u001b[0m\n",
      "                          \u001b[2m  2 = also keep SAM (enables fast rerun in different mode),\u001b[0m\n",
      "                          \u001b[2m  3 = keep all temp files and save all graphs (for debugging)\u001b[0m\n",
      "  --vcf                   \u001b[2mProduce a VCF by mapping the short reads to the\u001b[0m\n",
      "                          \u001b[2mfinal assembly (experimental, default: do not\u001b[0m\n",
      "                          \u001b[2mproduce a vcf file)\u001b[0m\n",
      "\n",
      "\u001b[1mOther\u001b[0m:\n",
      "  -t THREADS, --threads THREADS\n",
      "                          \u001b[2mNumber of threads used (default: 8)\u001b[0m\n",
      "  --mode {conservative,normal,bold}\n",
      "                          \u001b[2mBridging mode (default: normal)\u001b[0m\n",
      "                          \u001b[2m  conservative = smaller contigs, lowest misassembly\u001b[0m\n",
      "                          \u001b[2m                 rate\u001b[0m\n",
      "                          \u001b[2m  normal = moderate contig size and misassembly rate\u001b[0m\n",
      "                          \u001b[2m  bold = longest contigs, higher misassembly rate\u001b[0m\n",
      "  --linear_seqs LINEAR_SEQS\n",
      "                          \u001b[2mThe expected number of linear (i.e. non-circular)\u001b[0m\n",
      "                          \u001b[2msequences in the underlying sequence (default: 0)\u001b[0m\n"
     ]
    },
    {
     "ename": "",
     "evalue": "1",
     "execution_count": 5,
     "output_type": "error",
     "traceback": []
    }
   ],
   "source": [
    "unicycler"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Run unicycler - illumina only."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Run Unicycler with Illumina only"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\u001b[93m\u001b[1m\u001b[4mStarting Unicycler\u001b[0m \u001b[2m(2019-04-21 11:28:28)\u001b[0m\n",
      "\u001b[2m    Welcome to Unicycler, an assembly pipeline for bacterial genomes. Since you\u001b[0m\n",
      "\u001b[2mprovided only short reads, Unicycler will essentially function as a SPAdes-\u001b[0m\n",
      "\u001b[2moptimiser. It will try many k-mer sizes, choose the best based on contig length\u001b[0m\n",
      "\u001b[2mand graph connectivity, and scaffold the graph using SPAdes repeat resolution.\u001b[0m\n",
      "\u001b[2m    For more information, please see https://github.com/rrwick/Unicycler\u001b[0m\n",
      "\n",
      "Command: \u001b[1m/home/ubuntu/.anaconda3/envs/unic/bin/unicycler -1 s_6_1.fastq.gz -2 s_6_2.fastq.gz -o illumina_only -t 14\u001b[0m\n",
      "\n",
      "Unicycler version: v0.4.7\n",
      "Using 14 threads\n",
      "\n",
      "Making output directory:\n",
      "  /home/ubuntu/EColi/illumina_only\n",
      "\n",
      "Dependencies:\n",
      "  \u001b[4mProgram         Version             Status  \u001b[0m\n",
      "  spades.py       3.13.0              \u001b[32mgood\u001b[0m    \n",
      "  \u001b[2mracon                               not used\u001b[0m\n",
      "  makeblastdb     2.5.0+              \u001b[32mgood\u001b[0m    \n",
      "  tblastn         2.5.0+              \u001b[32mgood\u001b[0m    \n",
      "  bowtie2-build   2.3.4.3             \u001b[32mgood\u001b[0m    \n",
      "  bowtie2         2.3.4.3             \u001b[32mgood\u001b[0m    \n",
      "  samtools        1.9                 \u001b[32mgood\u001b[0m    \n",
      "  java            1.8.0_152-release   \u001b[32mgood\u001b[0m    \n",
      "  pilon           1.23                \u001b[32mgood\u001b[0m    \n",
      "  \u001b[2mbcftools                            not used\u001b[0m\n",
      "\n",
      "\n",
      "\u001b[93m\u001b[1m\u001b[4mSPAdes read error correction\u001b[0m \u001b[2m(2019-04-21 11:31:49)\u001b[0m\n",
      "\u001b[2m    Unicycler uses the SPAdes read error correction module to reduce the number\u001b[0m\n",
      "\u001b[2mof errors in the short read before SPAdes assembly. This can make the assembly\u001b[0m\n",
      "\u001b[2mfaster and simplify the assembly graph structure.\u001b[0m\n",
      "\n",
      "Command: \u001b[1m/home/ubuntu/.anaconda3/envs/unic/bin/spades.py -1 /home/ubuntu/EColi/s_6_1.fastq.gz -2 /home/ubuntu/EColi/s_6_2.fastq.gz -o /home/ubuntu/EColi/illumina_only/spades_assembly/read_correction --threads 14 --only-error-correction\u001b[0m\n",
      "\n",
      "Corrected reads:\n",
      "  /home/ubuntu/EColi/illumina_only/spades_assembly/corrected_1.fastq.gz\n",
      "  /home/ubuntu/EColi/illumina_only/spades_assembly/corrected_2.fastq.gz\n",
      "\n",
      "\n",
      "\u001b[93m\u001b[1m\u001b[4mChoosing k-mer range for assembly\u001b[0m \u001b[2m(2019-04-21 11:56:55)\u001b[0m\n",
      "\u001b[2m    Unicycler chooses a k-mer range for SPAdes based on the length of the input\u001b[0m\n",
      "\u001b[2mreads. It uses a wide range of many k-mer sizes to maximise the chance of\u001b[0m\n",
      "\u001b[2mfinding an ideal assembly.\u001b[0m\n",
      "\n",
      "SPAdes maximum k-mer: 127\n",
      "Median read length: 100\n",
      "K-mer range: 21, 35, 47, 57, 67, 73, 81, 85, 91, 95\n",
      "\n",
      "\n",
      "\u001b[93m\u001b[1m\u001b[4mSPAdes assemblies\u001b[0m \u001b[2m(2019-04-21 12:00:17)\u001b[0m\n",
      "\u001b[2m    Unicycler now uses SPAdes to assemble the short reads. It scores the\u001b[0m\n",
      "\u001b[2massembly graph for each k-mer using the number of contigs (fewer is better) and\u001b[0m\n",
      "\u001b[2mthe number of dead ends (fewer is better). The score function is 1/(c*(d+2)),\u001b[0m\n",
      "\u001b[2mwhere c is the contig count and d is the dead end count.\u001b[0m\n",
      "\n",
      "\u001b[4mK-mer   Contigs   Dead ends   Score      \u001b[0m\n",
      "   21                         too complex\n",
      "   35     1,007           5      1.42e-04\n",
      "   47       470           3      4.26e-04\n",
      "   57       397           3      5.04e-04\n",
      "   67       361           4      4.62e-04\n",
      "   73       340           4      4.90e-04\n",
      "   81       310           4      5.38e-04\n",
      "\u001b[32m   85       286           4      5.83e-04 <-best\u001b[0m\n",
      "   91       238           9      3.82e-04\n",
      "   95       215          15      2.74e-04\n",
      "\n",
      "Deleting /home/ubuntu/EColi/illumina_only/spades_assembly/\n",
      "\n",
      "\n",
      "\u001b[93m\u001b[1m\u001b[4mDetermining graph multiplicity\u001b[0m \u001b[2m(2019-04-21 12:21:26)\u001b[0m\n",
      "\u001b[2m    Multiplicity is the number of times a sequence occurs in the underlying\u001b[0m\n",
      "\u001b[2msequence. Single-copy contigs (those with a multiplicity of one, occurring only\u001b[0m\n",
      "\u001b[2monce in the underlying sequence) are particularly useful.\u001b[0m\n",
      "\n",
      "Saving /home/ubuntu/EColi/illumina_only/001_best_spades_graph.gfa\n",
      "\n",
      "\n",
      "\u001b[93m\u001b[1m\u001b[4mCleaning graph\u001b[0m \u001b[2m(2019-04-21 12:21:26)\u001b[0m\n",
      "\u001b[2m    Unicycler now performs various cleaning procedures on the graph to remove\u001b[0m\n",
      "\u001b[2moverlaps and simplify the graph structure. The end result is a graph ready for\u001b[0m\n",
      "\u001b[2mbridging.\u001b[0m\n",
      "\n",
      "Graph overlaps removed\n",
      "\n",
      "Removed zero-length segments:\n",
      "    179, 180, 181, 182, 188, 194, 196, 199, 200, 209, 210, 224, 227, 230, 231,\n",
      "232, 233, 242, 259, 274\n",
      "\n",
      "Removed zero-length segments:\n",
      "    185, 190, 257\n",
      "\n",
      "Merged small segments:\n",
      "    238, 239, 240, 245, 246, 248, 249, 251, 252, 253, 254, 256, 260, 261, 262,\n",
      "263, 264, 265, 266, 267, 269, 270, 271, 272, 273, 275, 276, 277, 278, 279, 280,\n",
      "281\n",
      "\n",
      "Saving /home/ubuntu/EColi/illumina_only/002_overlaps_removed.gfa\n",
      "\n",
      "\u001b[2m    Unicycler now selects a set of anchor contigs from the single-copy contigs.\u001b[0m\n",
      "\u001b[2mThese are the contigs which will be connected via bridges to form the final\u001b[0m\n",
      "\u001b[2massembly.\u001b[0m\n",
      "\n",
      "82 anchor segments (4,507,278 bp) out of 232 total segments (4,554,121 bp)\n",
      "\n",
      "\n",
      "\u001b[93m\u001b[1m\u001b[4mCreating SPAdes contig bridges\u001b[0m \u001b[2m(2019-04-21 12:21:26)\u001b[0m\n",
      "\u001b[2m    SPAdes uses paired-end information to perform repeat resolution (RR) and\u001b[0m\n",
      "\u001b[2mproduce contigs from the assembly graph. SPAdes saves the graph paths\u001b[0m\n",
      "\u001b[2mcorresponding to these contigs in the contigs.paths file. When one of these\u001b[0m\n",
      "\u001b[2mpaths contains two or more anchor contigs, Unicycler can create a bridge from\u001b[0m\n",
      "\u001b[2mthe path.\u001b[0m\n",
      "\n",
      "                                        Bridge\u001b[0m\n",
      "\u001b[4mStart           Path           End     quality\u001b[0m\n",
      "  -73           142            76         40.9\n",
      "  -33           -170           43         63.0\n",
      "  -30           -136           50         62.4\n",
      "  -16           -170           34         58.2\n",
      "  -11                          67         63.2\n",
      "   21           201            41         63.1\n",
      "   23        214 -> 179        62         61.1\n",
      "   41           164            55         63.2\n",
      "   42           -164           58         62.7\n",
      "   48   -131 -> -228 -> -131   19          7.8\n",
      "   54           183            14         62.2\n",
      "   57   -146 -> 210 -> -146    56         14.3\n",
      "   62    -179 -> -85 -> 179    -82         4.8\n",
      "   66       -183 -> -201       60         60.0\n",
      "   73           144            52         61.2\n",
      "   77       -179 -> -214       82         57.8\n",
      "\n",
      "\n",
      "\u001b[93m\u001b[1m\u001b[4mCreating loop unrolling bridges\u001b[0m \u001b[2m(2019-04-21 12:21:26)\u001b[0m\n",
      "\u001b[2m    When a SPAdes contig path connects an anchor contig with the middle contig\u001b[0m\n",
      "\u001b[2mof a simple loop, Unicycler concludes that the sequences are contiguous (i.e.\u001b[0m\n",
      "\u001b[2mthe loop is not a separate piece of DNA). It then uses the read depth of the\u001b[0m\n",
      "\u001b[2mmiddle and repeat contigs to guess the number of times to traverse the loop and\u001b[0m\n",
      "\u001b[2mmakes a bridge.\u001b[0m\n",
      "\n",
      "                                  Loop count   Loop count    Loop    Bridge\u001b[0m\n",
      "\u001b[4mStart   Repeat   Middle     End    by repeat    by middle   count   quality\u001b[0m\n",
      "  -19      131      228     -48         0.34         0.60       1      23.9\n",
      "  -35      165      211       9         0.18         0.52       1      20.8\n",
      "   57     -146      210      56         0.48         0.96       1      32.7\n",
      "  -42     -169      217      48         0.00         0.56       1       7.7\n",
      "   59      130      124       6         0.68         1.59       1      35.7\n",
      "\n",
      "\n",
      "\u001b[93m\u001b[1m\u001b[4mApplying bridges\u001b[0m \u001b[2m(2019-04-21 12:21:26)\u001b[0m\n",
      "\u001b[2m    Unicycler now applies to the graph in decreasing order of quality. This\u001b[0m\n",
      "\u001b[2mensures that when multiple, contradictory bridges exist, the most supported\u001b[0m\n",
      "\u001b[2moption is used.\u001b[0m\n",
      "\n",
      "\u001b[4mBridge type   Start -> end   Path              Quality\u001b[0m\n",
      "SPAdes          41 -> 55     164                63.242\n",
      "SPAdes         -11 -> 67                        63.177\n",
      "SPAdes          21 -> 41     201                63.108\n",
      "SPAdes         -33 -> 43     -170               63.013\n",
      "SPAdes          42 -> 58     -164               62.694\n",
      "SPAdes         -30 -> 50     -136               62.377\n",
      "SPAdes          54 -> 14     183                62.222\n",
      "SPAdes          73 -> 52     144                61.233\n",
      "SPAdes          23 -> 62     214, 179           61.058\n",
      "SPAdes          66 -> 60     -183, -201         60.001\n",
      "SPAdes         -16 -> 34     -170               58.233\n",
      "SPAdes          77 -> 82     -179, -214         57.827\n",
      "SPAdes         -73 -> 76     142                40.880\n",
      "SPAdes          57 -> 56     -146, 210, -146    14.286\n",
      "loop            59 -> 6      130, 124, 130      35.749\n",
      "loop           -19 -> -48    131, 228, 131      23.886\n",
      "loop           -35 -> 9      165, 211, 165      20.824\n",
      "\n",
      "Saving /home/ubuntu/EColi/illumina_only/003_bridges_applied.gfa\n",
      "\n",
      "\n",
      "\u001b[93m\u001b[1m\u001b[4mBridged assembly graph\u001b[0m \u001b[2m(2019-04-21 12:21:27)\u001b[0m\n",
      "\u001b[2m    The assembly is now mostly finished and no more structural changes will be\u001b[0m\n",
      "\u001b[2mmade. Ideally the assembly graph should now have one contig per replicon and no\u001b[0m\n",
      "\u001b[2merroneous contigs (i.e a complete assembly). If there are more contigs, then\u001b[0m\n",
      "\u001b[2mthe assembly is not complete.\u001b[0m\n",
      "\n",
      "Saving /home/ubuntu/EColi/illumina_only/004_final_clean.gfa\n",
      "\n",
      "\u001b[4mComponent   Segments   Links   Length      N50       Longest segment   Status    \u001b[0m\n",
      "        1        197     280   4,554,227   125,238           223,820  \u001b[31m incomplete\u001b[0m\n",
      "\n",
      "\n",
      "\u001b[93m\u001b[1m\u001b[4mPolishing assembly with Pilon\u001b[0m \u001b[2m(2019-04-21 12:21:27)\u001b[0m\n",
      "\u001b[2m    Unicycler now conducts multiple rounds of Pilon in an attempt to repair any\u001b[0m\n",
      "\u001b[2mremaining small-scale errors with the assembly.\u001b[0m\n",
      "\n",
      "Aligning reads to find appropriate insert size range...\n",
      "Insert size 1st percentile:  193\n",
      "Insert size 99th percentile: 238\n",
      "\n",
      "\u001b[4mPilon polish round 1\u001b[0m\n",
      "Total number of changes: 82\n",
      "\n",
      "\u001b[4mPilon polish round 2\u001b[0m\n",
      "Total number of changes: 31\n",
      "\n",
      "\u001b[4mPilon polish round 3\u001b[0m\n",
      "Total number of changes: 15\n",
      "\n",
      "\u001b[4mPilon polish round 4\u001b[0m\n",
      "Total number of changes: 13\n",
      "\n",
      "\u001b[4mPilon polish round 5\u001b[0m\n",
      "Total number of changes: 2\n",
      "\n",
      "\u001b[4mPilon polish round 6\u001b[0m\n",
      "Total number of changes: 1\n",
      "\n",
      "\u001b[4mPilon polish round 7\u001b[0m\n",
      "No Pilon changes\n",
      "\n",
      "Saving /home/ubuntu/EColi/illumina_only/005_polished.gfa\n",
      "\n",
      "\n",
      "\u001b[93m\u001b[1m\u001b[4mAssembly complete\u001b[0m \u001b[2m(2019-04-21 14:08:10)\u001b[0m\n",
      "Saving /home/ubuntu/EColi/illumina_only/assembly.gfa\n",
      "Saving /home/ubuntu/EColi/illumina_only/assembly.fasta\n",
      "\n"
     ]
    }
   ],
   "source": [
    "unicycler \\\n",
    " -1 s_6_1.fastq.gz \\\n",
    " -2 s_6_2.fastq.gz \\\n",
    " -o illumina_only \\\n",
    " -t 14"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Nice! That took probably around 2 hours and 30 minutes. Let's try doing with just long reads."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\u001b[93m\u001b[1m\u001b[4mStarting Unicycler\u001b[0m \u001b[2m(2019-04-21 16:21:39)\u001b[0m\n",
      "\u001b[2m    Welcome to Unicycler, an assembly pipeline for bacterial genomes. Since you\u001b[0m\n",
      "\u001b[2mprovided only long reads, Unicycler will assemble the reads with miniasm and\u001b[0m\n",
      "\u001b[2mthen run repeated polishing rounds using Racon.\u001b[0m\n",
      "\u001b[2m    For more information, please see https://github.com/rrwick/Unicycler\u001b[0m\n",
      "\n",
      "Command: \u001b[1m/home/ubuntu/.anaconda3/envs/unic/bin/unicycler -l ecoli_allreads.fasta -o nanopore_only\u001b[0m\n",
      "\n",
      "Unicycler version: v0.4.7\n",
      "Using 8 threads\n",
      "\n",
      "Making output directory:\n",
      "  /home/ubuntu/EColi/nanopore_only\n",
      "\n",
      "Dependencies:\n",
      "  \u001b[4mProgram         Version   Status  \u001b[0m\n",
      "  \u001b[2mspades.py                 not used\u001b[0m\n",
      "  racon           1.3.2     \u001b[32mgood\u001b[0m    \n",
      "  makeblastdb     2.5.0+    \u001b[32mgood\u001b[0m    \n",
      "  tblastn         2.5.0+    \u001b[32mgood\u001b[0m    \n",
      "  \u001b[2mbowtie2-build             not used\u001b[0m\n",
      "  \u001b[2mbowtie2                   not used\u001b[0m\n",
      "  \u001b[2msamtools                  not used\u001b[0m\n",
      "  \u001b[2mjava                      not used\u001b[0m\n",
      "  \u001b[2mpilon                     not used\u001b[0m\n",
      "  \u001b[2mbcftools                  not used\u001b[0m\n",
      "\n",
      "\n",
      "\u001b[93m\u001b[1m\u001b[4mLoading reads\u001b[0m \u001b[2m(2019-04-21 16:21:39)\u001b[0m\n",
      "150,604 / 150,604 (100.0%) - 5,014,576,373 bp\n",
      "\n",
      "\n",
      "\u001b[93m\u001b[1m\u001b[4mAssembling contigs and long reads with miniasm\u001b[0m \u001b[2m(2019-04-21 16:21:55)\u001b[0m\n",
      "Saving to /home/ubuntu/EColi/nanopore_only/miniasm_assembly/01_assembly_reads.fastq:\n",
      "  150,604 long reads\n",
      "\n",
      "Finding overlaps with minimap... Traceback (most recent call last):\n",
      "  File \"/home/ubuntu/.anaconda3/envs/unic/bin/unicycler\", line 12, in <module>\n",
      "    sys.exit(main())\n",
      "  File \"/home/ubuntu/.anaconda3/envs/unic/lib/python3.6/site-packages/unicycler/unicycler.py\", line 149, in main\n",
      "    anchor_segments, args.existing_long_read_assembly)\n",
      "  File \"/home/ubuntu/.anaconda3/envs/unic/lib/python3.6/site-packages/unicycler/miniasm_assembly.py\", line 118, in make_miniasm_string_graph\n",
      "    mappings.write(minimap_alignment_str)\n",
      "OSError: [Errno 28] No space left on device\n"
     ]
    },
    {
     "ename": "",
     "evalue": "1",
     "execution_count": 8,
     "output_type": "error",
     "traceback": []
    }
   ],
   "source": [
    "unicycler -l ecoli_allreads.fasta \\\n",
    " -o nanopore_only"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Wow, I ran out of storage, this is bad, I can't even save this notebook. Let's remove some files - I'm going to remove the Illumina files."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "rm -rf s_6_1.fastq.gz s_6_2.fastq.gz"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's check the storage size."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Filesystem      Size  Used Avail Use% Mounted on\n",
      "udev             60G     0   60G   0% /dev\n",
      "tmpfs            12G  8.8M   12G   1% /run\n",
      "/dev/xvda1       49G   47G  2.3G  96% /\n",
      "tmpfs            60G     0   60G   0% /dev/shm\n",
      "tmpfs           5.0M     0  5.0M   0% /run/lock\n",
      "tmpfs            60G     0   60G   0% /sys/fs/cgroup\n",
      "/dev/loop0       88M   88M     0 100% /snap/core/5742\n",
      "/dev/loop1       90M   90M     0 100% /snap/core/6673\n",
      "/dev/loop2       17M   17M     0 100% /snap/amazon-ssm-agent/784\n",
      "/dev/loop3       18M   18M     0 100% /snap/amazon-ssm-agent/1068\n",
      "tmpfs            12G   16K   12G   1% /run/user/1000\n"
     ]
    }
   ],
   "source": [
    "df -h"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "I'm not sure here, there are so many numbers. My guess is /dev/xvda would be the most important Filesystem since it is the biggest size. The filesystem has 47 Gb, and only 4% is left after removing the Illumina files - this is the problem.\n",
    "\n",
    "We have to think of a way to solve this."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Bash",
   "language": "bash",
   "name": "bash"
  },
  "language_info": {
   "codemirror_mode": "shell",
   "file_extension": ".sh",
   "mimetype": "text/x-sh",
   "name": "bash"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}