diff --git a/DEVELOP.rst b/DEVELOP.rst
new file mode 100644
index 0000000..4e7846d
--- /dev/null
+++ b/DEVELOP.rst
@@ -0,0 +1,194 @@
+Genomeshader development guide
+""""""""""""""""""""""""""""""
+
+We welcome contributions to the codebase. This document provides guidance
+on setting up your development environment to test new features locally, then
+contributing your changes to the main repository.
+
+
+About the codebase
+------------------
+Genomeshader is written in two languages: Rust (for "back-end" operations:
+multi-threaded data retrieval from Google Cloud Storage, alignment processing,
+columnar storage, etc.) and Python (for "front-end" operations: visualization,
+Jupyter notebook interaction, state maintainence, etc.).
+
+The Rust components of the library make certain computationally expensive tasks
+orders of magnitude faster than equivalent Python code. While the Python parts
+of the library will call into this code freely, it is generally expected that
+the end user will never call the Rust functions directly.
+
+The Python components of the library are mostly intended to supply the visualization
+capabilities, dataset grouping, or other high-level manipulation features. We
+rely on the `Datashader `_ library for rapid visualization
+of large datasets, and the `Polars `_ library
+for manipulation of large genomic dataframes.
+
+
+Setting up your development environment
+---------------------------------------
+
+
+We recommend using Visual Studio Code (VSCode) as your development environment.
+Follow the steps below to set up your development environment:
+
+1. Install Rust
+ Open a terminal and run the following command to install Rust:
+
+ .. code-block:: bash
+
+ curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
+
+
+ This will download a script and start the installation. Once the installation
+ is complete, close the terminal and open a new one to ensure the changes take effect.
+ You can verify that you have installed Rust correctly by running the following
+ command in your terminal:
+
+ .. code-block:: bash
+
+ rustc --version
+
+ If Rust is installed correctly, this command should display the version of Rust currently installed.
+
+2. Install Visual Studio Code
+ You can download it from `here `_.
+ Follow the instructions on the website to install it.
+
+3. Install Rust and Python extensions for VSCode
+ Open VSCode, go to the Extensions view by clicking on the Extensions icon in the Activity Bar on the side of the window. In the Extensions view, search for and install the following extensions:
+
+ - 'rust-analyzer': This extension provides advanced language support for Rust.
+ - 'CodeLLDB': This extension is an optional but recommended tool for developing and debugging Rust code.
+ - 'Jupyter': This extension provides Jupyter notebook support for VSCode.
+
+4. Create a virtual environment for development
+ Python's built-in `venv` module allows you to create virtual environments. These environments have their own installation directories and don't share libraries with other virtual environments. Here's how to create one:
+
+ - Open a terminal and navigate to the directory where you want to create the virtual environment.
+ - Run the following command to create a new virtual environment:
+
+ .. code-block:: bash
+
+ python3 -m venv genomeshader-env
+
+ This will create a new directory called `genomeshader-env` in your current directory. This directory will contain the Python executable files and a copy of the pip library which you can use to install other packages.
+
+ - To activate the virtual environment, run the following command:
+
+ .. code-block:: bash
+
+ source genomeshader-env/bin/activate
+
+ Once the virtual environment is activated, the name of your virtual environment will appear on left side of the terminal prompt. This will let you know that the virtual environment is currently active.
+
+ - In the virtual environment, you can use the command `pip` to install packages that will be isolated from the global Python installation. Install the required packages for Genomeshader development by running the following command:
+
+ .. code-block:: bash
+
+ pip install -r requirements.txt
+
+ - When you are done with your work, you can deactivate the virtual environment by running the following command:
+
+ .. code-block:: bash
+
+ deactivate
+
+ This command will deactivate the virtual environment and you will return to your normal shell.
+
+5. Compile and install Genomeshader into the virtual environment
+ To compile and install Genomeshader, we will use the `maturin develop --release` command. `maturin` is a build system for Rust-based Python extensions, and the `develop` command compiles and installs the package into the current Python interpreter. The `--release` flag is used to compile the package in release mode, which includes optimizations.
+
+ Run the following command in your terminal:
+
+ .. code-block:: bash
+
+ maturin develop --release
+
+ This command will compile the Genomeshader Rust code and install the resulting Python package into your active virtual environment. This means you can now import and use the Genomeshader library in your Python scripts and Jupyter notebooks.
+
+6. Open the `playground.ipynb` notebook.
+ - Make sure that the notebook is set to use the virtual environment you created earlier. In Jupyter, you can do this by clicking on 'Kernel' > 'Change kernel' and selecting your virtual environment.
+ - In the first cell of the notebook, import the Genomeshader library by running the following code:
+
+ .. code-block:: python
+
+ import genomeshader as gs
+
+ If the library imports successfully, you are ready to start using Genomeshader in your notebook.
+
+Each time you make changes to the codebase, recompile the library by rerunning
+step 5, and then trying out the changes in step 6.
+
+Now, you are ready to start developing with Genomeshader!
+
+
+Contributing to Genomeshader
+----------------------------
+
+1. Fork the Genomeshader repository
+ Go to the `Genomeshader repository `_ and click on the "Fork" button. This will create a copy of the repository in your own GitHub account.
+
+2. Clone the forked repository
+ On your GitHub account, navigate to the forked repository and click on the "Clone or download" button. Copy the URL.
+ Open a terminal and run the following git command:
+
+ .. code-block:: bash
+
+ git clone "url you just copied"
+
+3. Create a new branch
+ Change to the repository directory on your computer (if you are not already there):
+
+ .. code-block:: bash
+
+ cd genomeshader
+
+ Now create a branch using the `git checkout` command:
+
+ .. code-block:: bash
+
+ git checkout -b your-new-branch-name
+
+4. Make necessary changes and commit those changes
+ Now you can go ahead and make changes to the files. Once you've made changes or added files, you can see them listed with `git status`. Add those changes with `git add` and then commit those changes:
+
+ .. code-block:: bash
+
+ git add .
+ git commit -m "commit message"
+
+5. Push changes to GitHub
+ Push your changes using the command `git push`:
+
+ .. code-block:: bash
+
+ git push origin your-new-branch-name
+
+6. Submit your changes for review
+ If you go to your repository on GitHub, you'll see a "Compare & pull request" button. Click on that button and describe the changes you made. Once you submit the pull request, a Genomeshader reviewer will review your changes.
+
+
+Code Style Guidelines
+---------------------
+We follow the official style guides for our code. For Rust, we adhere to the `Rust Style Guide `_. For Python, we follow the `PEP 8 Style Guide `_. Please ensure your contributions adhere to these standards.
+
+Testing
+-------
+We use pytest for our Python tests and cargo test for our Rust tests. Please add tests for new features and ensure all tests pass before submitting a pull request.
+
+Documentation
+-------------
+Please update the documentation to reflect any changes you make to the codebase. This includes comments in the code, docstrings, and our user guides.
+
+Issue Tracking
+--------------
+We use GitHub issues to track work on Genomeshader. If you're adding a new feature or fixing a bug, please create an issue describing the work.
+
+Communication
+-------------
+If you have any questions or want to discuss your work, please join our community chat on Slack or by email. Our team is always happy to help.
+
+Code Review Process
+-------------------
+After you submit your pull request, it will be reviewed by at least one core contributor. We'll provide feedback and may request changes. Once your pull request is approved, it will be merged into the main codebase.
\ No newline at end of file
diff --git a/README.rst b/README.rst
index e637408..eef17a9 100644
--- a/README.rst
+++ b/README.rst
@@ -9,7 +9,7 @@ genomeshader
.. |PyPI version genomeshader| image:: https://img.shields.io/pypi/v/genomeshader.svg
:target: https://pypi.python.org/pypi/genomeshader/
-Genomeshader is a Rust/Python library for rapid visualization of read-level data spanning variants across huge numbers of samples.
+Genomeshader is a Rust-backed Python library for rapid visualization of read-level data spanning variants across huge numbers of samples. It is intended for use within Jupyter notebooks.
Documentation for the API can be found on the `documentation page `_.
@@ -26,7 +26,7 @@ Installation
Building from source
--------------------
-To build from source, follow the procedure below.
+To build from source (particularly for those interested in contributing to the code), follow the procedure below.
.. code-block:: bash
@@ -34,7 +34,8 @@ To build from source, follow the procedure below.
git clone https://github.com/broadinstitute/genomeshader.git
cd genomeshader
- # Create a Python virtual environment and install Maturin.
+ # Create a Python virtual environment and install Maturin, the tool that
+ # will compile the Rust and Python code into a complete library.
# For more information on Maturin, visit https://github.com/PyO3/maturin .
python -mvenv venv
. venv/bin/activate
@@ -47,7 +48,7 @@ To build from source, follow the procedure below.
Supported platforms
-------------------
-Genomeshader is compiled for Linux (x86_64) and MacOSX (x86_64, aarch64). Windows support is not currently available.
+Genomeshader is compiled for Linux and MacOSX. Windows is not currently supported.
Getting help
------------
@@ -57,4 +58,4 @@ If you encounter bugs or have questions/comments/concerns, please file an issue
Developers' guide
-----------------
-For information on contributing to Genomeshader development, visit our `developer documentation `_.
+For information on contributing to Genomeshader development, visit our `developer documentation `_.
diff --git a/playground.ipynb b/playground.ipynb
index fd762e1..d8e896c 100644
--- a/playground.ipynb
+++ b/playground.ipynb
@@ -7,7 +7,7 @@
"outputs": [
{
"data": {
- "application/javascript": "(function(root) {\n function now() {\n return new Date();\n }\n\n var force = true;\n var py_version = '3.3.2'.replace('rc', '-rc.').replace('.dev', '-dev.');\n var reloading = false;\n var Bokeh = root.Bokeh;\n\n if (typeof (root._bokeh_timeout) === \"undefined\" || force) {\n root._bokeh_timeout = Date.now() + 5000;\n root._bokeh_failed_load = false;\n }\n\n function run_callbacks() {\n try {\n root._bokeh_onload_callbacks.forEach(function(callback) {\n if (callback != null)\n callback();\n });\n } finally {\n delete root._bokeh_onload_callbacks;\n }\n console.debug(\"Bokeh: all callbacks have finished\");\n }\n\n function load_libs(css_urls, js_urls, js_modules, js_exports, callback) {\n if (css_urls == null) css_urls = [];\n if (js_urls == null) js_urls = [];\n if (js_modules == null) js_modules = [];\n if (js_exports == null) js_exports = {};\n\n root._bokeh_onload_callbacks.push(callback);\n\n if (root._bokeh_is_loading > 0) {\n console.debug(\"Bokeh: BokehJS is being loaded, scheduling callback at\", now());\n return null;\n }\n if (js_urls.length === 0 && js_modules.length === 0 && Object.keys(js_exports).length === 0) {\n run_callbacks();\n return null;\n }\n if (!reloading) {\n console.debug(\"Bokeh: BokehJS not loaded, scheduling load and callback at\", now());\n }\n\n function on_load() {\n root._bokeh_is_loading--;\n if (root._bokeh_is_loading === 0) {\n console.debug(\"Bokeh: all BokehJS libraries/stylesheets loaded\");\n run_callbacks()\n }\n }\n window._bokeh_on_load = on_load\n\n function on_error() {\n console.error(\"failed to load \" + url);\n }\n\n var skip = [];\n if (window.requirejs) {\n window.requirejs.config({'packages': {}, 'paths': {'jspanel': 'https://cdn.jsdelivr.net/npm/jspanel4@4.12.0/dist/jspanel', 'jspanel-modal': 'https://cdn.jsdelivr.net/npm/jspanel4@4.12.0/dist/extensions/modal/jspanel.modal', 'jspanel-tooltip': 'https://cdn.jsdelivr.net/npm/jspanel4@4.12.0/dist/extensions/tooltip/jspanel.tooltip', 'jspanel-hint': 'https://cdn.jsdelivr.net/npm/jspanel4@4.12.0/dist/extensions/hint/jspanel.hint', 'jspanel-layout': 'https://cdn.jsdelivr.net/npm/jspanel4@4.12.0/dist/extensions/layout/jspanel.layout', 'jspanel-contextmenu': 'https://cdn.jsdelivr.net/npm/jspanel4@4.12.0/dist/extensions/contextmenu/jspanel.contextmenu', 'jspanel-dock': 'https://cdn.jsdelivr.net/npm/jspanel4@4.12.0/dist/extensions/dock/jspanel.dock', 'gridstack': 'https://cdn.jsdelivr.net/npm/gridstack@7.2.3/dist/gridstack-all', 'notyf': 'https://cdn.jsdelivr.net/npm/notyf@3/notyf.min'}, 'shim': {'jspanel': {'exports': 'jsPanel'}, 'gridstack': {'exports': 'GridStack'}}});\n require([\"jspanel\"], function(jsPanel) {\n\twindow.jsPanel = jsPanel\n\ton_load()\n })\n require([\"jspanel-modal\"], function() {\n\ton_load()\n })\n require([\"jspanel-tooltip\"], function() {\n\ton_load()\n })\n require([\"jspanel-hint\"], function() {\n\ton_load()\n })\n require([\"jspanel-layout\"], function() {\n\ton_load()\n })\n require([\"jspanel-contextmenu\"], function() {\n\ton_load()\n })\n require([\"jspanel-dock\"], function() {\n\ton_load()\n })\n require([\"gridstack\"], function(GridStack) {\n\twindow.GridStack = GridStack\n\ton_load()\n })\n require([\"notyf\"], function() {\n\ton_load()\n })\n root._bokeh_is_loading = css_urls.length + 9;\n } else {\n root._bokeh_is_loading = css_urls.length + js_urls.length + js_modules.length + Object.keys(js_exports).length;\n }\n\n var existing_stylesheets = []\n var links = document.getElementsByTagName('link')\n for (var i = 0; i < links.length; i++) {\n var link = links[i]\n if (link.href != null) {\n\texisting_stylesheets.push(link.href)\n }\n }\n for (var i = 0; i < css_urls.length; i++) {\n var url = css_urls[i];\n if (existing_stylesheets.indexOf(url) !== -1) {\n\ton_load()\n\tcontinue;\n }\n const element = document.createElement(\"link\");\n element.onload = on_load;\n element.onerror = on_error;\n element.rel = \"stylesheet\";\n element.type = \"text/css\";\n element.href = url;\n console.debug(\"Bokeh: injecting link tag for BokehJS stylesheet: \", url);\n document.body.appendChild(element);\n } if (((window['jsPanel'] !== undefined) && (!(window['jsPanel'] instanceof HTMLElement))) || window.requirejs) {\n var urls = ['https://cdn.holoviz.org/panel/1.3.6/dist/bundled/floatpanel/jspanel4@4.12.0/dist/jspanel.js', 'https://cdn.holoviz.org/panel/1.3.6/dist/bundled/floatpanel/jspanel4@4.12.0/dist/extensions/modal/jspanel.modal.js', 'https://cdn.holoviz.org/panel/1.3.6/dist/bundled/floatpanel/jspanel4@4.12.0/dist/extensions/tooltip/jspanel.tooltip.js', 'https://cdn.holoviz.org/panel/1.3.6/dist/bundled/floatpanel/jspanel4@4.12.0/dist/extensions/hint/jspanel.hint.js', 'https://cdn.holoviz.org/panel/1.3.6/dist/bundled/floatpanel/jspanel4@4.12.0/dist/extensions/layout/jspanel.layout.js', 'https://cdn.holoviz.org/panel/1.3.6/dist/bundled/floatpanel/jspanel4@4.12.0/dist/extensions/contextmenu/jspanel.contextmenu.js', 'https://cdn.holoviz.org/panel/1.3.6/dist/bundled/floatpanel/jspanel4@4.12.0/dist/extensions/dock/jspanel.dock.js'];\n for (var i = 0; i < urls.length; i++) {\n skip.push(urls[i])\n }\n } if (((window['GridStack'] !== undefined) && (!(window['GridStack'] instanceof HTMLElement))) || window.requirejs) {\n var urls = ['https://cdn.holoviz.org/panel/1.3.6/dist/bundled/gridstack/gridstack@7.2.3/dist/gridstack-all.js'];\n for (var i = 0; i < urls.length; i++) {\n skip.push(urls[i])\n }\n } if (((window['Notyf'] !== undefined) && (!(window['Notyf'] instanceof HTMLElement))) || window.requirejs) {\n var urls = ['https://cdn.holoviz.org/panel/1.3.6/dist/bundled/notificationarea/notyf@3/notyf.min.js'];\n for (var i = 0; i < urls.length; i++) {\n skip.push(urls[i])\n }\n } var existing_scripts = []\n var scripts = document.getElementsByTagName('script')\n for (var i = 0; i < scripts.length; i++) {\n var script = scripts[i]\n if (script.src != null) {\n\texisting_scripts.push(script.src)\n }\n }\n for (var i = 0; i < js_urls.length; i++) {\n var url = js_urls[i];\n if (skip.indexOf(url) !== -1 || existing_scripts.indexOf(url) !== -1) {\n\tif (!window.requirejs) {\n\t on_load();\n\t}\n\tcontinue;\n }\n var element = document.createElement('script');\n element.onload = on_load;\n element.onerror = on_error;\n element.async = false;\n element.src = url;\n console.debug(\"Bokeh: injecting script tag for BokehJS library: \", url);\n document.head.appendChild(element);\n }\n for (var i = 0; i < js_modules.length; i++) {\n var url = js_modules[i];\n if (skip.indexOf(url) !== -1 || existing_scripts.indexOf(url) !== -1) {\n\tif (!window.requirejs) {\n\t on_load();\n\t}\n\tcontinue;\n }\n var element = document.createElement('script');\n element.onload = on_load;\n element.onerror = on_error;\n element.async = false;\n element.src = url;\n element.type = \"module\";\n console.debug(\"Bokeh: injecting script tag for BokehJS library: \", url);\n document.head.appendChild(element);\n }\n for (const name in js_exports) {\n var url = js_exports[name];\n if (skip.indexOf(url) >= 0 || root[name] != null) {\n\tif (!window.requirejs) {\n\t on_load();\n\t}\n\tcontinue;\n }\n var element = document.createElement('script');\n element.onerror = on_error;\n element.async = false;\n element.type = \"module\";\n console.debug(\"Bokeh: injecting script tag for BokehJS library: \", url);\n element.textContent = `\n import ${name} from \"${url}\"\n window.${name} = ${name}\n window._bokeh_on_load()\n `\n document.head.appendChild(element);\n }\n if (!js_urls.length && !js_modules.length) {\n on_load()\n }\n };\n\n function inject_raw_css(css) {\n const element = document.createElement(\"style\");\n element.appendChild(document.createTextNode(css));\n document.body.appendChild(element);\n }\n\n var js_urls = [\"https://cdn.bokeh.org/bokeh/release/bokeh-3.3.2.min.js\", \"https://cdn.bokeh.org/bokeh/release/bokeh-gl-3.3.2.min.js\", \"https://cdn.bokeh.org/bokeh/release/bokeh-widgets-3.3.2.min.js\", \"https://cdn.bokeh.org/bokeh/release/bokeh-tables-3.3.2.min.js\", \"https://cdn.holoviz.org/panel/1.3.6/dist/panel.min.js\"];\n var js_modules = [];\n var js_exports = {};\n var css_urls = [];\n var inline_js = [ function(Bokeh) {\n Bokeh.set_log_level(\"info\");\n },\nfunction(Bokeh) {} // ensure no trailing comma for IE\n ];\n\n function run_inline_js() {\n if ((root.Bokeh !== undefined) || (force === true)) {\n for (var i = 0; i < inline_js.length; i++) {\n\ttry {\n inline_js[i].call(root, root.Bokeh);\n\t} catch(e) {\n\t if (!reloading) {\n\t throw e;\n\t }\n\t}\n }\n // Cache old bokeh versions\n if (Bokeh != undefined && !reloading) {\n\tvar NewBokeh = root.Bokeh;\n\tif (Bokeh.versions === undefined) {\n\t Bokeh.versions = new Map();\n\t}\n\tif (NewBokeh.version !== Bokeh.version) {\n\t Bokeh.versions.set(NewBokeh.version, NewBokeh)\n\t}\n\troot.Bokeh = Bokeh;\n }} else if (Date.now() < root._bokeh_timeout) {\n setTimeout(run_inline_js, 100);\n } else if (!root._bokeh_failed_load) {\n console.log(\"Bokeh: BokehJS failed to load within specified timeout.\");\n root._bokeh_failed_load = true;\n }\n root._bokeh_is_initializing = false\n }\n\n function load_or_wait() {\n // Implement a backoff loop that tries to ensure we do not load multiple\n // versions of Bokeh and its dependencies at the same time.\n // In recent versions we use the root._bokeh_is_initializing flag\n // to determine whether there is an ongoing attempt to initialize\n // bokeh, however for backward compatibility we also try to ensure\n // that we do not start loading a newer (Panel>=1.0 and Bokeh>3) version\n // before older versions are fully initialized.\n if (root._bokeh_is_initializing && Date.now() > root._bokeh_timeout) {\n root._bokeh_is_initializing = false;\n root._bokeh_onload_callbacks = undefined;\n console.log(\"Bokeh: BokehJS was loaded multiple times but one version failed to initialize.\");\n load_or_wait();\n } else if (root._bokeh_is_initializing || (typeof root._bokeh_is_initializing === \"undefined\" && root._bokeh_onload_callbacks !== undefined)) {\n setTimeout(load_or_wait, 100);\n } else {\n root._bokeh_is_initializing = true\n root._bokeh_onload_callbacks = []\n var bokeh_loaded = Bokeh != null && (Bokeh.version === py_version || (Bokeh.versions !== undefined && Bokeh.versions.has(py_version)));\n if (!reloading && !bokeh_loaded) {\n\troot.Bokeh = undefined;\n }\n load_libs(css_urls, js_urls, js_modules, js_exports, function() {\n\tconsole.debug(\"Bokeh: BokehJS plotting callback run at\", now());\n\trun_inline_js();\n });\n }\n }\n // Give older versions of the autoload script a head-start to ensure\n // they initialize before we start loading newer version.\n setTimeout(load_or_wait, 100)\n}(window));",
+ "application/javascript": "(function(root) {\n function now() {\n return new Date();\n }\n\n var force = true;\n var py_version = '3.1.1'.replace('rc', '-rc.').replace('.dev', '-dev.');\n var is_dev = py_version.indexOf(\"+\") !== -1 || py_version.indexOf(\"-\") !== -1;\n var reloading = false;\n var Bokeh = root.Bokeh;\n var bokeh_loaded = Bokeh != null && (Bokeh.version === py_version || (Bokeh.versions !== undefined && Bokeh.versions.has(py_version)));\n\n if (typeof (root._bokeh_timeout) === \"undefined\" || force) {\n root._bokeh_timeout = Date.now() + 5000;\n root._bokeh_failed_load = false;\n }\n\n function run_callbacks() {\n try {\n root._bokeh_onload_callbacks.forEach(function(callback) {\n if (callback != null)\n callback();\n });\n } finally {\n delete root._bokeh_onload_callbacks;\n }\n console.debug(\"Bokeh: all callbacks have finished\");\n }\n\n function load_libs(css_urls, js_urls, js_modules, js_exports, callback) {\n if (css_urls == null) css_urls = [];\n if (js_urls == null) js_urls = [];\n if (js_modules == null) js_modules = [];\n if (js_exports == null) js_exports = {};\n\n root._bokeh_onload_callbacks.push(callback);\n\n if (root._bokeh_is_loading > 0) {\n console.debug(\"Bokeh: BokehJS is being loaded, scheduling callback at\", now());\n return null;\n }\n if (js_urls.length === 0 && js_modules.length === 0 && Object.keys(js_exports).length === 0) {\n run_callbacks();\n return null;\n }\n if (!reloading) {\n console.debug(\"Bokeh: BokehJS not loaded, scheduling load and callback at\", now());\n }\n\n function on_load() {\n root._bokeh_is_loading--;\n if (root._bokeh_is_loading === 0) {\n console.debug(\"Bokeh: all BokehJS libraries/stylesheets loaded\");\n run_callbacks()\n }\n }\n window._bokeh_on_load = on_load\n\n function on_error() {\n console.error(\"failed to load \" + url);\n }\n\n var skip = [];\n if (window.requirejs) {\n window.requirejs.config({'packages': {}, 'paths': {'jspanel': 'https://cdn.jsdelivr.net/npm/jspanel4@4.12.0/dist/jspanel', 'jspanel-modal': 'https://cdn.jsdelivr.net/npm/jspanel4@4.12.0/dist/extensions/modal/jspanel.modal', 'jspanel-tooltip': 'https://cdn.jsdelivr.net/npm/jspanel4@4.12.0/dist/extensions/tooltip/jspanel.tooltip', 'jspanel-hint': 'https://cdn.jsdelivr.net/npm/jspanel4@4.12.0/dist/extensions/hint/jspanel.hint', 'jspanel-layout': 'https://cdn.jsdelivr.net/npm/jspanel4@4.12.0/dist/extensions/layout/jspanel.layout', 'jspanel-contextmenu': 'https://cdn.jsdelivr.net/npm/jspanel4@4.12.0/dist/extensions/contextmenu/jspanel.contextmenu', 'jspanel-dock': 'https://cdn.jsdelivr.net/npm/jspanel4@4.12.0/dist/extensions/dock/jspanel.dock', 'gridstack': 'https://cdn.jsdelivr.net/npm/gridstack@7.2.3/dist/gridstack-all', 'notyf': 'https://cdn.jsdelivr.net/npm/notyf@3/notyf.min'}, 'shim': {'jspanel': {'exports': 'jsPanel'}, 'gridstack': {'exports': 'GridStack'}}});\n require([\"jspanel\"], function(jsPanel) {\n\twindow.jsPanel = jsPanel\n\ton_load()\n })\n require([\"jspanel-modal\"], function() {\n\ton_load()\n })\n require([\"jspanel-tooltip\"], function() {\n\ton_load()\n })\n require([\"jspanel-hint\"], function() {\n\ton_load()\n })\n require([\"jspanel-layout\"], function() {\n\ton_load()\n })\n require([\"jspanel-contextmenu\"], function() {\n\ton_load()\n })\n require([\"jspanel-dock\"], function() {\n\ton_load()\n })\n require([\"gridstack\"], function(GridStack) {\n\twindow.GridStack = GridStack\n\ton_load()\n })\n require([\"notyf\"], function() {\n\ton_load()\n })\n root._bokeh_is_loading = css_urls.length + 9;\n } else {\n root._bokeh_is_loading = css_urls.length + js_urls.length + js_modules.length + Object.keys(js_exports).length;\n }\n\n var existing_stylesheets = []\n var links = document.getElementsByTagName('link')\n for (var i = 0; i < links.length; i++) {\n var link = links[i]\n if (link.href != null) {\n\texisting_stylesheets.push(link.href)\n }\n }\n for (var i = 0; i < css_urls.length; i++) {\n var url = css_urls[i];\n if (existing_stylesheets.indexOf(url) !== -1) {\n\ton_load()\n\tcontinue;\n }\n const element = document.createElement(\"link\");\n element.onload = on_load;\n element.onerror = on_error;\n element.rel = \"stylesheet\";\n element.type = \"text/css\";\n element.href = url;\n console.debug(\"Bokeh: injecting link tag for BokehJS stylesheet: \", url);\n document.body.appendChild(element);\n } if (((window['jsPanel'] !== undefined) && (!(window['jsPanel'] instanceof HTMLElement))) || window.requirejs) {\n var urls = ['https://cdn.holoviz.org/panel/1.2.3/dist/bundled/floatpanel/jspanel4@4.12.0/dist/jspanel.js', 'https://cdn.holoviz.org/panel/1.2.3/dist/bundled/floatpanel/jspanel4@4.12.0/dist/extensions/modal/jspanel.modal.js', 'https://cdn.holoviz.org/panel/1.2.3/dist/bundled/floatpanel/jspanel4@4.12.0/dist/extensions/tooltip/jspanel.tooltip.js', 'https://cdn.holoviz.org/panel/1.2.3/dist/bundled/floatpanel/jspanel4@4.12.0/dist/extensions/hint/jspanel.hint.js', 'https://cdn.holoviz.org/panel/1.2.3/dist/bundled/floatpanel/jspanel4@4.12.0/dist/extensions/layout/jspanel.layout.js', 'https://cdn.holoviz.org/panel/1.2.3/dist/bundled/floatpanel/jspanel4@4.12.0/dist/extensions/contextmenu/jspanel.contextmenu.js', 'https://cdn.holoviz.org/panel/1.2.3/dist/bundled/floatpanel/jspanel4@4.12.0/dist/extensions/dock/jspanel.dock.js'];\n for (var i = 0; i < urls.length; i++) {\n skip.push(urls[i])\n }\n } if (((window['GridStack'] !== undefined) && (!(window['GridStack'] instanceof HTMLElement))) || window.requirejs) {\n var urls = ['https://cdn.holoviz.org/panel/1.2.3/dist/bundled/gridstack/gridstack@7.2.3/dist/gridstack-all.js'];\n for (var i = 0; i < urls.length; i++) {\n skip.push(urls[i])\n }\n } if (((window['Notyf'] !== undefined) && (!(window['Notyf'] instanceof HTMLElement))) || window.requirejs) {\n var urls = ['https://cdn.holoviz.org/panel/1.2.3/dist/bundled/notificationarea/notyf@3/notyf.min.js'];\n for (var i = 0; i < urls.length; i++) {\n skip.push(urls[i])\n }\n } var existing_scripts = []\n var scripts = document.getElementsByTagName('script')\n for (var i = 0; i < scripts.length; i++) {\n var script = scripts[i]\n if (script.src != null) {\n\texisting_scripts.push(script.src)\n }\n }\n for (var i = 0; i < js_urls.length; i++) {\n var url = js_urls[i];\n if (skip.indexOf(url) !== -1 || existing_scripts.indexOf(url) !== -1) {\n\tif (!window.requirejs) {\n\t on_load();\n\t}\n\tcontinue;\n }\n var element = document.createElement('script');\n element.onload = on_load;\n element.onerror = on_error;\n element.async = false;\n element.src = url;\n console.debug(\"Bokeh: injecting script tag for BokehJS library: \", url);\n document.head.appendChild(element);\n }\n for (var i = 0; i < js_modules.length; i++) {\n var url = js_modules[i];\n if (skip.indexOf(url) !== -1 || existing_scripts.indexOf(url) !== -1) {\n\tif (!window.requirejs) {\n\t on_load();\n\t}\n\tcontinue;\n }\n var element = document.createElement('script');\n element.onload = on_load;\n element.onerror = on_error;\n element.async = false;\n element.src = url;\n element.type = \"module\";\n console.debug(\"Bokeh: injecting script tag for BokehJS library: \", url);\n document.head.appendChild(element);\n }\n for (const name in js_exports) {\n var url = js_exports[name];\n if (skip.indexOf(url) >= 0 || root[name] != null) {\n\tif (!window.requirejs) {\n\t on_load();\n\t}\n\tcontinue;\n }\n var element = document.createElement('script');\n element.onerror = on_error;\n element.async = false;\n element.type = \"module\";\n console.debug(\"Bokeh: injecting script tag for BokehJS library: \", url);\n element.textContent = `\n import ${name} from \"${url}\"\n window.${name} = ${name}\n window._bokeh_on_load()\n `\n document.head.appendChild(element);\n }\n if (!js_urls.length && !js_modules.length) {\n on_load()\n }\n };\n\n function inject_raw_css(css) {\n const element = document.createElement(\"style\");\n element.appendChild(document.createTextNode(css));\n document.body.appendChild(element);\n }\n\n var js_urls = [\"https://cdn.bokeh.org/bokeh/release/bokeh-3.1.1.min.js\", \"https://cdn.bokeh.org/bokeh/release/bokeh-gl-3.1.1.min.js\", \"https://cdn.bokeh.org/bokeh/release/bokeh-widgets-3.1.1.min.js\", \"https://cdn.bokeh.org/bokeh/release/bokeh-tables-3.1.1.min.js\", \"https://cdn.holoviz.org/panel/1.2.3/dist/panel.min.js\"];\n var js_modules = [];\n var js_exports = {};\n var css_urls = [];\n var inline_js = [ function(Bokeh) {\n Bokeh.set_log_level(\"info\");\n },\nfunction(Bokeh) {} // ensure no trailing comma for IE\n ];\n\n function run_inline_js() {\n if ((root.Bokeh !== undefined) || (force === true)) {\n for (var i = 0; i < inline_js.length; i++) {\n inline_js[i].call(root, root.Bokeh);\n }\n // Cache old bokeh versions\n if (Bokeh != undefined && !reloading) {\n\tvar NewBokeh = root.Bokeh;\n\tif (Bokeh.versions === undefined) {\n\t Bokeh.versions = new Map();\n\t}\n\tif (NewBokeh.version !== Bokeh.version) {\n\t Bokeh.versions.set(NewBokeh.version, NewBokeh)\n\t}\n\troot.Bokeh = Bokeh;\n }} else if (Date.now() < root._bokeh_timeout) {\n setTimeout(run_inline_js, 100);\n } else if (!root._bokeh_failed_load) {\n console.log(\"Bokeh: BokehJS failed to load within specified timeout.\");\n root._bokeh_failed_load = true;\n }\n root._bokeh_is_initializing = false\n }\n\n function load_or_wait() {\n // Implement a backoff loop that tries to ensure we do not load multiple\n // versions of Bokeh and its dependencies at the same time.\n // In recent versions we use the root._bokeh_is_initializing flag\n // to determine whether there is an ongoing attempt to initialize\n // bokeh, however for backward compatibility we also try to ensure\n // that we do not start loading a newer (Panel>=1.0 and Bokeh>3) version\n // before older versions are fully initialized.\n if (root._bokeh_is_initializing && Date.now() > root._bokeh_timeout) {\n root._bokeh_is_initializing = false;\n root._bokeh_onload_callbacks = undefined;\n console.log(\"Bokeh: BokehJS was loaded multiple times but one version failed to initialize.\");\n load_or_wait();\n } else if (root._bokeh_is_initializing || (typeof root._bokeh_is_initializing === \"undefined\" && root._bokeh_onload_callbacks !== undefined)) {\n setTimeout(load_or_wait, 100);\n } else {\n Bokeh = root.Bokeh;\n bokeh_loaded = Bokeh != null && (Bokeh.version === py_version || (Bokeh.versions !== undefined && Bokeh.versions.has(py_version)));\n root._bokeh_is_initializing = true\n root._bokeh_onload_callbacks = []\n if (!reloading && (!bokeh_loaded || is_dev)) {\n\troot.Bokeh = undefined;\n }\n load_libs(css_urls, js_urls, js_modules, js_exports, function() {\n\tconsole.debug(\"Bokeh: BokehJS plotting callback run at\", now());\n\trun_inline_js();\n });\n }\n }\n // Give older versions of the autoload script a head-start to ensure\n // they initialize before we start loading newer version.\n setTimeout(load_or_wait, 100)\n}(window));",
"application/vnd.holoviews_load.v0+json": ""
},
"metadata": {},
@@ -45,84 +45,6 @@
"metadata": {},
"output_type": "display_data"
},
- {
- "data": {
- "application/vnd.holoviews_exec.v0+json": "",
- "text/html": [
- "\n",
- ""
- ]
- },
- "metadata": {
- "application/vnd.holoviews_exec.v0+json": {
- "id": "p1002"
- }
- },
- "output_type": "display_data"
- },
{
"data": {
"text/html": [
@@ -231,6 +153,26 @@
"cell_type": "code",
"execution_count": 3,
"metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "'0.1.11'"
+ ]
+ },
+ "execution_count": 3,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "s.get_version()"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 4,
+ "metadata": {},
"outputs": [],
"source": [
"s.attach_reads([\n",
@@ -242,7 +184,7 @@
},
{
"cell_type": "code",
- "execution_count": 4,
+ "execution_count": 5,
"metadata": {},
"outputs": [],
"source": [
@@ -253,14 +195,37 @@
},
{
"cell_type": "code",
- "execution_count": 5,
+ "execution_count": 6,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
- "Execution time: 2.2379889488220215 seconds\n"
+ "Reads:\n",
+ " - gs://fc-8c3900db-633f-477f-96b3-fb31ae265c44/results/PBFlowcell/m84175_231021_212604_s2/reads/ccs/aligned/m84175_231021_212604_s2.bam\n",
+ " - gs://fc-8c3900db-633f-477f-96b3-fb31ae265c44/results/PBFlowcell/m84175_231021_215710_s3/reads/ccs/aligned/m84175_231021_215710_s3.bam\n",
+ " - gs://fc-8c3900db-633f-477f-96b3-fb31ae265c44/results/PBFlowcell/m84175_231021_222816_s4/reads/ccs/aligned/m84175_231021_222816_s4.bam\n",
+ "Loci:\n",
+ " - (\"chr15\", 23960193, 23963918)\n",
+ "Staging:\n"
+ ]
+ }
+ ],
+ "source": [
+ "s.print()"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 12,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Execution time: 4.681073904037476 seconds\n"
]
}
],
@@ -275,7 +240,7 @@
},
{
"cell_type": "code",
- "execution_count": 6,
+ "execution_count": 9,
"metadata": {},
"outputs": [
{
@@ -283,6 +248,8 @@
"output_type": "stream",
"text": [
"Reads:\n",
+ " - gs://fc-8c3900db-633f-477f-96b3-fb31ae265c44/results/PBFlowcell/m84175_231021_212604_s2/reads/ccs/aligned/m84175_231021_212604_s2.bam\n",
+ " - gs://fc-8c3900db-633f-477f-96b3-fb31ae265c44/results/PBFlowcell/m84175_231021_215710_s3/reads/ccs/aligned/m84175_231021_215710_s3.bam\n",
" - gs://fc-8c3900db-633f-477f-96b3-fb31ae265c44/results/PBFlowcell/m84175_231021_222816_s4/reads/ccs/aligned/m84175_231021_222816_s4.bam\n",
"Loci:\n",
" - (\"chr15\", 23960193, 23963918)\n",
@@ -297,14 +264,14 @@
},
{
"cell_type": "code",
- "execution_count": 7,
+ "execution_count": 13,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
- "-rw-r--r-- 1 kiran staff 5.5K Jan 15 11:10 /var/folders/3b/jc95zqdn22jfb2lnfy4k_p580000gn/T/chr15_23960193_23963918.parquet\n"
+ "-rw-r--r-- 1 kiran staff 16K Jan 15 16:37 /var/folders/3b/jc95zqdn22jfb2lnfy4k_p580000gn/T/chr15_23960193_23963918.parquet\n"
]
}
],
@@ -314,7 +281,16 @@
},
{
"cell_type": "code",
- "execution_count": 10,
+ "execution_count": 11,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "#!rm /var/folders/3b/jc95zqdn22jfb2lnfy4k_p580000gn/T/chr15_23960193_23963918.parquet"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 14,
"metadata": {},
"outputs": [
{
@@ -333,7 +309,7 @@
" 'column_width']"
]
},
- "execution_count": 10,
+ "execution_count": 14,
"metadata": {},
"output_type": "execute_result"
}
@@ -347,78 +323,1022 @@
},
{
"cell_type": "code",
- "execution_count": 9,
+ "execution_count": 15,
"metadata": {},
"outputs": [
{
"data": {
- "text/html": [
- "\n",
- "
shape: (343, 10)| reference_contig | reference_start | reference_end | is_forward | query_name | read_group | sample_name | element_type | sequence | column_width |
|---|
| str | u32 | u32 | bool | str | str | str | u8 | str | u32 |
| "chr15" | 23941696 | 23960621 | true | "m84175_231021_… | "0a758968/17--1… | "CP-102020-50" | 0 | "" | 1 |
| "chr15" | 23941755 | 23941756 | true | "m84175_231021_… | "0a758968/17--1… | "CP-102020-50" | 2 | "T" | 1 |
| "chr15" | 23941807 | 23941808 | true | "m84175_231021_… | "0a758968/17--1… | "CP-102020-50" | 2 | "G" | 1 |
| "chr15" | 23942168 | 23942169 | true | "m84175_231021_… | "0a758968/17--1… | "CP-102020-50" | 2 | "T" | 1 |
| "chr15" | 23942237 | 23942238 | true | "m84175_231021_… | "0a758968/17--1… | "CP-102020-50" | 3 | "" | 1 |
| "chr15" | 23942383 | 23942384 | true | "m84175_231021_… | "0a758968/17--1… | "CP-102020-50" | 1 | "T" | 1 |
| "chr15" | 23942619 | 23942620 | true | "m84175_231021_… | "0a758968/17--1… | "CP-102020-50" | 1 | "T" | 1 |
| "chr15" | 23943295 | 23943296 | true | "m84175_231021_… | "0a758968/17--1… | "CP-102020-50" | 3 | "" | 1 |
| "chr15" | 23943300 | 23943301 | true | "m84175_231021_… | "0a758968/17--1… | "CP-102020-50" | 1 | "T" | 1 |
| "chr15" | 23943592 | 23943593 | true | "m84175_231021_… | "0a758968/17--1… | "CP-102020-50" | 2 | "A" | 1 |
| "chr15" | 23943733 | 23943734 | true | "m84175_231021_… | "0a758968/17--1… | "CP-102020-50" | 2 | "C" | 1 |
| "chr15" | 23943769 | 23943770 | true | "m84175_231021_… | "0a758968/17--1… | "CP-102020-50" | 2 | "T" | 1 |
| … | … | … | … | … | … | … | … | … | … |
| "chr15" | 23974554 | 23974555 | true | "m84175_231021_… | "0a758968/17--1… | "CP-102020-50" | 1 | "C" | 1 |
| "chr15" | 23974655 | 23974656 | true | "m84175_231021_… | "0a758968/17--1… | "CP-102020-50" | 1 | "C" | 1 |
| "chr15" | 23974726 | 23974727 | true | "m84175_231021_… | "0a758968/17--1… | "CP-102020-50" | 1 | "C" | 1 |
| "chr15" | 23975068 | 23975069 | true | "m84175_231021_… | "0a758968/17--1… | "CP-102020-50" | 3 | "" | 1 |
| "chr15" | 23975368 | 23975369 | true | "m84175_231021_… | "0a758968/17--1… | "CP-102020-50" | 1 | "C" | 1 |
| "chr15" | 23975420 | 23975421 | true | "m84175_231021_… | "0a758968/17--1… | "CP-102020-50" | 1 | "G" | 1 |
| "chr15" | 23975671 | 23975672 | true | "m84175_231021_… | "0a758968/17--1… | "CP-102020-50" | 1 | "G" | 1 |
| "chr15" | 23975679 | 23975680 | true | "m84175_231021_… | "0a758968/17--1… | "CP-102020-50" | 1 | "A" | 1 |
| "chr15" | 23975939 | 23975940 | true | "m84175_231021_… | "0a758968/17--1… | "CP-102020-50" | 1 | "C" | 1 |
| "chr15" | 23976000 | 23976001 | true | "m84175_231021_… | "0a758968/17--1… | "CP-102020-50" | 1 | "A" | 1 |
| "chr15" | 23976004 | 23976005 | true | "m84175_231021_… | "0a758968/17--1… | "CP-102020-50" | 1 | "A" | 1 |
| "chr15" | 23976030 | 23976031 | true | "m84175_231021_… | "0a758968/17--1… | "CP-102020-50" | 1 | "T" | 1 |
\n",
- "
shape: (343, 10)| reference_contig | reference_start | reference_end | is_forward | query_name | read_group | sample_name | element_type | sequence | column_width |
|---|
| str | u32 | u32 | bool | str | str | str | u8 | str | u32 |
| "chr15" | 23941696 | 23960621 | true | "m84175_231021_… | "0a758968/17--1… | "CP-102020-50" | 0 | "" | 1 |
| "chr15" | 23941755 | 23941756 | true | "m84175_231021_… | "0a758968/17--1… | "CP-102020-50" | 2 | "T" | 1 |
| "chr15" | 23941807 | 23941808 | true | "m84175_231021_… | "0a758968/17--1… | "CP-102020-50" | 2 | "G" | 1 |
| "chr15" | 23942168 | 23942169 | true | "m84175_231021_… | "0a758968/17--1… | "CP-102020-50" | 2 | "T" | 1 |
| "chr15" | 23942237 | 23942238 | true | "m84175_231021_… | "0a758968/17--1… | "CP-102020-50" | 3 | "" | 1 |
| "chr15" | 23942383 | 23942384 | true | "m84175_231021_… | "0a758968/17--1… | "CP-102020-50" | 1 | "T" | 1 |
| "chr15" | 23942619 | 23942620 | true | "m84175_231021_… | "0a758968/17--1… | "CP-102020-50" | 1 | "T" | 1 |
| "chr15" | 23943295 | 23943296 | true | "m84175_231021_… | "0a758968/17--1… | "CP-102020-50" | 3 | "" | 1 |
| "chr15" | 23943300 | 23943301 | true | "m84175_231021_… | "0a758968/17--1… | "CP-102020-50" | 1 | "T" | 1 |
| "chr15" | 23943592 | 23943593 | true | "m84175_231021_… | "0a758968/17--1… | "CP-102020-50" | 2 | "A" | 1 |
| "chr15" | 23943733 | 23943734 | true | "m84175_231021_… | "0a758968/17--1… | "CP-102020-50" | 2 | "C" | 1 |
| "chr15" | 23943769 | 23943770 | true | "m84175_231021_… | "0a758968/17--1… | "CP-102020-50" | 2 | "T" | 1 |
| … | … | … | … | … | … | … | … | … | … |
| "chr15" | 23974554 | 23974555 | true | "m84175_231021_… | "0a758968/17--1… | "CP-102020-50" | 1 | "C" | 1 |
| "chr15" | 23974655 | 23974656 | true | "m84175_231021_… | "0a758968/17--1… | "CP-102020-50" | 1 | "C" | 1 |
| "chr15" | 23974726 | 23974727 | true | "m84175_231021_… | "0a758968/17--1… | "CP-102020-50" | 1 | "C" | 1 |
| "chr15" | 23975068 | 23975069 | true | "m84175_231021_… | "0a758968/17--1… | "CP-102020-50" | 3 | "" | 1 |
| "chr15" | 23975368 | 23975369 | true | "m84175_231021_… | "0a758968/17--1… | "CP-102020-50" | 1 | "C" | 1 |
| "chr15" | 23975420 | 23975421 | true | "m84175_231021_… | "0a758968/17--1… | "CP-102020-50" | 1 | "G" | 1 |
| "chr15" | 23975671 | 23975672 | true | "m84175_231021_… | "0a758968/17--1… | "CP-102020-50" | 1 | "G" | 1 |
| "chr15" | 23975679 | 23975680 | true | "m84175_231021_… | "0a758968/17--1… | "CP-102020-50" | 1 | "A" | 1 |
| "chr15" | 23975939 | 23975940 | true | "m84175_231021_… | "0a758968/17--1… | "CP-102020-50" | 1 | "C" | 1 |
| "chr15" | 23976000 | 23976001 | true | "m84175_231021_… | "0a758968/17--1… | "CP-102020-50" | 1 | "A" | 1 |
| "chr15" | 23976004 | 23976005 | true | "m84175_231021_… | "0a758968/17--1… | "CP-102020-50" | 1 | "A" | 1 |
| "chr15" | 23976030 | 23976031 | true | "m84175_231021_… | "0a758968/17--1… | "CP-102020-50" | 1 | "T" | 1 |