Merge branch 'main' of https://github.com/AllenInstitute/openscope_da…

…tabook

docs/_toc.yml

@@ -29,6 +29,7 @@ parts:
 - caption: Higher-Order Analysis
   chapters:
   - file: higher-order/cebra_time.ipynb
+  - file: higher-order/tca.ipynb
 - caption: Replicating Figures
   chapters:
   - file: replication/cred_assign_figures.ipynb

docs/basics/background.md

@@ -1,5 +1,11 @@
 # Background
 
+### OpenScope
+
+[OpenScope](https://openscope.ai) freely opens the Allen Brain Observatory pipeline to the neuroscience community, enabling theoretical, computational, and experimental scientists to test sophisticated hypotheses on brain function in a process analogous to astronomical observatories that survey the night sky. Once a year, OpenScope will accept experimental proposals from external scientists, which will be reviewed by a panel of leading experts from the international community for their feasibility and scientific merit. The Allen Institute will carry out the selected in vivo experiments in mice brains following verified, reproducible, and open protocols for in vivo single- and multi-area two photon calcium imaging and Neuropixels electrophysiology, making the data openly available to these scientists and to the community. This will lower barriers to testing new hypotheses about brain function, bring new computational and theoretical talents into the field, and enhance the reproducibility of results in brain research, thereby accelerating progress toward an integrated understanding of neural activity in health and disease.
+
+[Apply Here!](https://alleninstitute.org/division/neural-dynamics/openscope/)
+
 ### DANDI
 At the Allen Institute, we frequently utilize a platform called [DANDI](https://dandiarchive.org/) (Data Archive and Neurophysiology Imaging). DANDI is a platform that allows open-source data sharing and archiving and acts as a centralized repository where researchers can deposit data. While some of these notebooks use pre-loaded data from DANDI, the ultimate purpose of this Databook is to teach users to take any dataset off DANDI and reproduce the analysis within these notebooks.
 

docs/basics/download_nwb.ipynb

@@ -6,7 +6,7 @@
    "metadata": {},
    "source": [
     "# Downloading an NWB File\n",
-    "In order to analyze some data, you'll need to have some data. The [DANDI Archive](https://dandiarchive.org/) is used to store [NWB](https://www.nwb.org/) files in datasets called **dandisets**. Typically an NWB file contains the data for just one experimental session, while a dandiset contains all the related data files yielded from a project. This notebook allows you to download from public dandisets or private dandisets (called **embargoed** dandisets) via the [DANDI Python API](https://dandi.readthedocs.io/en/latest/modref/index.html). To download embargoed dandisets from DANDI, you will need to make an account on the DANDI Archive and must be given access by the owner of the dandiset."
+    "In order to analyze some data, you'll need to have some data. The [DANDI Archive](https://dandiarchive.org/) is used to store [NWB](https://www.nwb.org/) files in datasets called **dandisets** {cite}`Rubel2022`. Typically an NWB file contains the data for just one experimental session, while a dandiset contains all the related data files yielded from a project. This notebook allows you to download from public dandisets or private dandisets (called **embargoed** dandisets) via the [DANDI Python API](https://dandi.readthedocs.io/en/latest/modref/index.html). To download embargoed dandisets from DANDI, you will need to make an account on the DANDI Archive and must be given access by the owner of the dandiset."
    ]
   },
   {

docs/basics/read_nwb.ipynb

@@ -498,6 +498,7 @@
     }
    ],
    "source": [
+    "# takes relative or absolute filepath as first argument\n",
     "io = NWBHDF5IO(f\"{download_loc}/{filename}\", mode=\"r\", load_namespaces=True)\n",
     "nwb = io.read()\n",
     "nwb"

docs/embargoed/cell_matching.ipynb

@@ -152,7 +152,7 @@
    "metadata": {},
    "source": [
     "### Generating Cell-Matching Input File\n",
-    "The Allen Institute has created a robust pipeline for conducting cell matching between 2 or more sessions called [**Nway Matching**](https://github.com/AllenInstitute/ophys_nway_matching). For those who are inquisitive, the general algorithm is described in page 31 of [{cite}`Garrett2023`](https://www.biorxiv.org/content/10.1101/2023.02.14.528085v2.full.pdf). In summary, the input images are aligned together with a euclidian image registration technique. Then the given input ROI masks are used with the [Blossom Algorithm](https://en.wikipedia.org/wiki/Blossom_algorithm) as a bipartite graph matching algorithm. This identifies which ROIs represent the same cells with high probability.\n",
+    "The Allen Institute has created a robust pipeline for conducting cell matching between 2 or more sessions called [**Nway Matching**](https://github.com/AllenInstitute/ophys_nway_matching). For those who are inquisitive, the general algorithm is described in page 31 of {cite}`Garrett2023`. In summary, the input images are aligned together with a euclidian image registration technique. Then the given input ROI masks are used with the [Blossom Algorithm](https://en.wikipedia.org/wiki/Blossom_algorithm) as a bipartite graph matching algorithm. This identifies which ROIs represent the same cells with high probability.\n",
     "\n",
     "The caveat to using this pipeline is that it requires input in a specific JSON format rather than an NWB File. The following functions are used to generate a file, `input.json`, which can be used to obtain important output. The input json contains an entry for each input experiment. 2 or more experiments can be used. The first experiment is called *fixed* and the other ones are *moving*. For each experiment, a unique experiment ID, a path to the experiment projection image, and a list of objects for each ROI are required."
    ]

docs/embargoed/test_2p_responses_embargoed.ipynb

@@ -7,7 +7,7 @@
    "metadata": {},
    "source": [
     "# Statistically Testing 2P Responses to Stimulus\n",
-    "In most analyses, some form of *inclusion criteria* is used to select neurons that are \"responsive\" to the stimulus conditions presented. There are no universally agreed upon inclusion criteria for this type of selection. In [{cite}`Mesa2021`](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8114876/), it is demonstrated that the choice of inclusion criteria can dramatically affect what neurons are selected as responsive. This notebook does a similar demonstration, using the same five inclusion criteria on Ophys dF/F recordings to select responsive neurons from one experimental session. It can be seen that very different selections are made depending on the criteria used. This also underscores how different critera might be more or less appropriate for the type of stimulus and the type of measurements being used for analysis. We are sharing these so that this comparison can be reproduced on data and so results can more effectively use this choice to make their results more robust. For more information about dF/F data, see [Visualizing 2P Responses to Stimulus](../visualization/visualize_2p_responses.ipynb)."
+    "In most analyses, some form of *inclusion criteria* is used to select neurons that are \"responsive\" to the stimulus conditions presented. There are no universally agreed upon inclusion criteria for this type of selection. In {cite}`Mesa2021`, it is demonstrated that the choice of inclusion criteria can dramatically affect what neurons are selected as responsive. This notebook does a similar demonstration, using the same five inclusion criteria on Ophys dF/F recordings to select responsive neurons from one experimental session. It can be seen that very different selections are made depending on the criteria used. This also underscores how different critera might be more or less appropriate for the type of stimulus and the type of measurements being used for analysis. We are sharing these so that this comparison can be reproduced on data and so results can more effectively use this choice to make their results more robust. For more information about dF/F data, see [Visualizing 2P Responses to Stimulus](../visualization/visualize_2p_responses.ipynb)."
    ]
   },
   {
@@ -974,7 +974,7 @@
     "#### Inclusion Criterion 1\n",
     "The first criterion is one of the simplest. This one is easy to alter and has a medium level of permissiveness. Below is shown a distribution of the ROI's maximum mean evoked response. That is, the maximum dF/F value achieved during the evoked averaged response of an ROI to all stimulus trials. A line is drawn to show where the selection is made within the distribution. Also below are the neuronwise response windows of some of the selected neurons. It can be seen that these neurons appear more responsive than the (pseudo) random sample shown above.\n",
     "\n",
-    "*The maximum value of the mean evoked response is >10%* [{cite}`Sun2016`](https://rdcu.be/dbFb9)"
+    "*The maximum value of the mean evoked response is >10%* {cite}`Sun2016`"
    ]
   },
   {
@@ -1071,7 +1071,7 @@
     "#### Inclusion Criterion 2\n",
     "This criterion is slightly more complicated, and accounts for the fact that neurons might behave inconsistently. It uses the maximum values of the evoked response, and compares it to an absolue value (5%) as well as a value relative to its own baseline activity (3x the standard deviation). It is difficult to show the exact metric being used as a distribution, but to assist in interpretation, a distribution is shown below of the the mean ratio between the maximum evoked value and the standard distribution of the baseline behavior. Note that, this is not the exact metric used to select rois; as this criterion selects rois that exhibit significant activity in 50% or more trials. For our dataset, this criterion is rather stringent, only including 4 rois in the final selection.\n",
     "\n",
-    "*In 50% of trials, the evoked response is A) larger than 3x the SD of the baseline, and B) larger than 5% dF/F* [{cite}`Roth2012`](https://www.jneurosci.org/content/jneuro/32/28/9716.full.pdf)"
+    "*In 50% of trials, the evoked response is A) larger than 3x the SD of the baseline, and B) larger than 5% dF/F* {cite}`Roth2012`"
    ]
   },
   {
@@ -1179,7 +1179,7 @@
     "#### Inclusion Criterion 3\n",
     "This is perhaps the most sophisticated criterion in our list. For each ROI, It compares the distribution of responses between the baseline activity and the evoked activity to identify any statistically significant response. Below, an example is shown of these distributions for one ROI. Change `roi` below to view the comparison for other cells. Additionally, the distribution of p-values of all rois is shown.\n",
     "\n",
-    "*Paired T-test (p < 0.05) with Bonferroni correction, comparing the mean baseline to the mean evoked response* [{cite}`Andermann2011`](https://pubmed.ncbi.nlm.nih.gov/22196337/)"
+    "*Paired T-test (p < 0.05) with Bonferroni correction, comparing the mean baseline to the mean evoked response* {cite}`Andermann2011`"
    ]
   },
   {
@@ -1322,7 +1322,7 @@
     "#### Inclusion Criterion 4\n",
     "This criterion is the most stringent for our dataset. It relies on calculating the measure of `reliability` of a cell's activity. It can be seen that in this example, no ROIs are selected. This is probably due to no ROIs having reliable responses to the stimulus. Perhaps conducting more trials could improve the reliability.\n",
     "\n",
-    "*A) The mean response to any stimulus condition is is > 6% dF/F and B) reliability > 1* [{cite}`Marshel2012`](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3248795/)."
+    "*A) The mean response to any stimulus condition is is > 6% dF/F and B) reliability > 1* {cite}`Marshel2012`."
    ]
   },
   {
@@ -1419,7 +1419,7 @@
     "#### Inclusion Criterion 5\n",
     "Finally, the simplest criterion. This criterion is clearly the most permissive, as it includes all ROIs for this dataset.\n",
     "\n",
-    "*The maximum evoked response to any stimulus condition is > 4% dF/F* [{cite}`Tohmi2014`](https://pubmed.ncbi.nlm.nih.gov/24583013/)."
+    "*The maximum evoked response to any stimulus condition is > 4% dF/F* {cite}`Tohmi2014`."
    ]
   },
   {