Skip to content

Commit 78d0311

Browse files
committedSep 26, 2019
wip
1 parent a5dafac commit 78d0311

File tree

8 files changed

+116
-88
lines changed

8 files changed

+116
-88
lines changed
 

‎CITATION.txt

+1-1
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ To cite Neo in publications, please use:
77

88
A BibTeX entry for LaTeX users is::
99

10-
@article{neo09,
10+
@article{neo14,
1111
author = {Garcia S. and Guarino D. and Jaillet F. and Jennings T.R. and Pröpper R. and
1212
Rautenberg P.L. and Rodgers C. and Sobolev A. and Wachtler T. and Yger P.
1313
and Davison A.P.},

‎doc/source/developers_guide.rst

+8-7
Original file line numberDiff line numberDiff line change
@@ -39,14 +39,15 @@ Requirements
3939
------------
4040

4141
* Python_ 2.7, 3.4 or later
42-
* numpy_ >= 1.7.1
43-
* quantities_ >= 0.9.0
44-
* nose_ >= 0.11.1 (for running tests)
45-
* Sphinx_ >= 0.6.4 (for building documentation)
46-
* (optional) tox_ >= 0.9 (makes it easier to test with multiple Python versions)
42+
* numpy_ >= 1.10.0
43+
* quantities_ >= 0.12.1
44+
* nose_ >= 1.1.2 (for running tests)
45+
* Sphinx_ (for building documentation)
4746
* (optional) coverage_ >= 2.85 (for measuring test coverage)
4847
* (optional) scipy >= 0.12 (for MatlabIO)
4948
* (optional) h5py >= 2.5 (for KwikIO, NeoHdf5IO)
49+
* (optional) nixio (for NixIO)
50+
* (optional) pillow (for TiffIO)
5051

5152
We strongly recommend you develop within a virtual environment (from virtualenv, venv or conda).
5253
It is best to have at least one virtual environment with Python 2.7 and one with Python 3.x.
@@ -63,7 +64,7 @@ To get a local copy of the repository::
6364

6465
$ cd /some/directory
6566
$ git clone git@github.com:<username>/python-neo.git
66-
67+
6768
Now you need to make sure that the ``neo`` package is on your PYTHONPATH.
6869
You can do this either by installing Neo::
6970

@@ -263,7 +264,7 @@ Michael Denker and Julia Sprenger have the necessary permissions to do this)::
263264

264265
.. talk about readthedocs
265266
266-
267+
267268
268269
.. make a release branch
269270

‎doc/source/index.rst

+5-5
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22

33
.. image:: images/neologo.png
44
:width: 600 px
5-
5+
66
Neo is a Python package for working with electrophysiology data in Python, together
77
with support for reading a wide range of neurophysiology file formats, including
88
Spike2, NeuroExplorer, AlphaOmega, Axon, Blackrock, Plexon, Tdt, Igor Pro, and support for
@@ -14,7 +14,7 @@ shared object model. In order to be as lightweight a dependency as possible,
1414
Neo is deliberately limited to represention of data, with no functions for data
1515
analysis or visualization.
1616

17-
Neo is used by a number of other software tools, including
17+
Neo is used by a number of other software tools, including
1818
SpykeViewer_ (data analysis and visualization), Elephant_ (data analysis),
1919
the G-node_ suite (databasing), PyNN_ (simulations), tridesclous_ (spike sorting)
2020
and ephyviewer_ (data visualization).
@@ -36,7 +36,7 @@ Documentation
3636

3737
.. toctree::
3838
:maxdepth: 1
39-
39+
4040
install
4141
core
4242
usecases
@@ -83,10 +83,10 @@ Citation
8383
.. _G-node: http://www.g-node.org/
8484
.. _Neuroshare: http://neuroshare.org/
8585
.. _SpykeViewer: https://spyke-viewer.readthedocs.org/en/latest/
86-
.. _NiBabel: http://nipy.sourceforge.net/nibabel/
86+
.. _NiBabel: https://nipy.org/nibabel/
8787
.. _PyNN: http://neuralensemble.org/PyNN
8888
.. _quantities: http://pypi.python.org/pypi/quantities
89-
.. _`NeuralEnsemble mailing list`: http://groups.google.com/group/neuralensemble
89+
.. _`NeuralEnsemble mailing list`: https://groups.google.com/forum/#!forum/neuralensemble
9090
.. _`issue tracker`: https://github.com/NeuralEnsemble/python-neo/issues
9191
.. _tridesclous: https://github.com/tridesclous/tridesclous
9292
.. _ephyviewer: https://github.com/NeuralEnsemble/ephyviewer

‎doc/source/install.rst

+7-7
Original file line numberDiff line numberDiff line change
@@ -7,9 +7,9 @@ system.
77

88
Dependencies
99
============
10-
10+
1111
* Python_ >= 2.7
12-
* numpy_ >= 1.7.1
12+
* numpy_ >= 1.10.0
1313
* quantities_ >= 0.12.1
1414

1515
For Debian/Ubuntu, you can install these using::
@@ -40,20 +40,20 @@ Installing from the Python Package Index
4040
If you have pip_ installed::
4141

4242
$ pip install neo
43-
43+
4444
This will automatically download and install the latest release (again
4545
you may need to have administrator privileges on the machine you are installing
4646
on).
47-
47+
4848
To download and install manually, download:
4949

5050
|neo_github_url|
51-
51+
5252

5353
Then:
5454

5555
.. parsed-literal::
56-
56+
5757
$ unzip neo-|release|.zip
5858
$ cd neo-|release|
5959
$ python setup.py install
@@ -63,7 +63,7 @@ Then:
6363
or::
6464

6565
$ python3 setup.py install
66-
66+
6767
depending on which version of Python you are using.
6868

6969

‎doc/source/io.rst

+37-40
Original file line numberDiff line numberDiff line change
@@ -59,7 +59,7 @@ you can replace :class:`MyFormatIO` by any implemented class, see :ref:`list_of_
5959
Modes
6060
======
6161

62-
IO can be based on a single file, a directory containing files, or a database.
62+
An IO module can be based on a single file, a directory containing files, or a database.
6363
This is described in the :attr:`mode` attribute of the IO class.
6464

6565
>>> from neo.io import MyFormatIO
@@ -108,84 +108,81 @@ All IOs have a read() method that returns a list of :class:`Block` objects (repr
108108
>>> print bl[0].segments[0]
109109
neo.core.Segment
110110

111-
111+
112112
Read a time slice of Segment
113113
============================
114114

115-
Some object support ``time_slice`` arguments in ``read_segment()``.
116-
This is usefull to read only a subset of a dataset clipped in time.
115+
Some objects support the ``time_slice`` argument in ``read_segment()``.
116+
This is useful to read only a subset of a dataset clipped in time.
117117
By default ``time_slice=None`` meaning load eveything.
118118

119-
This read everything::
119+
This reads everything::
120120

121-
seg = reader.load_segment(time_slice=None)
122-
shape0 = seg.analogsignals[0].shape # total shape
123-
shape1 = seg.spiketrains[0].shape # total shape
121+
seg = reader.read_segment(time_slice=None)
124122

125-
126-
This read only the first 5 seconds::
123+
This reads only the first 5 seconds::
127124

128-
seg = reader.load_segment(time_slice=(0*pq.s, 5.*pq.s))
129-
shape0 = seg.analogsignals[0].shape # more small shape
130-
shape1 = seg.spiketrains[0].shape # more small shape
125+
seg = reader.read_segment(time_slice=(0*pq.s, 5.*pq.s))
131126

132127

133-
Lazy option and proxy object
134-
============================
128+
.. _section-lazy:
129+
130+
Lazy option and proxy objects
131+
=============================
135132

136133
In some cases you may not want to load everything in memory because it could be too big.
137-
For this scenario, some IOs implement ``lazy=True/False``.
138-
Since neo 0.7, a new lazy sytem have been added for some IOs (all IOs that inherits rawio).
134+
For this scenario, some IOs implement ``lazy=True/False``.
135+
Since neo 0.7, a new lazy sytem have been added for some IO modules (all IO classes that inherit from rawio).
139136
To know if a class supports lazy mode use ``ClassIO.support_lazy``.
140137

141-
With ``lazy=True`` all data object (AnalogSignal/SpikeTrain/Event/Epoch) are replace by
142-
ProxyObjects (AnalogSignalProxy/SpikeTrainProxy/EventProxy/EpochProxy).
143-
138+
With ``lazy=True`` all data objects (AnalogSignal/SpikeTrain/Event/Epoch) are replaced by
139+
proxy objects (AnalogSignalProxy/SpikeTrainProxy/EventProxy/EpochProxy).
144140

145141
By default (if not specified), ``lazy=False``, i.e. all data is loaded.
146142

147-
Theses ProxyObjects contain metadata (name, sampling_rate, id, ...) so they can be inspected
143+
These proxy objects contain metadata (name, sampling_rate, id, ...) so they can be inspected
148144
but they do not contain any array-like data.
149-
All ProxyObjects contain a ``load()`` method to postpone the real load of array like data.
145+
All proxy objects contain a ``load()`` method to postpone the real load of array like data.
150146

151-
Further more the ``load()`` method have a ``time_slice`` arguments to load only a slice
152-
from the file. So the consumption of memory can be finely controlled.
147+
Further more the ``load()`` method has a ``time_slice`` argument to load only a slice
148+
from the file. In this way the consumption of memory can be finely controlled.
153149

154150

155-
Here 2 examples that do read a dataset, load triggers a do trigger averaging on signals.
151+
Here are two examples that read a dataset, extract sections of the signal based on recorded events,
152+
and averages the sections.
156153

157-
The first example is without lazy, so it consume more memory::
158-
159-
lim0, lim1 = -500*pq.ms, +1500*pq.ms
154+
The first example is without lazy mode, so it consumes more memory::
155+
156+
lim0, lim1 = -500 * pq.ms, +1500 * pq.ms
160157
seg = reader.read_segment(lazy=False)
161158
triggers = seg.events[0]
162-
anasig = seg.analogsignals[0] # here anasig contain the whole recording in memory
159+
sig = seg.analogsignals[0] # here sig contain the whole recording in memory
163160
all_sig_chunks = []
164161
for t in triggers.times:
165162
t0, t1 = (t + lim0), (t + lim1)
166-
anasig_chunk = anasig.time_slice(t0, t1)
167-
all_sig_chunks.append(anasig_chunk)
163+
sig_chunk = sig.time_slice(t0, t1)
164+
all_sig_chunks.append(sig_chunk)
168165
apply_my_fancy_average(all_sig_chunks)
169-
170-
The second example use lazy so it consume fewer memory::
166+
167+
The second example uses lazy mode, so it consumes less memory::
171168

172169
lim0, lim1 = -500*pq.ms, +1500*pq.ms
173170
seg = reader.read_segment(lazy=True)
174-
triggers = seg.events[0].load(time_slice=None) # this load all trigers in memory
175-
anasigproxy = seg.analogsignals[0] # this is a proxy
171+
triggers = seg.events[0].load(time_slice=None) # this loads all triggers in memory
172+
sigproxy = seg.analogsignals[0] # this is a proxy
176173
all_sig_chunks = []
177174
for t in triggers.times:
178175
t0, t1 = (t + lim0), (t + lim1)
179-
anasig_chunk = anasigproxy.load(time_slice=(t0, t1)) # here real data are loaded
180-
all_sig_chunks.append(anasig_chunk)
176+
sig_chunk = sigproxy.load(time_slice=(t0, t1)) # here real data are loaded
177+
all_sig_chunks.append(sig_chunk)
181178
apply_my_fancy_average(all_sig_chunks)
182179

183-
In addition to ``time_slice`` AnalogSignalProxy contains ``channel_indexes`` arguments.
184-
This control also slicing in channel. This is usefull in case the channel count is very high.
180+
In addition to ``time_slice``, AnalogSignalProxy supports the ``channel_indexes`` argument.
181+
This allows loading only a subset of channels. This is useful where the channel count is very high.
185182

186183
.. TODO: add something about magnitude mode when implemented for all objects.
187184
188-
In this example, we read only 3 selected channels::
185+
In this example, we read only three selected channels::
189186

190187
seg = reader.read_segment(lazy=True)
191188
anasig = seg.analogsignals[0].load(time_slice=None, channel_indexes=[0, 2, 18])

‎doc/source/releases/0.8.0.rst

+54-24
Original file line numberDiff line numberDiff line change
@@ -2,20 +2,49 @@
22
Neo 0.8.0 release notes
33
=======================
44

5-
XXth July 2019
5+
27th September 2019
6+
7+
Lazy loading
8+
------------
9+
10+
Neo 0.8 sees a major new feature, the ability to selectively load only parts of a data file
11+
(for supported file formats) into memory, for example only a subset of the signals
12+
in a segment, a subset of the channels in a signal, or even only a certain time slice of a given signal.
13+
14+
This can lead to major savings in time and memory consumption, or can allow files that are too
15+
large to be loaded into memory in their entirety to be processed one section at a time.
16+
17+
Here is an example, loading only certain sections of a signal::
18+
19+
lim0, lim1 = -500*pq.ms, +1500*pq.ms
20+
seg = reader.read_segment(lazy=True) # this loads only the segment structure and metadata
21+
# but all data objects are replaced by proxies
22+
triggers = seg.events[0].load() # this loads all triggers in memory
23+
sigproxy = seg.analogsignals[0] # this is a proxy object
24+
all_sig_chunks = []
25+
for t in triggers.times:
26+
t0, t1 = (t + lim0), (t + lim1)
27+
sig_chunk = sigproxy.load(time_slice=(t0, t1)) # here the actual data are loaded
28+
all_sig_chunks.append(sig_chunk)
29+
30+
Not all IO modules support lazy loading (but many do). To know whether a given IO class supports lazy mode,
31+
use ``SomeIO.support_lazy``.
32+
33+
For more details, see :ref:`section-lazy`.
34+
35+
Other new features
36+
------------------
637

7-
Main added features:
8-
* Proxy object implementation and new lazy mode
938
* new neo.utils module
10-
11-
Other features:
12-
* Event and Epoch do not use array_annotations machinery
13-
* deep copy annotations when slicing
14-
* numpy 1.16 compatibility
15-
* time_shift() method for Epoch/Event/AnalogSignal
16-
* time_slice more robust
17-
18-
Many bug fixes and improvements in IO:
39+
* Numpy 1.16+ compatibility
40+
* :meth:`time_shift()` method for :class:`Epoch`/:class:`Event`/:class:`AnalogSignal`
41+
* :meth:`time_slice()` method is now more robust
42+
43+
See all `pull requests`_ included in this release and the `list of closed issues`_.
44+
45+
Bug fixes and improvements in IO modules
46+
----------------------------------------
47+
1948
* Blackrock
2049
* Neuroshare
2150
* NixIOFr
@@ -25,21 +54,22 @@ Many bug fixes and improvements in IO:
2554
* Brainvision
2655
* NeuralynxIO
2756

28-
Warning:
29-
* Some IOs (base on rawio) when loading can choose to split each
30-
channel into separted AnalogSignal or to group then.
31-
The default behavior (either signal_group_mode='split-all'
32-
or 'group-same-units') was not the same for all IOs for retro
33-
compatibility reasons. In next release, all IOs will be
34-
signal_group_mode='group-same-units'
35-
* last release supporting python2.7
57+
.. Warning:: Some IOs (based on rawio) when loading can choose to split each
58+
channel into its own 1-channel :class:`AnalogSignal` or to group them
59+
in a multi-channel :class:`AnalogSignal`.
60+
The default behavior (either ``signal_group_mode='split-all'``
61+
or ``'group-same-units'``) is not the same for all IOs for backwards
62+
compatibility reasons. In the next release, all IOs will have the default
63+
``signal_group_mode='group-same-units'``
3664

37-
See all [github PR](https://github.com/NeuralEnsemble/python-neo/pulls?q=is%3Apr+is%3Aclosed+merged%3A%3E2018-11-27+milestone%3A0.8.0)
38-
inclued in this release.
65+
Acknowledgements
66+
----------------
3967

40-
Thanks to Achileas Koutsou, Chek Yin Choi, Richard C Gerkin,
68+
Thanks to Achileas Koutsou, Chek Yin Choi, Richard C. Gerkin,
4169
Alexander Kleinjohann, Björn Müller, Jeffrey Gill, Christian Kothe,
42-
Mike Sintsov, rishidhingra, Michael Denker, Julia Sprenger,
70+
Mike Sintsov, @rishidhingra, Michael Denker, Julia Sprenger, Corentin Fragnaud,
4371
Andrew Davison and Samuel Garcia for their contributions to this release.
4472

4573

74+
.. _`list of closed issues`: https://github.com/NeuralEnsemble/python-neo/issues?q=is%3Aissue+milestone%3A0.8.0+is%3Aclosed
75+
.. _`pull requests`: https://github.com/NeuralEnsemble/python-neo/pulls?q=is%3Apr+is%3Aclosed+merged%3A%3E2018-11-27+milestone%3A0.8.0

‎neo/io/basefromrawio.py

+2-2
Original file line numberDiff line numberDiff line change
@@ -210,7 +210,7 @@ def read_segment(self, block_index=0, seg_index=0, lazy=False,
210210
signal_group_mode=None, load_waveforms=False, time_slice=None,
211211
strict_slicing=True):
212212
"""
213-
:param block_index: int default 0. In case of several block block_index can be specified.
213+
:param block_index: int default 0. In case of several blocks block_index can be specified.
214214
215215
:param seg_index: int default 0. Index of segment.
216216
@@ -229,7 +229,7 @@ def read_segment(self, block_index=0, seg_index=0, lazy=False,
229229
All object AnalogSignal, SpikeTrain, Event, Epoch will load only in the slice.
230230
231231
:param strict_slicing: True by default.
232-
Control if an error is raise or not when one of time_slice member (t_start or t_stop)
232+
Control if an error is raised or not when t_start or t_stop
233233
is outside the real time range of the segment.
234234
"""
235235

‎requirements.txt

+2-2
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
# essential
2-
numpy>=1.7.1,!=1.16.0
3-
quantities>=0.9.0
2+
numpy>=1.10.0,!=1.16.0
3+
quantities>=0.12.1
44
# for running tests
55
nose>=1.1.2

0 commit comments

Comments
 (0)
Please sign in to comment.