Merge pull request #681 from rossant/kill-kwik-rebase3

Full rewrite

.coveragerc

@@ -4,3 +4,12 @@ source = phy
 omit =
     */phy/ext/*
     */phy/utils/tempdir.py
+    */default_settings.py
+
+[report]
+exclude_lines =
+    pragma: no cover
+    raise AssertionError
+    raise NotImplementedError
+    pass
+    return

.gitignore

@@ -1,18 +1,24 @@
+contrib
 data
 doc
+phy-doc
 docker
 experimental
 htmlcov
 format
 wiki
+.cache
 .idea
 .ipynb_checkpoints
 .*fuse*
 *.orig
+*.log*
 .eggs
+.profile
 __pycache__
-
+_old
 *.py[cod]
+*~
 .coverage*
 *credentials
 

.travis.yml

@@ -2,7 +2,6 @@ language: python
 sudo: false
 python:
   - "2.7"
-  - "3.4"
   - "3.5"
 before_install:
   - pip install codecov
@@ -19,12 +18,8 @@ install:
   - conda update -q conda
   - conda info -a
   # Create the environment.
-  - conda create -q -n testenv python=$TRAVIS_PYTHON_VERSION
-  - source activate testenv
-  - conda install pip numpy vispy matplotlib scipy h5py pyqt pyzmq ipython requests six
-  # NOTE: cython is only temporarily needed for building KK2.
-  # Once we have KK2 binary builds on binstar we can remove this dependency.
-  - conda install cython && pip install klustakwik2
+  - conda env create python=$TRAVIS_PYTHON_VERSION
+  - source activate phy
   # Dev requirements
   - pip install -r requirements-dev.txt
   - pip install -e .

CONTRIBUTE.md

@@ -7,37 +7,35 @@ Please read this entire document before contributing any code.
 On your development computer:
 
 * Use the very latest Anaconda release (`conda update conda anaconda`).
-* Use a special `phy` conda environment based on the latest Python 3.x (3.4 at the time of writing).
+* Use a special `phy` conda environment based on Python 3.5.
 * Have another `phy2` clone environment based on Python 2.7.
-* phy only supports Python 2.7, and Python 3.4+.
-* Use `six` for writing compatible code (see [the documentation here](http://pythonhosted.org/six/))
-* You need the following dependencies for development (not required for using phy): pytest, pip, flake8, coverage, coveralls.
-* For IPython, use the IPython git `master` branch (or version 3.0 when it will be released in early 2015).
+* Install the dev dependencies: `pip install -r requirements-dev.txt`
+* phy only supports Python 2.7 and Python 3.4+.
+* Use the `six` library for writing compatible code (see [the documentation here](http://pythonhosted.org/six/))
 
 A few rules:
 
 * Every module `phy/mypackage/mymodule.py` must come with a `phy/mypackage/tests/test_mymodule.py` test module that contains a bunch of `test_*()` functions.
 * Never import test modules in the main code.
-* Do not import packages from `phy/__init__.py`. Every subpackage `phy.stuff` will need to be imported explicitly by the user. Dependencies required by this subpackage will only be loaded when the subpackage is loaded. This ensures that users can use `phy.subpackageA` without having to install the dependencies required by `phy.subpackageB`.
-* phy's required dependencies are: numpy. Every subpackage can come with further dependencies. For example, `phy.io.kwik` depends on h5py.
+* In general, do not import packages from `phy/__init__.py`. Every subpackage `phy.stuff` will need to be imported explicitly by the user. Dependencies required by this subpackage will only be loaded when the subpackage is loaded. This ensures that users can use `phy.subpackageA` without having to install the dependencies required by `phy.subpackageB`.
+* phy's required dependencies are: pip, traitlets, click, numpy. Every subpackage may come with further dependencies.
 * You can experiment with ideas and prototypes in the `kwikteam/experimental` repo. Use a different folder for every experiment.
-* `kwikteam/phy` will only contain a `master` branch and release tags. There should be no experimental/debugging code in the entire repository.
+* `kwikteam/phy` will only contain a `master` branch, release tags, and possibly one refactoring branch. There should be no experimental/debugging code in the entire repository.
 
 
 ### GitHub flow
 
-* Work through PRs from `yourfork/specialbranch` against `phy/master` exclusively.
+* Work through PRs from `yourfork/specialbranch` against `phy/master` or `phy/kill-kwik` exclusively.
 * Set `upstream` to `kwikteam/phy` and `origin` to your fork.
 * When master and your PR's branch are out of sync, [rebase your branch in your fork](https://groups.google.com/forum/#!msg/vispy-dev/q-UNjxburGA/wYNkZRXiySwJ).
 * Two-pairs-of-eyes rule: every line of code needs to be reviewed by 2 people, including the author.
 * Never merge your own PR to the main phy repository, unless in exceptional circumstances.
 * A PR is assumed to be **not ready for merge** unless explicitly stated otherwise.
 * Always run `make test` before stating that a PR is ready to merge (and ideally before pushing on your PR's branch).
-* We try to have a code coverage close to 100%: always test all features you implement, and verify through code coverage that all lines are covered by your tests.
+* We try to have a code coverage close to 100%: always test all features you implement, and verify through code coverage that all lines are covered by your tests. Use `#pragma: no cover` comments for lines that don't absolutely need to be covered (for example, rare exception-raising code).
 * Always wait for Travis to be green before merging.
 * `phy/master` should always be stable and deployable.
 * Use [semantic versioning](http://www.semver.org) for stable releases.
-* Do not make too many releases until the software is mature enough. Early adopters can work directly off `master`.
 * We follow almost all [PEP8 rules](https://www.python.org/dev/peps/pep-0008/), except [for a few exceptions](https://github.com/kwikteam/phy/blob/master/Makefile#L24).
 
 
@@ -55,10 +53,10 @@ A few rules:
 
 Make sure your text editor is configured to:
 
-* automatically clean blank lines (i.e. no lines containing only whitespace)
-* use four spaces per indent level, and **never** tab indents
-* enforce an empty blank line at the end of every text file
-* display a vertical ruler at 79 characters (length limit of every line)
+* Automatically clean blank lines (i.e. no lines containing only whitespace)
+* Use four spaces per indent level, and **never** tab indents
+* Enforce an empty blank line at the end of every text file
+* Display a vertical ruler at 79 characters (length limit of every line)
 
 Below is a settings script for the popular text editor [Sublime](http://www.sublimetext.com) which you can put into your ```Preferences -> Settings (User)```:
 

MANIFEST.in

@@ -4,7 +4,8 @@ include README.md
 recursive-include tests *
 recursive-include phy/electrode/probes *.prb
 recursive-include phy/plot/glsl *.vert *.frag *.glsl
+recursive-include phy/plot/static *.npy *.gz *.txt
 recursive-include phy/cluster/manual/static *.html *.css
-recursive-include phy/gui/static *.html *.css
+recursive-include phy/gui/static *.html *.css *.js
 recursive-exclude * __pycache__
 recursive-exclude * *.py[co]

Makefile

@@ -1,14 +1,3 @@
-help:
-	@echo "clean - remove all build, test, coverage and Python artifacts"
-	@echo "clean-build - remove build artifacts"
-	@echo "clean-pyc - remove Python file artifacts"
-	@echo "lint - check style with flake8"
-	@echo "test - run tests quickly with the default Python"
-	@echo "release - package and upload a release"
-	@echo "apidoc - build API doc"
-
-clean: clean-build clean-pyc
-
 clean-build:
 	rm -fr build/
 	rm -fr dist/
@@ -20,21 +9,22 @@ clean-pyc:
 	find . -name '*~' -exec rm -f {} +
 	find . -name '__pycache__' -exec rm -fr {} +
 
+clean: clean-build clean-pyc
+
 lint:
 	flake8 phy
 
 test: lint
-	py.test
+	py.test --cov-report term-missing --cov=phy phy
 
 coverage:
 	coverage --html
 
-test-quick: lint
-	python setup.py test -a "-m \"not long\" phy"
-
-unit-tests: lint
-	python setup.py test -a phy
+apidoc:
+	python tools/api.py
 
-integration-tests: lint
-	python setup.py test -a tests
+build:
+	python setup.py sdist --formats=zip
 
+upload:
+	python setup.py sdist --formats=zip upload

README.md

@@ -1,12 +1,47 @@
 # phy project
 
-[![Build Status](https://travis-ci.org/kwikteam/phy.svg?branch=master)](https://travis-ci.org/kwikteam/phy)
-[![Build Status](https://ci.appveyor.com/api/projects/status/fuoyuo113domjplr/branch/master?svg=true)](https://ci.appveyor.com/project/kwikteam/phy/)
-[![codecov.io](https://img.shields.io/codecov/c/github/kwikteam/phy.svg?)](http://codecov.io/github/kwikteam/phy?branch=master)
+[![Build Status](https://img.shields.io/travis/kwikteam/phy.svg)](https://travis-ci.org/kwikteam/phy)
+[![Build Status](https://img.shields.io/appveyor/ci/kwikteam/phy.svg)](https://ci.appveyor.com/project/kwikteam/phy/)
+[![codecov.io](https://img.shields.io/codecov/c/github/kwikteam/phy.svg)](http://codecov.io/github/kwikteam/phy?branch=master)
 [![Documentation Status](https://readthedocs.org/projects/phy/badge/?version=latest)](https://readthedocs.org/projects/phy/?badge=latest)
 [![PyPI release](https://img.shields.io/pypi/v/phy.svg)](https://pypi.python.org/pypi/phy)
+[![GitHub release](https://img.shields.io/github/release/kwikteam/phy.svg)](https://github.com/kwikteam/phy/releases/latest)
 [![Join the chat at https://gitter.im/kwikteam/phy](https://badges.gitter.im/Join%20Chat.svg)](https://gitter.im/kwikteam/phy?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)
 
-[**phy**](https://github.com/kwikteam/phy) is an open source electrophysiological data analysis package in Python for neuronal recordings made with high-density multielectrode arrays containing up to thousands of channels.
+[**phy**](https://github.com/kwikteam/phy) is an open source neurophysiological data analysis package in Python. It provides features for sorting, analyzing, and visualizing extracellular recordings made with high-density multielectrode arrays containing hundreds to thousands of recording sites.
 
-* [Documentation](http://phy.cortexlab.net)
+
+## Overview
+
+**phy** contains the following subpackages:
+
+* **phy.cluster.manual**: an API for manual sorting, used to create graphical interfaces for neurophysiological data
+* **phy.gui**: a generic API for creating desktop applications with PyQt.
+* **phy.plot**: a generic API for creating high-performance plots with VisPy (using the graphics processor via OpenGL)
+
+phy doesn't provide any I/O code. It only provides Python routines to process and visualize data.
+
+
+## phy-contrib
+
+The [phy-contrib](https://github.com/kwikteam/phy-contrib) repo contains a set of plugins with integrated GUIs that work with dedicated automatic clustering software. Currently it provides:
+
+* **KwikGUI**: a manual sorting GUI that works with data processed with **klusta**, an automatic clustering package.
+* **TemplateGUI**: a manual sorting GUI that works with data processed with **Spyking Circus** and **KiloSort** (not released yet), which are template-matching-based spike sorting algorithms.
+
+
+## Getting started
+
+You will find installation instructions and a quick start guide in the [documentation](http://phy.readthedocs.org/en/latest/) (work in progress).
+
+
+## Links
+
+* [Documentation](http://phy.readthedocs.org/en/latest/) (work in progress)
+* [Mailing list](https://groups.google.com/forum/#!forum/phy-users)
+* [Sample data repository](http://phy.cortexlab.net/data/) (work in progress)
+
+
+## Credits
+
+**phy** is developed by [Cyrille Rossant](http://cyrille.rossant.net), [Shabnam Kadir](https://iris.ucl.ac.uk/iris/browse/profile?upi=SKADI56), [Dan Goodman](http://thesamovar.net/), [Max Hunter](https://iris.ucl.ac.uk/iris/browse/profile?upi=MLDHU99), and [Kenneth Harris](https://iris.ucl.ac.uk/iris/browse/profile?upi=KDHAR02), in the [Cortexlab](https://www.ucl.ac.uk/cortexlab), University College London.

docs/api.md

@@ -0,0 +1,3 @@
+# API reference
+
+TODO. In the meantime, see the code directly on GitHub.