executablebooks · chrisjsewell · Feb 29, 2020 · Feb 29, 2020 · Feb 29, 2020 · Feb 29, 2020
diff --git a/README.md b/README.md
@@ -41,7 +41,7 @@ to come ...
 
 ## Example CLI usage
 
-From checked-out folder:
+From checked-out repository folder:
 
 ```console
 $ jcache -h
@@ -50,112 +50,136 @@ Usage: jcache [OPTIONS] COMMAND [ARGS]...
   The command line interface of jupyter-cache.
 
 Options:
-  -v, --version     Show the version and exit.
-  -p, --cache-path  Print the current cache path and exit.
-  -h, --help        Show this message and exit.
+  -v, --version       Show the version and exit.
+  -p, --cache-path    Print the current cache path and exit.
+  -a, --autocomplete  Print the terminal autocompletion command and exit.
+  -h, --help          Show this message and exit.
 
 Commands:
-  cat-artifact    Print the contents of a commit artefact.
-  clear           Clear the cache completely.
-  commit-limit    Change the commit limit of the cache.
-  commit-nb       Commit a notebook that has already been executed.
-  commit-nbs      Commit notebook(s) that have already been executed.
-  diff-nb         Print a diff of a notebook to one stored in the cache.
-  execute         Execute outdated notebooks.
-  list-commits    List committed notebook records in the cache.
-  list-staged     List notebooks staged for possible execution.
-  remove-commits  Remove notebook commit(s) from the cache.
-  show-commit     Show details of a committed notebook in the cache.
-  show-staged     Show details of a staged notebook.
-  stage-nb        Commit a notebook, with possible assets.
-  stage-nbs       Stage notebook(s) for execution.
-  unstage-nbs     Unstage notebook(s) for execution.
+  cache    Commands for adding to and inspecting the cache.
+  clear    Clear the cache completely.
+  config   Commands for configuring the cache.
+  execute  Execute staged notebooks that are outdated.
+  stage    Commands for staging notebooks to be executed.
 ```
 
-### Commit Executed Notebooks
+**Important**: Execute this in the terminal for auto-completion:
 
-You can commit notebooks straight into the cache. When committing, a check will be made that the notebooks look to have been executed correctly, i.e. the cell execution counts go sequentially up from 1.
+```console
+eval "$(_JCACHE_COMPLETE=source jcache)"
+```
+
+### Caching Executed Notebooks
 
 ```console
-$ jcache commit-nbs tests/notebooks/basic.ipynb
-Cache path: /Users/cjs14/GitHub/sandbox/.jupyter_cache
+$ jcache cache -h
+Usage: jcache cache [OPTIONS] COMMAND [ARGS]...
+
+  Commands for adding to and inspecting the cache.
+
+Options:
+  -h, --help  Show this message and exit.
+
+Commands:
+  add-many      Cache notebook(s) that have already been executed.
+  add-one       Cache a notebook, with possible artefact files.
+  cat-artifact  Print the contents of a cached artefact.
+  diff-nb       Print a diff of a notebook to one stored in the cache.
+  list          List cached notebook records in the cache.
+  remove        Remove notebooks stored in the cache.
+  show          Show details of a cached notebook in the cache.
+```
+
+You can add notebooks straight into the cache. When caching, a check will be made that the notebooks look to have been executed correctly, i.e. the cell execution counts go sequentially up from 1.
+
+```console
+$ jcache cache add-many tests/notebooks/basic.ipynb
+Cache path: jupyter-cache/.jupyter_cache
 The cache does not yet exist, do you want to create it? [y/N]: y
-Committing: /Users/cjs14/GitHub/sandbox/tests/notebooks/basic.ipynb
+Caching: jupyter-cache/tests/notebooks/basic.ipynb
 Validity Error: Expected cell 1 to have execution_count 1 not 2
-The notebook may not have been executed, continue committing? [y/N]: y
+The notebook may not have been executed, continue caching? [y/N]: y
 Success!
 ```
 
 Or to skip validation:
 
 ```console
-$ jcache commit-nbs --no-validate tests/notebooks/*.ipynb
-Committing: /Users/cjs14/GitHub/sandbox/tests/notebooks/basic.ipynb
-Committing: /Users/cjs14/GitHub/sandbox/tests/notebooks/basic_failing.ipynb
-Committing: /Users/cjs14/GitHub/sandbox/tests/notebooks/basic_unrun.ipynb
-Committing: /Users/cjs14/GitHub/sandbox/tests/notebooks/complex_outputs.ipynb
+jcache cache add-many --no-validate tests/notebooks/*.ipynb
+Caching: jupyter-cache/tests/notebooks/basic.ipynb
+Caching: jupyter-cache/tests/notebooks/basic_failing.ipynb
+Caching: jupyter-cache/tests/notebooks/basic_unrun.ipynb
+Caching: jupyter-cache/tests/notebooks/complex_outputs.ipynb
+Caching: jupyter-cache/tests/notebooks/external_output.ipynb
 Success!
 ```
 
-Once you've committed some notebooks, you can look at the 'commit records' for what has been cached.
+Once you've cached some notebooks, you can look at the 'cache records' for what has been cached.
 
-Each notebook is hashed (code cells and kernel spec only), which is used to compare against 'staged' notebooks. Multiple hashes for the same URI can be added (the URI is just there for inspetion) and the size of the cache is limited (current default 1000) so that, at this size, the last accessed records begin to be deleted. You can remove cached records by the Primary Key (PK).
+Each notebook is hashed (code cells and kernel spec only), which is used to compare against 'staged' notebooks. Multiple hashes for the same URI can be added (the URI is just there for inspetion) and the size of the cache is limited (current default 1000) so that, at this size, the last accessed records begin to be deleted. You can remove cached records by their ID.
 
 ```console
-$ jcache list-commits --hashkeys
-  PK  URI                    Created           Accessed          Hashkey
-----  ---------------------  ----------------  ----------------  --------------------------------
-   4  complex_outputs.ipynb  2020-02-23 20:33  2020-02-23 20:33  800c4a057730a55a384cfe579e3850aa
-   3  basic_unrun.ipynb      2020-02-23 20:33  2020-02-23 20:33  818f3412b998fcf4fe9ca3cca11a3fc3
-   2  basic_failing.ipynb    2020-02-23 20:33  2020-02-23 20:33  72859c2bf1e12f35f30ef131f0bef320
+$ jcache cache list
+  ID  URI                                    Created           Accessed
+----  -------------------------------------  ----------------  ----------------
+   5  tests/notebooks/external_output.ipynb  2020-02-29 03:17  2020-02-29 03:17
+   4  tests/notebooks/complex_outputs.ipynb  2020-02-29 03:17  2020-02-29 03:17
+   3  tests/notebooks/basic_unrun.ipynb      2020-02-29 03:17  2020-02-29 03:17
+   2  tests/notebooks/basic_failing.ipynb    2020-02-29 03:17  2020-02-29 03:17
 ```
 
-You can also commit with artefacts (external outputs of the notebook execution).
+You can also cache notebooks with artefacts (external outputs of the notebook execution).
 
 ```console
-$ jcache commit-nb -nb tests/notebooks/basic.ipynb tests/notebooks/artifact_folder/artifact.txt
-Committing: /Users/cjs14/GitHub/jupyter-cache/tests/notebooks/basic.ipynb
+$ jcache cache add-one -nb tests/notebooks/basic.ipynb tests/notebooks/artifact_folder/artifact.txt
+Caching: jupyter-cache/tests/notebooks/basic.ipynb
 Success!
 ```
 
+Show a full description of a cached notebook by referring to its ID
+
 ```console
-$ jcache show-commit 1
-PK: 1
-URI: /Users/cjs14/GitHub/jupyter-cache/tests/notebooks/basic.ipynb
-Created: 2020-02-24 14:58
-Accessed: 2020-02-24 14:58
+$ jcache cache show 6
+ID: 6
+URI: jupyter-cache/tests/notebooks/basic.ipynb
+Created: 2020-02-29 03:19
+Accessed: 2020-02-29 03:19
 Hashkey: 818f3412b998fcf4fe9ca3cca11a3fc3
 Artifacts:
 - artifact_folder/artifact.txt
 ```
 
-```console
-$ jcache cat-artifact 1 artifact_folder/artifact.txt
-An artifact
+Note artefact paths must be 'upstream' of the notebook folder:
 
+```console
+$ jcache cache add-one -nb tests/notebooks/basic.ipynb tests/test_db.py
+Caching: jupyter-cache/tests/notebooks/basic.ipynb
+Artifact Error: Path 'jupyter-cache/tests/test_db.py' is not in folder 'jupyter-cache/tests/notebooks''
 ```
 
-These must be 'upstream' of the notebook folder:
+To view the contents of an execution artefact:
 
 ```console
-$ jcache commit-nb -nb tests/notebooks/basic.ipynb tests/test_db.py
-Committing: /Users/cjs14/GitHub/jupyter-cache/tests/notebooks/basic.ipynb
-Artifact Error: Path '/Users/cjs14/GitHub/jupyter-cache/tests/test_db.py' is not in folder '/Users/cjs14/GitHub/jupyter-cache/tests/notebooks''
+$ jcache cache cat-artifact 1 artifact_folder/artifact.txt
+An artifact
+
 ```
 
+You can directly remove a cached notebook by its ID:
+
 ```console
-$ jcache remove-commits 3
-Removing PK = 3
+$ jcache cache remove 4
+Removing Cache ID = 4
 Success!
 ```
 
-You can also diff any of the commit records with any (external) notebook:
+You can also diff any of the cached notebooks with any (external) notebook:
 
 ```console
-$ jcache diff-nb 2 tests/notebooks/basic.ipynb
+$ jcache cache diff-nb 2 tests/notebooks/basic.ipynb
 nbdiff
---- committed pk=2
-+++ other: /Users/cjs14/GitHub/sandbox/tests/notebooks/basic.ipynb
+--- cached pk=2
++++ other: sandbox/tests/notebooks/basic.ipynb
 ## inserted before nb/cells/1:
 +  code cell:
 +    execution_count: 2
@@ -177,96 +201,132 @@ nbdiff
 
 ### Staging Notebooks for execution
 
+```console
+$ jcache stage -h
+Usage: jcache stage [OPTIONS] COMMAND [ARGS]...
+
+  Commands for staging notebooks to be executed.
+
+Options:
+  -h, --help  Show this message and exit.
+
+Commands:
+  add-many     Stage notebook(s) for execution.
+  add-one      Stage a notebook, with possible asset files.
+  list         List notebooks staged for possible execution.
+  remove-ids   Un-stage notebook(s), by ID.
+  remove-uris  Un-stage notebook(s), by URI.
+  show         Show details of a staged notebook.
+```
+
 Staged notebooks are recorded as pointers to their URI,
 i.e. no physical copying takes place until execution time.
 
-If you stage some notebooks for execution, then you can list them to see which have existing records in the cache (by hash) and which will require execution:
+If you stage some notebooks for execution,
+then you can list them to see which have existing records in the cache (by hash),
+and which will require execution:
 
 ```console
-$ jcache stage-nbs tests/notebooks/*.ipynb
-Staging: /Users/cjs14/GitHub/sandbox/tests/notebooks/basic.ipynb
-Staging: /Users/cjs14/GitHub/sandbox/tests/notebooks/basic_failing.ipynb
-Staging: /Users/cjs14/GitHub/sandbox/tests/notebooks/basic_unrun.ipynb
-Staging: /Users/cjs14/GitHub/sandbox/tests/notebooks/complex_outputs.ipynb
+$ jcache stage add-many tests/notebooks/*.ipynb
+Staging: jupyter-cache/tests/notebooks/basic.ipynb
+Staging: jupyter-cache/tests/notebooks/basic_failing.ipynb
+Staging: jupyter-cache/tests/notebooks/basic_unrun.ipynb
+Staging: jupyter-cache/tests/notebooks/complex_outputs.ipynb
+Staging: jupyter-cache/tests/notebooks/external_output.ipynb
 Success!
 ```
 
 ```console
-$ jcache list-staged
-  PK  URI                                    Created             Commit Pk
-----  -------------------------------------  ----------------  -----------
-   4  tests/notebooks/complex_outputs.ipynb  2020-02-23 20:48            4
-   3  tests/notebooks/basic_unrun.ipynb      2020-02-23 20:48
-   2  tests/notebooks/basic_failing.ipynb    2020-02-23 20:48            2
-   1  tests/notebooks/basic.ipynb            2020-02-23 20:48
+$ jcache stage list
+  ID  URI                                    Created             Assets    Cache ID
+----  -------------------------------------  ----------------  --------  ----------
+   5  tests/notebooks/external_output.ipynb  2020-02-29 03:29         0           5
+   4  tests/notebooks/complex_outputs.ipynb  2020-02-29 03:29         0
+   3  tests/notebooks/basic_unrun.ipynb      2020-02-29 03:29         0           6
+   2  tests/notebooks/basic_failing.ipynb    2020-02-29 03:29         0           2
+   1  tests/notebooks/basic.ipynb            2020-02-29 03:29         0           6
+```
+
+You can remove a staged notebook by its URI or ID:
+
+```console
+$ jcache stage remove-ids 4
+Unstaging ID: 4
+Success!
 ```
 
 You can then run a basic execution of the required notebooks:
 
 ```console
+$ jcache cache remove 6
+Removing Cache ID = 6
+Success!
 $ jcache execute
-Executing: /Users/cjs14/GitHub/sandbox/tests/notebooks/basic.ipynb
-Success: /Users/cjs14/GitHub/sandbox/tests/notebooks/basic.ipynb
-Executing: /Users/cjs14/GitHub/sandbox/tests/notebooks/basic_unrun.ipynb
-Success: /Users/cjs14/GitHub/sandbox/tests/notebooks/basic_unrun.ipynb
+Executing: jupyter-cache/tests/notebooks/basic.ipynb
+Success: jupyter-cache/tests/notebooks/basic.ipynb
+Executing: jupyter-cache/tests/notebooks/basic_unrun.ipynb
+Success: jupyter-cache/tests/notebooks/basic_unrun.ipynb
 Finished!
 ```
 
-Successfully executed notebooks will be committed to the cache,
+Successfully executed notebooks will be cached to the cache,
 along with any 'artefacts' created by the execution, that are inside the notebook folder, and data supplied by the executor.
 
 ```console
-$ jcache list-staged
-  PK  URI                                    Created             Commit Pk
-----  -------------------------------------  ----------------  -----------
-   5  tests/notebooks/basic.ipynb            2020-02-23 20:57            5
-   4  tests/notebooks/complex_outputs.ipynb  2020-02-23 20:48            4
-   3  tests/notebooks/basic_unrun.ipynb      2020-02-23 20:48            6
-   2  tests/notebooks/basic_failing.ipynb    2020-02-23 20:48            2
+$ jcache stage list
+  ID  URI                                    Created             Assets    Cache ID
+----  -------------------------------------  ----------------  --------  ----------
+   5  tests/notebooks/external_output.ipynb  2020-02-29 03:29         0           5
+   3  tests/notebooks/basic_unrun.ipynb      2020-02-29 03:29         0           6
+   2  tests/notebooks/basic_failing.ipynb    2020-02-29 03:29         0           2
+   1  tests/notebooks/basic.ipynb            2020-02-29 03:29         0           6
 ```
 
 ```console
-jcache show-commit 5
-PK: 1
-URI: /Users/cjs14/GitHub/jupyter-cache/tests/notebooks/basic.ipynb
-Created: 2020-02-25 19:21
-Accessed: 2020-02-25 19:21
+$ jcache cache show 6
+ID: 6
+URI: jupyter-cache/tests/notebooks/basic_unrun.ipynb
+Created: 2020-02-29 03:41
+Accessed: 2020-02-29 03:41
 Hashkey: 818f3412b998fcf4fe9ca3cca11a3fc3
 Data:
-  execution_seconds: 1.4187269599999999
+  execution_seconds: 1.2328746560000003
 ```
 
 Once executed you may leave staged notebooks, for later re-execution, or remove them:
 
 ```console
-$ jcache unstage-nbs --all
+$ jcache stage remove-ids --all
 Are you sure you want to remove all? [y/N]: y
-Unstaging: /Users/cjs14/GitHub/jupyter-cache/tests/notebooks/basic.ipynb
+Unstaging ID: 1
+Unstaging ID: 2
+Unstaging ID: 3
+Unstaging ID: 5
 Success!
 ```
 
 You can also stage notebooks with assets; external files that are required by the notebook during execution. As with artefacts,
 these files must be in the same folder as the notebook, or a sub-folder.
 
 ```console
-$ jcache stage-nb -nb tests/notebooks/basic.ipynb tests/notebooks/artifact_folder/artifact.txt
+$ jcache stage add-one -nb tests/notebooks/basic.ipynb tests/notebooks/artifact_folder/artifact.txt
 Success!
 ```
 
 ```console
-$ jcache list-staged
-  PK  URI                          Created             Assets
+$ jcache stage list
+  ID  URI                          Created             Assets
 ----  ---------------------------  ----------------  --------
    1  tests/notebooks/basic.ipynb  2020-02-25 10:01         1
 ```
 
 ```console
-$ jcache show-staged 1
-PK: 1
-URI: /Users/cjs14/GitHub/jupyter-cache/tests/notebooks/basic.ipynb
+$ jcache stage show 1
+ID: 1
+URI: jupyter-cache/tests/notebooks/basic.ipynb
 Created: 2020-02-25 10:01
 Assets:
-- /Users/cjs14/GitHub/jupyter-cache/tests/notebooks/artifact_folder/artifact.txt
+- jupyter-cache/tests/notebooks/artifact_folder/artifact.txt
 ```
 
 ## Contributing