Python update mechanism

[monsieurh]

Purpose
implement proposition #34 and provide a way for the user to upgrade ZPUI from within ZPUI

Implementation
The settings app was renamed to update ZPUI.
It comports two entries to respectively update zpui to the latest master branch or latest devel branch.
It can be easily extended to update to any branch/commit.

The user is shown a progressbar with the name of the current step while updating.

The class UpdateStep describe an atomic update step, it has a do() and undo() function as well as an accepted return code to check if a subprocess ran successfully.
The update app runs a succession of update steps :

• 'opening tmp dir'
• 'copying repo'
• 'pulling git'
• 'installing deps'
• 'running tests'
• 'change cwd back'
• 'patching ZPUI'
• 'restarting zpui'
• 'cleaning tmp dir'

If any step fail, the user is asked if he wants to cancel the update. If he agrees, all the previous steps are undo()'ed.

Discussion

• Are there some missing steps ?
• Is the user flow acceptable ? (prompted to cancel on each failure)
  The undo() method for installing deps step consists on reinstalling (as in pip install) the previous requirements.txt file. Is that safe enough ?

[CRImier]

Only tested the changes, haven't read into the code yet. Finishing the review for now, will read it as soon as I have some time =)

[CRImier]

I don't think having separate menu options is a good idea.

1. as we add more possible branches for users to try out and, possibly, use, we'd need to add (and hardcode! ) more menu entries
2. the user is supposed to remember which branch they're on before updating, so the update is no longer as straightforward as "press this button"
3. a misclick will make the user update from a wrong branch, which will lead in 1) long update times 2) possible problems during update if the branch they're updating from is not tested enough. Basically, switching ZPUI branches has to be a conscious choice.

[CRImier]

['git', 'pull', 'origin', 'master', '--ff-only']
error: unknown option `ff-only'
usage: git fetch [<options>] [<repository> [<refspec>...]]
   or: git fetch [<options>] <group>
   or: git fetch --multiple [<options>] [(<repository> | <group>)...]
   or: git fetch --all [<options>]

    -v, --verbose         be more verbose
    -q, --quiet           be more quiet
    --all                 fetch from all remotes
    -a, --append          append to .git/FETCH_HEAD instead of overwriting
    --upload-pack <path>  path to upload pack on remote end
    -f, --force           force overwrite of local branch
    -m, --multiple        fetch from multiple remotes
    -t, --tags            fetch all tags and associated objects
    -n                    do not fetch all tags (--no-tags)
    -p, --prune           prune remote-tracking branches no longer on remote
    --recurse-submodules[=<on-demand>]
                          control recursive fetching of submodules
    --dry-run             dry run
    -k, --keep            keep downloaded pack
    -u, --update-head-ok  allow updating of HEAD ref
    --progress            force progress reporting
    --depth <depth>       deepen history of shallow clone
    --unshallow           convert to a complete repository
    --update-shallow      accept refs that update .git/shallow
    --refmap <refmap>     specify fetch refmap

[CRImier]

['python2', '-B', '-m', 'pytest', '--doctest-modules', '-v', '--doctest-ignore-import[17/1812]
 '--ignore=output/drivers', '--ignore=input/drivers', '--ignore=apps/hardware_apps/status/', '
--ignore=apps/example_apps/fire_detector', '--ignore=apps/test_hardware']
==================================== test session starts =====================================
platform linux2 -- Python 2.7.9, pytest-3.3.1, py-1.5.2, pluggy-0.6.0 -- /usr/bin/python2
cachedir: .cache
rootdir: /tmp/zpuiMi8zqD, inifile:
collected 13 items / 2 errors / 5 skipped

=========================================== ERRORS ===========================================
_________________________ ERROR collecting ui/tests/test_checkbox.py _________________________
ImportError while importing test module '/tmp/zpuiMi8zqD/ui/tests/test_checkbox.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
ui/tests/test_checkbox.py:9: in <module>
    from checkbox import Checkbox
E   ImportError: No module named checkbox
______________________ ERROR collecting ui/tests/test_config_manager.py ______________________
ImportError while importing test module '/tmp/zpuiMi8zqD/ui/tests/test_config_manager.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
ui/tests/test_config_manager.py:9: in <module>
    from config_manager import UIConfigManager
E   ImportError: No module named config_manager
!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 2 errors during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!
============================ 5 skipped, 2 error in 17.87 seconds =============================

[CRImier]

Merging latest changes in devel into your branch should fix that error. If not, before testing, remove all *.pyc files and the __pycache__ folder from ui/tests .

[CRImier]

[ERROR] 2017-12-28 10:05:45 main.py [26565]: Error updating step 'running tests'
[ERROR] 2017-12-28 10:05:45 main.py [26565]: 'running tests' failed
Traceback (most recent call last):
  File "/root/ZPUI/apps/settings/main.py", line 53, in start_update
    step.do()
  File "/root/ZPUI/apps/settings/main.py", line 150, in do
    raise Exception("'{}' failed".format(self.name))
Exception: 'running tests' failed
[WARNING] 2017-12-28 10:05:47 main.py [26565]: no undo method for 'copying repo'
[WARNING] 2017-12-28 10:05:47 main.py [26565]: no undo method for 'pulling git'
[ERROR] 2017-12-28 10:05:48 input.py [26565]: Exception 'str' object is not callable caused by
 callback <function <lambda> at 0xb50e1df0> when key KEY_ENTER was received
[ERROR] 2017-12-28 10:05:48 input.py [26565]: Traceback (most recent call last):
  File "/root/ZPUI/input/input.py", line 179, in handle_callback
    callback()
  File "/root/ZPUI/ui/base_list_ui.py", line 278, in <lambda>
    "KEY_ENTER":lambda: self.select_entry()
  File "/root/ZPUI/ui/utils.py", line 16, in wrapper
    return func(self, *args, **kwargs)
  File "/root/ZPUI/ui/menu.py", line 85, in select_entry
    entry[1]()
  File "/root/ZPUI/apps/settings/main.py", line 38, in update_zpui
    self.start_update(branch="master")
  File "/root/ZPUI/apps/settings/main.py", line 61, in start_update
    self.rollback_update(bar, e, finished_steps, step)
  File "/root/ZPUI/apps/settings/main.py", line 72, in rollback_update
    passed.undo()
  File "/root/ZPUI/apps/settings/main.py", line 158, in undo
    self._undo()
TypeError: 'str' object is not callable

[ERROR] 2017-12-28 10:05:48 input.py [26565]: Locals:
[ERROR] 2017-12-28 10:05:48 input.py [26565]: {'self': <apps.settings.main.UpdateStep object a
t 0xb5071d90>}

This was the last problem I've encountered, and it completely stopped the update process for me - I had to Ctrl+C out of it (which won't be possible for an user that doesn't have access to the console and is trying to update from within ZPUI).

[CRImier]

['pip2', 'install', '-r', 'requirements.txt']
Could not open requirements file: [Errno 2] No such file or directory: 'requirements.txt'

Any idea why this could be happening?