diff --git a/.gitignore b/.gitignore index fe1f1c0..9120502 100644 --- a/.gitignore +++ b/.gitignore @@ -6,5 +6,6 @@ pip-wheel-metadata/ .ipynb_checkpoints/ *.ipynb notes.md -*.yaml +config.yaml dist/ +build/ diff --git a/CHANGES.md b/CHANGES.md index d677b31..30a41a4 100644 --- a/CHANGES.md +++ b/CHANGES.md @@ -1,5 +1,12 @@ # Changes +## 0.0.3 (2020-07-19) + +- FEATURE: Added wizard GUI for backup tasks (`snap`, `backup`, `cleanup`) +- FEATURE: Added new configuration options (`always_changed`, `written_threshold`, `check_diff`) for detecting snapshot tasks +- FEATURE: CLI and GUI translations (i18n) +- FIX: Added missing type checks + ## 0.0.2 (2020-07-14) - FEATURE: New, fully object oriented base library diff --git a/MANIFEST.in b/MANIFEST.in new file mode 100644 index 0000000..ab70d3c --- /dev/null +++ b/MANIFEST.in @@ -0,0 +1 @@ +recursive-include src/abgleich/share/ *.* diff --git a/README.md b/README.md index d907f16..9b85e9d 100644 --- a/README.md +++ b/README.md @@ -2,23 +2,33 @@ ## SYNOPSIS -`abgleich` is a simple ZFS sync tool. It displays source and target ZFS zpool, dataset and snapshot trees. It creates meaningful snapshots only if datasets have actually been changed. It compares a source zpool tree to a target, backup zpool tree. It pushes backups from a source to a target. It cleanes up older snapshots on the source side if they are present on the target side. It runs on a command line and produces nice, user-friendly, human-readable, colorized output. +`abgleich` is a simple ZFS sync tool. It displays source and target ZFS zpool, dataset and snapshot trees. It creates meaningful snapshots only if datasets have actually been changed. It compares a source zpool tree to a target, backup zpool tree. It pushes backups from a source to a target. It cleanes up older snapshots on the source side if they are present on the target side. It runs on a command line and produces nice, user-friendly, human-readable, colorized output. It also includes a GUI. + +## CLI EXAMPLE ![demo](https://github.com/pleiszenburg/abgleich/blob/master/docs/demo.png?raw=true "demo") +## GUI EXAMPLE + +| snap | backup | cleanup | +|:----:|:------:|:-------:| +| ![snap](https://github.com/pleiszenburg/abgleich/blob/master/docs/demo_gui01.png?raw=true "snap") | ![backup](https://github.com/pleiszenburg/abgleich/blob/master/docs/demo_gui02.png?raw=true "backup") | ![cleanup](https://github.com/pleiszenburg/abgleich/blob/master/docs/demo_gui03.png?raw=true "cleanup") | + ## INSTALLATION +The base CLI tool can be installed as follows: + ```bash pip install -vU abgleich ``` -or +An installation also including a GUI can be triggered by running: ```bash -pip install -vU git+https://github.com/pleiszenburg/abgleich.git@master +pip install -vU abgleich[gui] ``` -Requires [CPython](https://en.wikipedia.org/wiki/CPython) 3.6 or later, a [Unix shell](https://en.wikipedia.org/wiki/Unix_shell) and [ssh](https://en.wikipedia.org/wiki/Secure_Shell). Tested with [OpenZFS](https://en.wikipedia.org/wiki/OpenZFS) 0.8.x on Linux. +Requires [CPython](https://en.wikipedia.org/wiki/CPython) 3.6 or later, a [Unix shell](https://en.wikipedia.org/wiki/Unix_shell) and [ssh](https://en.wikipedia.org/wiki/Secure_Shell). GUI support requires [Qt5](https://en.wikipedia.org/wiki/Qt_(software)) in addition. Tested with [OpenZFS](https://en.wikipedia.org/wiki/OpenZFS) 0.8.x on Linux. `abgleich`, CPython and the Unix shell must only be installed on one of the involved systems. Any remote system will be contacted via ssh and provided with direct ZFS commands. @@ -55,6 +65,9 @@ target: host: bigdata user: zfsadmin keep_snapshots: 2 +always_changed: no +written_threshold: 1048576 +check_diff: yes suffix: _backup digits: 2 ignore: @@ -65,7 +78,7 @@ ssh: cipher: aes256-gcm@openssh.com ``` -The prefix can be empty on either side. If a `host` is set to `localhost`, the `user` field can be left empty. Both source and target can be remote hosts or localhost at the same time. `keep_snapshots` is an integer and must be greater or equal to `1`. It specifies the number of snapshots that are kept per dataset on the source side when a cleanup operation is triggered. `suffix` contains the name suffix for new snapshots. `digits` specifies how many digits are used for a decimal number describing the n-th snapshot per dataset per day as part of the name of new snapshots. `ignore` lists stuff underneath the `prefix` which will be ignored by this tool, i.e. no snapshots, backups or cleanups. `ssh` allows to fine-tune the speed of backups. In fast local networks, it is best to set `compression` to `no` because the compression is usually slowing down the transfer. However, for low-bandwidth transmissions, it makes sense to set it to `yes`. For significantly better speed in fast local networks, make sure that both the source and the target system support a common cipher, which is accelerated by [AES-NI](https://en.wikipedia.org/wiki/AES_instruction_set) on both ends. +The prefix can be empty on either side. If a `host` is set to `localhost`, the `user` field can be left empty. Both source and target can be remote hosts or localhost at the same time. `keep_snapshots` is an integer and must be greater or equal to `1`. It specifies the number of snapshots that are kept per dataset on the source side when a cleanup operation is triggered. `suffix` contains the name suffix for new snapshots. Setting `always_changed` to `yes` causes `abgleich` to beliefe that all datasets have always changed since the last snapshot, completely ignoring what ZFS actually reports. No diff will be checked produced for values of `written` lower than `written_threshold`. Checking diffs can be completely deactivated by setting `check_diff` to `no`. `digits` specifies how many digits are used for a decimal number describing the n-th snapshot per dataset per day as part of the name of new snapshots. `ignore` lists stuff underneath the `prefix` which will be ignored by this tool, i.e. no snapshots, backups or cleanups. `ssh` allows to fine-tune the speed of backups. In fast local networks, it is best to set `compression` to `no` because the compression is usually slowing down the transfer. However, for low-bandwidth transmissions, it makes sense to set it to `yes`. For significantly better speed in fast local networks, make sure that both the source and the target system support a common cipher, which is accelerated by [AES-NI](https://en.wikipedia.org/wiki/AES_instruction_set) on both ends. ## USAGE @@ -91,6 +104,10 @@ Send (new) datasets and new snapshots from source to target. Cleanup older local snapshots on source side if they are present on both sides. Of those snapshots present on both sides, keep at least `keep_snapshots` number of snapshots on source side. +### `abgleich wizard config.yaml` + +Runs a sequence of `snap`, `backup` and `cleanup` in a wizard GUI. This command is only available if `abgleich` was installed with GUI support. + ## SPEED `abgleich` uses Python's [type hints](https://docs.python.org/3/library/typing.html) and enforces them with [typeguard](https://github.com/agronholm/typeguard) at runtime. It furthermore makes countless assertions. diff --git a/docs/demo_gui01.png b/docs/demo_gui01.png new file mode 100644 index 0000000..57e6dc4 Binary files /dev/null and b/docs/demo_gui01.png differ diff --git a/docs/demo_gui02.png b/docs/demo_gui02.png new file mode 100644 index 0000000..9a62c71 Binary files /dev/null and b/docs/demo_gui02.png differ diff --git a/docs/demo_gui03.png b/docs/demo_gui03.png new file mode 100644 index 0000000..8da247b Binary files /dev/null and b/docs/demo_gui03.png differ diff --git a/makefile b/makefile index cff9b42..23200e7 100644 --- a/makefile +++ b/makefile @@ -1,4 +1,7 @@ +black: + black . + clean: -rm -r build/* find src/ -name '*.pyc' -exec sudo rm -f {} + @@ -14,14 +17,14 @@ clean: release: make clean - # python setup.py sdist bdist_wheel + python setup.py sdist bdist_wheel python setup.py sdist - # gpg --detach-sign -a dist/abgleich*.whl + gpg --detach-sign -a dist/abgleich*.whl gpg --detach-sign -a dist/abgleich*.tar.gz install: pip install -vU pip setuptools - pip install -v -e .[dev] + pip install -v -e .[all] upload: for filename in $$(ls dist/*.tar.gz dist/*.whl) ; do \ diff --git a/pyproject.toml b/pyproject.toml index d1e6ae6..fe89541 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -1,2 +1,22 @@ [build-system] requires = ["setuptools", "wheel"] + +[tool.black] +target-version = ['py36'] +include = '\.pyi?$' +exclude = ''' +/( + \.eggs + | \.git + | \.hg + | \.mypy_cache + | \.tox + | \.venv + | _build + | buck-out + | build + | dist + | env?? + | env +)/ +''' diff --git a/setup.py b/setup.py index 4904d7e..45d2fc6 100644 --- a/setup.py +++ b/setup.py @@ -6,9 +6,9 @@ zfs sync tool https://github.com/pleiszenburg/abgleich - setup.py: Used for package distribution + setup.py: Used for package distribution - Copyright (C) 2019-2020 Sebastian M. Ernst + Copyright (C) 2019-2020 Sebastian M. Ernst The contents of this file are subject to the GNU Lesser General Public License @@ -40,7 +40,7 @@ # +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ # Package version -__version__ = "0.0.2" +__version__ = "0.0.3" # List all versions of Python which are supported python_minor_min = 6 @@ -57,6 +57,15 @@ # Define source directory (path) SRC_DIR = "src" +# Requirements +extras_require = { + "dev": ["black", "python-language-server[all]", "setuptools", "twine", "wheel",], + "gui": ["pyqt5",], +} +extras_require["all"] = list( + {rq for target in extras_require.keys() for rq in extras_require[target]} +) + # Install package setup( name="abgleich", @@ -78,20 +87,13 @@ python_requires=">=3.{MINOR:d}".format(MINOR=python_minor_min), setup_requires=[], install_requires=["click", "tabulate", "pyyaml", "typeguard",], - extras_require={ - "dev": [ - "black", - "python-language-server[all]", - "setuptools", - "twine", - "wheel", - ] - }, + extras_require=extras_require, zip_safe=False, entry_points={"console_scripts": ["abgleich = abgleich.cli:cli",],}, classifiers=[ "Development Status :: 5 - Production/Stable", "Environment :: Console", + "Environment :: X11 Applications", "Intended Audience :: Developers", "Intended Audience :: Education", "Intended Audience :: Information Technology", diff --git a/src/abgleich/cli/_main_.py b/src/abgleich/cli/_main_.py index e423bf3..43a6572 100644 --- a/src/abgleich/cli/_main_.py +++ b/src/abgleich/cli/_main_.py @@ -6,9 +6,9 @@ zfs sync tool https://github.com/pleiszenburg/abgleich - src/abgleich/cli/_main_.py: CLI auto-detection + src/abgleich/cli/_main_.py: CLI auto-detection - Copyright (C) 2019-2020 Sebastian M. Ernst + Copyright (C) 2019-2020 Sebastian M. Ernst The contents of this file are subject to the GNU Lesser General Public License @@ -46,7 +46,12 @@ def _add_commands(ctx): for item in os.listdir(os.path.dirname(__file__)) if not item.startswith("_") ): - ctx.add_command(getattr(importlib.import_module("abgleich.cli.%s" % cmd), cmd)) + try: + ctx.add_command( + getattr(importlib.import_module("abgleich.cli.%s" % cmd), cmd) + ) + except ModuleNotFoundError: # likely no gui support + continue @click.group() diff --git a/src/abgleich/cli/backup.py b/src/abgleich/cli/backup.py index 543e75f..74303a2 100644 --- a/src/abgleich/cli/backup.py +++ b/src/abgleich/cli/backup.py @@ -32,6 +32,7 @@ import click from ..core.config import Config +from ..core.i18n import t from ..core.zpool import Zpool # +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ @@ -51,10 +52,10 @@ def backup(configfile): transactions = source_zpool.get_backup_transactions(target_zpool) if len(transactions) == 0: - print("nothing to do") + print(t("nothing to do")) return transactions.print_table() - click.confirm("Do you want to continue?", abort=True) + click.confirm(t("Do you want to continue?"), abort=True) transactions.run() diff --git a/src/abgleich/cli/cleanup.py b/src/abgleich/cli/cleanup.py index dfc63a0..52a08d1 100644 --- a/src/abgleich/cli/cleanup.py +++ b/src/abgleich/cli/cleanup.py @@ -34,6 +34,7 @@ import click from ..core.config import Config +from ..core.i18n import t from ..core.io import humanize_size from ..core.zpool import Zpool @@ -55,11 +56,11 @@ def cleanup(configfile): transactions = source_zpool.get_cleanup_transactions(target_zpool) if len(transactions) == 0: - print("nothing to do") + print(t("nothing to do")) return transactions.print_table() - click.confirm("Do you want to continue?", abort=True) + click.confirm(t("Do you want to continue?"), abort=True) transactions.run() diff --git a/src/abgleich/cli/snap.py b/src/abgleich/cli/snap.py index e3d1d04..a2d62f9 100644 --- a/src/abgleich/cli/snap.py +++ b/src/abgleich/cli/snap.py @@ -32,6 +32,7 @@ import click from ..core.config import Config +from ..core.i18n import t from ..core.zpool import Zpool # +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ @@ -47,10 +48,10 @@ def snap(configfile): transactions = zpool.get_snapshot_transactions() if len(transactions) == 0: - print("nothing to do") + print(t("nothing to do")) return transactions.print_table() - click.confirm("Do you want to continue?", abort=True) + click.confirm(t("Do you want to continue?"), abort=True) transactions.run() diff --git a/src/abgleich/cli/wizard.py b/src/abgleich/cli/wizard.py new file mode 100644 index 0000000..616a344 --- /dev/null +++ b/src/abgleich/cli/wizard.py @@ -0,0 +1,47 @@ +# -*- coding: utf-8 -*- + +""" + +ABGLEICH +zfs sync tool +https://github.com/pleiszenburg/abgleich + + src/abgleich/cli/wizard.py: wizard command entry point + + Copyright (C) 2019-2020 Sebastian M. Ernst + + +The contents of this file are subject to the GNU Lesser General Public License +Version 2.1 ("LGPL" or "License"). You may not use this file except in +compliance with the License. You may obtain a copy of the License at +https://www.gnu.org/licenses/old-licenses/lgpl-2.1.txt +https://github.com/pleiszenburg/abgleich/blob/master/LICENSE + +Software distributed under the License is distributed on an "AS IS" basis, +WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License for the +specific language governing rights and limitations under the License. + + +""" + + +# +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ +# IMPORT +# +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ + +import click + +from ..core.config import Config +from ..gui.lib import run_app +from ..gui.wizard import WizardUi + +# +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ +# ROUTINES +# +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ + + +@click.command(short_help="run wizard gui") +@click.argument("configfile", type=click.File("r", encoding="utf-8")) +def wizard(configfile): + + run_app(WizardUi, config=Config.from_fd(configfile)) diff --git a/src/abgleich/core/abc.py b/src/abgleich/core/abc.py index 7b30159..58a076a 100644 --- a/src/abgleich/core/abc.py +++ b/src/abgleich/core/abc.py @@ -51,6 +51,10 @@ class ComparisonItemABC(abc.ABC): pass +class ConfigABC(abc.ABC): + pass + + class DatasetABC(abc.ABC): pass diff --git a/src/abgleich/core/command.py b/src/abgleich/core/command.py index ba43f7e..94d9bf8 100644 --- a/src/abgleich/core/command.py +++ b/src/abgleich/core/command.py @@ -48,7 +48,7 @@ def __init__(self, cmd: typing.List[str]): def __str__(self) -> str: - return " ".join(self._cmd) + return " ".join([item.replace(" ", "\\ ") for item in self._cmd]) def run(self): @@ -60,7 +60,7 @@ def run(self): output, errors = output.decode("utf-8"), errors.decode("utf-8") if not status or len(errors.strip()) > 0: - raise SystemError("command failed", self.cmd, output, errors) + raise SystemError("command failed", str(self), output, errors) return output, errors @@ -93,7 +93,11 @@ def run_pipe(self, other: CommandABC): ) ): raise SystemError( - "command pipe failed", self.cmd, other.cmd, errors_1, output_2, errors_2 + "command pipe failed", + f"{str(self):s} | {str(other):s}", + errors_1, + output_2, + errors_2, ) return errors_1, output_2, errors_2 diff --git a/src/abgleich/core/comparison.py b/src/abgleich/core/comparison.py index b1b541a..e9c08b2 100644 --- a/src/abgleich/core/comparison.py +++ b/src/abgleich/core/comparison.py @@ -72,6 +72,10 @@ def __init__( self._a, self._b, self._merged = a, b, merged + def __len__(self) -> int: + + return len(self._merged) + @property def a(self) -> ComparisonParentTypes: diff --git a/src/abgleich/core/config.py b/src/abgleich/core/config.py index f207f33..251c3a9 100644 --- a/src/abgleich/core/config.py +++ b/src/abgleich/core/config.py @@ -35,6 +35,7 @@ import yaml from yaml import CLoader +from .abc import ConfigABC from .lib import valid_name # +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ @@ -43,7 +44,7 @@ @typeguard.typechecked -class Config(dict): +class Config(ConfigABC, dict): @classmethod def from_fd(cls, fd: typing.TextIO): diff --git a/src/abgleich/core/dataset.py b/src/abgleich/core/dataset.py index c1451aa..8182b31 100644 --- a/src/abgleich/core/dataset.py +++ b/src/abgleich/core/dataset.py @@ -33,8 +33,9 @@ import typeguard -from .abc import DatasetABC, PropertyABC, TransactionABC, SnapshotABC +from .abc import ConfigABC, DatasetABC, PropertyABC, TransactionABC, SnapshotABC from .command import Command +from .i18n import t from .lib import root from .property import Property from .transaction import Transaction, TransactionMeta @@ -53,7 +54,7 @@ def __init__( properties: typing.Dict[str, PropertyABC], snapshots: typing.List[SnapshotABC], side: str, - config: typing.Dict, + config: ConfigABC, ): self._name = name @@ -86,13 +87,20 @@ def changed(self) -> bool: if len(self) == 0: return True + if self._config["always_changed"]: + return True if self._properties["written"].value == 0: return False - if self._properties["written"].value > (1024 ** 2): - return True if self._properties["type"].value == "volume": return True + if self._config["written_threshold"] is not None: + if self._properties["written"].value > self._config["written_threshold"]: + return True + + if not self._config["check_diff"]: + return True + output, _ = Command.on_side( ["zfs", "diff", f"{self._name:s}@{self._snapshots[-1].name:s}"], self._side, @@ -126,10 +134,12 @@ def get_snapshot_transaction(self) -> TransactionABC: return Transaction( TransactionMeta( - type="snapshot", - dataset_subname=self._subname, - snapshot_name=snapshot_name, - written=self._properties["written"].value, + **{ + t("type"): t("snapshot"), + t("dataset_subname"): self._subname, + t("snapshot_name"): snapshot_name, + t("written"): self._properties["written"].value, + } ), [ Command.on_side( @@ -179,7 +189,7 @@ def from_entities( name: str, entities: typing.OrderedDict[str, typing.List[typing.List[str]]], side: str, - config: typing.Dict, + config: ConfigABC, ) -> DatasetABC: properties = { diff --git a/src/abgleich/core/i18n.py b/src/abgleich/core/i18n.py new file mode 100644 index 0000000..5007675 --- /dev/null +++ b/src/abgleich/core/i18n.py @@ -0,0 +1,88 @@ +# -*- coding: utf-8 -*- + +""" + +ABGLEICH +zfs sync tool +https://github.com/pleiszenburg/abgleich + + src/abgleich/core/i18n.py: Translations + + Copyright (C) 2019-2020 Sebastian M. Ernst + + +The contents of this file are subject to the GNU Lesser General Public License +Version 2.1 ("LGPL" or "License"). You may not use this file except in +compliance with the License. You may obtain a copy of the License at +https://www.gnu.org/licenses/old-licenses/lgpl-2.1.txt +https://github.com/pleiszenburg/abgleich/blob/master/LICENSE + +Software distributed under the License is distributed on an "AS IS" basis, +WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License for the +specific language governing rights and limitations under the License. + + +""" + +# +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ +# IMPORT +# +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ + +import locale +import os + +import typeguard +import yaml +from yaml import CDumper, CLoader + +# +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ +# CLASS +# +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ + + +@typeguard.typechecked +class _Lang(dict): + def __init__(self): + + super().__init__() + self._lang = locale.getlocale()[0].split("_")[0] + self._path = os.path.join( + os.path.dirname(__file__), "..", "share", "translations.yaml" + ) + self._load() + + def __call__(self, name: str) -> str: + + assert len(name) > 0 + + if int(os.environ.get("ABGLEICH_TRANSLATE", "0")) == 1: + if name not in self.keys(): + self._add_item(name) + + return self.get(name, {}).get(self._lang, name) + + def _add_item(self, name: str): + + self[name] = {} + self._dump() + + def _load(self): + + self.clear() + + with open(self._path, "r") as f: + self.update(yaml.load(f.read(), Loader=CLoader)) + + def _dump(self): + + with open(self._path, "w") as f: + f.write( + yaml.dump(self.copy(), Dumper=CDumper, allow_unicode=True, indent=4) + ) + + +# +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ +# API +# +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ + +t = _Lang() diff --git a/src/abgleich/core/io.py b/src/abgleich/core/io.py index 425d2eb..bd2fae0 100644 --- a/src/abgleich/core/io.py +++ b/src/abgleich/core/io.py @@ -6,9 +6,9 @@ zfs sync tool https://github.com/pleiszenburg/abgleich - src/abgleich/core/io.py: Command line IO + src/abgleich/core/io.py: Command line IO - Copyright (C) 2019-2020 Sebastian M. Ernst + Copyright (C) 2019-2020 Sebastian M. Ernst The contents of this file are subject to the GNU Lesser General Public License @@ -24,6 +24,13 @@ """ +# +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ +# IMPORT +# +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ + +import typing + +import typeguard # +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ # CONSTANTS @@ -50,26 +57,32 @@ # +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ -def colorize(text, col): +@typeguard.typechecked +def colorize(text: str, col: str) -> str: return c.get(col.upper(), c["GREY"]) + text + c["RESET"] -def humanize_size(size, add_color=False): +@typeguard.typechecked +def humanize_size( + size: typing.Union[float, int], add_color: bool = False, get_rgb: bool = False +) -> str: suffix = "B" - for unit, color in ( - ("", "cyan"), - ("Ki", "green"), - ("Mi", "yellow"), - ("Gi", "red"), - ("Ti", "magenta"), - ("Pi", "white"), - ("Ei", "white"), - ("Zi", "white"), - ("Yi", "white"), + for unit, color, rgb in ( + ("", "cyan", "#0000FF"), + ("Ki", "green", "#00FF00"), + ("Mi", "yellow", "#FFFF00"), + ("Gi", "red", "#FF0000"), + ("Ti", "magenta", "#FF00FF"), + ("Pi", "white", "#FFFFFF"), + ("Ei", "white", "#FFFFFF"), + ("Zi", "white", "#FFFFFF"), + ("Yi", "white", "#FFFFFF"), ): if abs(size) < 1024.0: + if get_rgb: + return rgb text = "%3.1f %s%s" % (size, unit, suffix) if add_color: text = colorize(text, color) diff --git a/src/abgleich/core/snapshot.py b/src/abgleich/core/snapshot.py index 8c200dc..1a5841e 100644 --- a/src/abgleich/core/snapshot.py +++ b/src/abgleich/core/snapshot.py @@ -32,8 +32,9 @@ import typeguard -from .abc import PropertyABC, SnapshotABC, TransactionABC +from .abc import ConfigABC, PropertyABC, SnapshotABC, TransactionABC from .command import Command +from .i18n import t from .lib import root from .property import Property from .transaction import Transaction, TransactionMeta @@ -52,7 +53,7 @@ def __init__( properties: typing.Dict[str, PropertyABC], context: typing.List[SnapshotABC], side: str, - config: typing.Dict, + config: ConfigABC, ): self._name = name @@ -81,9 +82,11 @@ def get_cleanup_transaction(self) -> TransactionABC: return Transaction( meta=TransactionMeta( - type="cleanup_snapshot", - snapshot_subparent=self._subparent, - snapshot_name=self._name, + **{ + t("type"): t("cleanup_snapshot"), + t("snapshot_subparent"): self._subparent, + t("snapshot_name"): self._name, + } ), commands=[ Command.on_side( @@ -124,12 +127,14 @@ def get_backup_transaction( return Transaction( meta=TransactionMeta( - type="push_snapshot" - if ancestor is None - else "push_snapshot_incremental", - snapshot_subparent=self._subparent, - ancestor_name="" if ancestor is None else ancestor.name, - snapshot_name=self.name, + **{ + t("type"): t("transfer_snapshot") + if ancestor is None + else t("transfer_snapshot_incremental"), + t("snapshot_subparent"): self._subparent, + t("ancestor_name"): "" if ancestor is None else ancestor.name, + t("snapshot_name"): self.name, + } ), commands=commands, ) @@ -171,7 +176,7 @@ def from_entity( entity: typing.List[typing.List[str]], context: typing.List[SnapshotABC], side: str, - config: typing.Dict, + config: ConfigABC, ) -> SnapshotABC: properties = { diff --git a/src/abgleich/core/transaction.py b/src/abgleich/core/transaction.py index 0228380..3c2e558 100644 --- a/src/abgleich/core/transaction.py +++ b/src/abgleich/core/transaction.py @@ -34,6 +34,7 @@ import typeguard from .abc import CommandABC, TransactionABC, TransactionListABC, TransactionMetaABC +from .i18n import t from .io import colorize, humanize_size # +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ @@ -55,6 +56,18 @@ def __init__( self._running = False self._error = None + self._changed = None + + @property + def changed(self) -> typing.Union[None, typing.Callable]: + + return self._changed + + @changed.setter + def changed(self, value: typing.Union[None, typing.Callable]): + + self._changed = value + @property def complete(self) -> bool: @@ -84,7 +97,10 @@ def run(self): if self._complete: return + self._running = True + if self._changed is not None: + self._changed() try: if len(self._commands) == 1: @@ -98,6 +114,8 @@ def run(self): finally: self._running = False self._complete = True + if self._changed is not None: + self._changed() MetaTypes = typing.Union[str, int, float] @@ -139,42 +157,105 @@ class TransactionList(TransactionListABC): def __init__(self): self._transactions = [] + self._changed = None def __len__(self) -> int: return len(self._transactions) + def __getitem__(self, index: int) -> TransactionABC: + + return self._transactions[index] + + @property + def changed(self) -> typing.Union[None, typing.Callable]: + + return self._changed + + @changed.setter + def changed(self, value: typing.Union[None, typing.Callable]): + + self._changed = value + + @property + def table_columns(self) -> typing.List[str]: + + headers = set() + for transaction in self._transactions: + keys = list(transaction.meta.keys()) + assert t("type") in keys + headers.update(keys) + headers = list(headers) + headers.sort() + + if len(headers) == 0: + return headers + + type_index = headers.index(t("type")) + if type_index != 0: + headers.pop(type_index) + headers.insert(0, t("type")) + + return headers + + @property + def table_rows(self) -> typing.List[str]: + + return [f'{t("transaction"):s} #{index:d}' for index in range(1, len(self) + 1)] + def append(self, transaction: TransactionABC): self._transactions.append(transaction) + if self._changed is not None: + self._link_transaction(transaction) def extend(self, transactions: TransactionIterableTypes): + transactions = list(transactions) self._transactions.extend(transactions) + if self._changed is not None: + for transaction in transactions: + self._link_transaction(transaction) + + def clear(self): + + self._transactions.clear() + self._changed() + + def _link_transaction(self, transaction: TransactionABC): + + transaction.changed = lambda: self._changed( + self._transactions.index(transaction) + ) + transaction.changed() def print_table(self): if len(self) == 0: return - headers = self._table_headers() - colalign = self._table_colalign(headers) + table_columns = self.table_columns + colalign = self._table_colalign(table_columns) table = [ [ self._table_format_cell(header, transaction.meta.get(header)) - for header in headers + for header in table_columns ] for transaction in self._transactions ] - print(tabulate(table, headers=headers, tablefmt="github", colalign=colalign,)) + print( + tabulate( + table, headers=table_columns, tablefmt="github", colalign=colalign, + ) + ) @staticmethod def _table_format_cell(header: str, value: MetaNoneTypes) -> str: FORMAT = { - "written": lambda v: humanize_size(v, add_color=True), + t("written"): lambda v: humanize_size(v, add_color=True), } return FORMAT.get(header, str)(value) @@ -182,7 +263,7 @@ def _table_format_cell(header: str, value: MetaNoneTypes) -> str: @staticmethod def _table_colalign(headers: typing.List[str]) -> typing.List[str]: - RIGHT = ("written",) + RIGHT = (t("written"),) DECIMAL = tuple() colalign = [] @@ -196,29 +277,12 @@ def _table_colalign(headers: typing.List[str]) -> typing.List[str]: return colalign - def _table_headers(self) -> typing.List[str]: - - headers = set() - for transaction in self._transactions: - keys = list(transaction.meta.keys()) - assert "type" in keys - headers.update(keys) - headers = list(headers) - headers.sort() - - type_index = headers.index("type") - if type_index != 0: - headers.pop(type_index) - headers.insert(0, "type") - - return headers - def run(self): for transaction in self._transactions: print( - f'({colorize(transaction.meta["type"], "white"):s}) ' + f'({colorize(transaction.meta[t("type")], "white"):s}) ' f'{colorize(" | ".join([str(command) for command in transaction.commands]), "yellow"):s}' ) @@ -231,7 +295,7 @@ def run(self): assert transaction.complete if transaction.error is not None: - print(colorize("FAILED", "red")) + print(colorize(t("FAILED"), "red")) raise transaction.error else: - print(colorize("OK", "green")) + print(colorize(t("OK"), "green")) diff --git a/src/abgleich/core/zpool.py b/src/abgleich/core/zpool.py index 728ef83..ca3015c 100644 --- a/src/abgleich/core/zpool.py +++ b/src/abgleich/core/zpool.py @@ -36,14 +36,17 @@ from .abc import ( ComparisonItemABC, + ConfigABC, DatasetABC, SnapshotABC, + TransactionABC, TransactionListABC, ZpoolABC, ) from .command import Command from .comparison import Comparison from .dataset import Dataset +from .i18n import t from .io import colorize, humanize_size from .lib import join, root from .property import Property @@ -57,7 +60,7 @@ @typeguard.typechecked class Zpool(ZpoolABC): def __init__( - self, datasets: typing.List[DatasetABC], side: str, config: typing.Dict, + self, datasets: typing.List[DatasetABC], side: str, config: ConfigABC, ): self._datasets = datasets @@ -85,7 +88,7 @@ def root(self) -> str: return self._root - def get_cleanup_transactions(self, other: ZpoolABC) -> TransactionListABC: + def get_cleanup_transactions(self, other: ZpoolABC,) -> TransactionListABC: assert self.side == "source" assert other.side == "target" @@ -95,25 +98,51 @@ def get_cleanup_transactions(self, other: ZpoolABC) -> TransactionListABC: for dataset_item in zpool_comparison.merged: - if dataset_item.get_item().subname in self._config["ignore"]: - continue - if dataset_item.a is None or dataset_item.b is None: + cleanup_transactions = self._get_cleanup_from_datasetitem(dataset_item) + if cleanup_transactions is None: continue + transactions.extend(cleanup_transactions) - dataset_comparison = Comparison.from_datasets( - dataset_item.a, dataset_item.b - ) - snapshots = dataset_comparison.a_overlap_tail[ - : -self._config["keep_snapshots"] - ] + return transactions - transactions.extend( - (snapshot.get_cleanup_transaction() for snapshot in snapshots) - ) + def generate_cleanup_transactions( + self, other: ZpoolABC, + ) -> typing.Generator[ + typing.Tuple[ + int, + typing.Union[ + None, typing.Union[None, typing.Generator[TransactionABC, None, None]] + ], + ], + None, + None, + ]: - return transactions + assert self.side == "source" + assert other.side == "target" + + zpool_comparison = Comparison.from_zpools(self, other) + + yield len(zpool_comparison), None + + for index, dataset_item in enumerate(zpool_comparison.merged): + yield index, self._get_cleanup_from_datasetitem(dataset_item) + + def _get_cleanup_from_datasetitem( + self, dataset_item: ComparisonItemABC, + ) -> typing.Union[None, typing.Generator[TransactionABC, None, None]]: - def get_backup_transactions(self, other: ZpoolABC) -> TransactionListABC: + if dataset_item.get_item().subname in self._config["ignore"]: + return + if dataset_item.a is None or dataset_item.b is None: + return + + dataset_comparison = Comparison.from_datasets(dataset_item.a, dataset_item.b) + snapshots = dataset_comparison.a_overlap_tail[: -self._config["keep_snapshots"]] + + return (snapshot.get_cleanup_transaction() for snapshot in snapshots) + + def get_backup_transactions(self, other: ZpoolABC,) -> TransactionListABC: assert self.side == "source" assert other.side == "target" @@ -122,58 +151,116 @@ def get_backup_transactions(self, other: ZpoolABC) -> TransactionListABC: transactions = TransactionList() for dataset_item in zpool_comparison.merged: - - if dataset_item.get_item().subname in self._config["ignore"]: - continue - if dataset_item.a is None: + backup_transactions = self._get_backup_transactions_from_datasetitem( + other, dataset_item + ) + if backup_transactions is None: continue + transactions.extend(backup_transactions) - if dataset_item.b is None: - snapshots = list(dataset_item.a.snapshots) - else: - dataset_comparison = Comparison.from_datasets( - dataset_item.a, dataset_item.b - ) - snapshots = dataset_comparison.a_head + return transactions - if len(snapshots) == 0: - continue + def generate_backup_transactions( + self, other: ZpoolABC, + ) -> typing.Generator[ + typing.Tuple[ + int, + typing.Union[ + None, typing.Union[None, typing.Generator[TransactionABC, None, None]] + ], + ], + None, + None, + ]: - source_dataset = ( - self.root - if len(dataset_item.a.subname) == 0 - else join(self.root, dataset_item.a.subname) - ) - target_dataset = ( - other.root - if len(dataset_item.a.subname) == 0 - else join(other.root, dataset_item.a.subname) + assert self.side == "source" + assert other.side == "target" + + zpool_comparison = Comparison.from_zpools(self, other) + + yield len(zpool_comparison), None + + for index, dataset_item in enumerate(zpool_comparison.merged): + yield index, self._get_backup_transactions_from_datasetitem( + other, dataset_item ) - transactions.extend( - ( - snapshot.get_backup_transaction(source_dataset, target_dataset,) - for snapshot in snapshots - ) + def _get_backup_transactions_from_datasetitem( + self, other: ZpoolABC, dataset_item: ComparisonItemABC, + ) -> typing.Union[None, typing.Generator[TransactionABC, None, None]]: + + if dataset_item.get_item().subname in self._config["ignore"]: + return + if dataset_item.a is None: + return + + if dataset_item.b is None: + snapshots = list(dataset_item.a.snapshots) + else: + dataset_comparison = Comparison.from_datasets( + dataset_item.a, dataset_item.b ) + snapshots = dataset_comparison.a_head - return transactions + if len(snapshots) == 0: + return + + source_dataset = ( + self.root + if len(dataset_item.a.subname) == 0 + else join(self.root, dataset_item.a.subname) + ) + target_dataset = ( + other.root + if len(dataset_item.a.subname) == 0 + else join(other.root, dataset_item.a.subname) + ) + + return ( + snapshot.get_backup_transaction(source_dataset, target_dataset,) + for snapshot in snapshots + ) def get_snapshot_transactions(self) -> TransactionListABC: assert self._side == "source" transactions = TransactionList() + for dataset in self._datasets: - if dataset.subname in self._config["ignore"]: + transaction = self._get_snapshot_transactions_from_dataset(dataset) + if transaction is None: continue - if dataset["mountpoint"].value is None: - continue - if dataset.changed: - transactions.append(dataset.get_snapshot_transaction()) + transactions.append(dataset.get_snapshot_transaction()) return transactions + def generate_snapshot_transactions( + self, + ) -> typing.Generator[ + typing.Tuple[int, typing.Union[None, TransactionABC]], None, None + ]: + + assert self._side == "source" + + yield len(self._datasets), None + + for index, dataset in enumerate(self._datasets): + yield index, self._get_snapshot_transactions_from_dataset(dataset) + + def _get_snapshot_transactions_from_dataset( + self, dataset: DatasetABC + ) -> typing.Union[None, TransactionABC]: + + if dataset.subname in self._config["ignore"]: + return + if dataset["mountpoint"].value is None: + return + if not dataset.changed: + return + + return dataset.get_snapshot_transaction() + def print_table(self): table = [] @@ -185,7 +272,7 @@ def print_table(self): print( tabulate( table, - headers=("NAME", "USED", "REFER", "compressratio"), + headers=(t("NAME"), t("USED"), t("REFER"), t("compressratio")), tablefmt="github", colalign=("left", "right", "right", "decimal"), ) @@ -222,7 +309,11 @@ def print_comparison_table(self, other: ZpoolABC): table.append(self._comparison_table_row(snapshot_item)) print( - tabulate(table, headers=["NAME", self.side, other.side], tablefmt="github",) + tabulate( + table, + headers=[t("NAME"), t(self.side), t(other.side)], + tablefmt="github", + ) ) @staticmethod @@ -247,7 +338,7 @@ def _comparison_table_row(item: ComparisonItemABC) -> typing.List[str]: ] @staticmethod - def available(side: str, config: typing.Dict,) -> int: + def available(side: str, config: ConfigABC,) -> int: output, _ = Command.on_side( [ @@ -265,7 +356,7 @@ def available(side: str, config: typing.Dict,) -> int: return Property.from_params(*output.strip().split("\t")[1:]).value @classmethod - def from_config(cls, side: str, config: typing.Dict,) -> ZpoolABC: + def from_config(cls, side: str, config: ConfigABC,) -> ZpoolABC: output, _ = Command.on_side( [ diff --git a/src/abgleich/gui/__init__.py b/src/abgleich/gui/__init__.py new file mode 100644 index 0000000..d82915c --- /dev/null +++ b/src/abgleich/gui/__init__.py @@ -0,0 +1,25 @@ +# -*- coding: utf-8 -*- + +""" + +ABGLEICH +zfs sync tool +https://github.com/pleiszenburg/abgleich + + src/abgleich/gui/__init__.py: GUI package root + + Copyright (C) 2019-2020 Sebastian M. Ernst + + +The contents of this file are subject to the GNU Lesser General Public License +Version 2.1 ("LGPL" or "License"). You may not use this file except in +compliance with the License. You may obtain a copy of the License at +https://www.gnu.org/licenses/old-licenses/lgpl-2.1.txt +https://github.com/pleiszenburg/abgleich/blob/master/LICENSE + +Software distributed under the License is distributed on an "AS IS" basis, +WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License for the +specific language governing rights and limitations under the License. + + +""" diff --git a/src/abgleich/gui/abc.py b/src/abgleich/gui/abc.py new file mode 100644 index 0000000..c73b29c --- /dev/null +++ b/src/abgleich/gui/abc.py @@ -0,0 +1,35 @@ +# -*- coding: utf-8 -*- + +""" + +ABGLEICH +zfs sync tool +https://github.com/pleiszenburg/abgleich + + src/abgleich/gui/abc.py: Abstract base classes + + Copyright (C) 2019-2020 Sebastian M. Ernst + + +The contents of this file are subject to the GNU Lesser General Public License +Version 2.1 ("LGPL" or "License"). You may not use this file except in +compliance with the License. You may obtain a copy of the License at +https://www.gnu.org/licenses/old-licenses/lgpl-2.1.txt +https://github.com/pleiszenburg/abgleich/blob/master/LICENSE + +Software distributed under the License is distributed on an "AS IS" basis, +WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License for the +specific language governing rights and limitations under the License. + + +""" + +# +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ +# IMPORT +# +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ + +import abc + +# +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ +# CLASSES +# +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ diff --git a/src/abgleich/gui/lib.py b/src/abgleich/gui/lib.py new file mode 100644 index 0000000..6df61bc --- /dev/null +++ b/src/abgleich/gui/lib.py @@ -0,0 +1,50 @@ +# -*- coding: utf-8 -*- + +""" + +ABGLEICH +zfs sync tool +https://github.com/pleiszenburg/abgleich + + src/abgleich/gui/lib.py: gui library + + Copyright (C) 2019-2020 Sebastian M. Ernst + + +The contents of this file are subject to the GNU Lesser General Public License +Version 2.1 ("LGPL" or "License"). You may not use this file except in +compliance with the License. You may obtain a copy of the License at +https://www.gnu.org/licenses/old-licenses/lgpl-2.1.txt +https://github.com/pleiszenburg/abgleich/blob/master/LICENSE + +Software distributed under the License is distributed on an "AS IS" basis, +WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License for the +specific language governing rights and limitations under the License. + + +""" + +# +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ +# IMPORT +# +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ + +import typing +import sys + +from PyQt5.QtWidgets import QApplication, QDialog +import typeguard + +from ..core.abc import ConfigABC + +# +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ +# ROUTINES +# +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ + + +@typeguard.typechecked +def run_app(Window: typing.Type[QDialog], config: ConfigABC): + + app = QApplication(sys.argv) + window = Window(config) + window.show() + sys.exit(app.exec_()) diff --git a/src/abgleich/gui/transaction.py b/src/abgleich/gui/transaction.py new file mode 100644 index 0000000..a94f1d9 --- /dev/null +++ b/src/abgleich/gui/transaction.py @@ -0,0 +1,130 @@ +# -*- coding: utf-8 -*- + +""" + +ABGLEICH +zfs sync tool +https://github.com/pleiszenburg/abgleich + + src/abgleich/gui/transaction.py: ZFS transactions + + Copyright (C) 2019-2020 Sebastian M. Ernst + + +The contents of this file are subject to the GNU Lesser General Public License +Version 2.1 ("LGPL" or "License"). You may not use this file except in +compliance with the License. You may obtain a copy of the License at +https://www.gnu.org/licenses/old-licenses/lgpl-2.1.txt +https://github.com/pleiszenburg/abgleich/blob/master/LICENSE + +Software distributed under the License is distributed on an "AS IS" basis, +WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License for the +specific language governing rights and limitations under the License. + + +""" + +# +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ +# IMPORT +# +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ + +import typing + +import typeguard +from PyQt5.QtCore import QAbstractTableModel, QModelIndex, Qt +from PyQt5.QtGui import QColor + +from ..core.abc import TransactionListABC +from ..core.io import humanize_size +from ..core.i18n import t + +# +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ +# CLASS +# +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ + + +@typeguard.typechecked +class TransactionListModel(QAbstractTableModel): + def __init__( + self, transactions: TransactionListABC, parent_changed: typing.Callable + ): + + super().__init__() + self._transactions = transactions + self._transactions.changed = self._transactions_changed + self._parent_changed = parent_changed + + self._rows, self._cols = None, None + self._update_labels() + + def data( + self, index: QModelIndex, role: int + ) -> typing.Union[None, str, QColor]: # TODO return type + + row, col = index.row(), index.column() + col_key = self._cols[col] + + if role == Qt.DisplayRole: + if col_key == t("written"): + return humanize_size(self._transactions[row].meta[col_key]) + return self._transactions[row].meta[col_key] + + if role == Qt.ForegroundRole: + if col_key != t("written"): + return + return QColor("#808080") + + if role == Qt.BackgroundRole: + if col_key != t("written"): + return + return QColor( + humanize_size(self._transactions[row].meta[col_key], get_rgb=True) + ) + + if role == Qt.DecorationRole: + if col_key != t("type"): + return + if self._transactions[row].error is not None: + return QColor("#FF0000") + if self._transactions[row].running: + return QColor("#FFFF00") + if self._transactions[row].complete: + return QColor("#00FF00") + return QColor("#0000FF") + + def headerData( + self, section: int, orientation: Qt.Orientation, role: int + ) -> typing.Union[None, str]: + + if role == Qt.DisplayRole: + + if orientation == Qt.Horizontal: + return self._cols[section] + + if orientation == Qt.Vertical: + return self._rows[section] + + def rowCount(self, index: QModelIndex) -> int: + + return len(self._transactions) + + def columnCount(self, index: QModelIndex) -> int: + + return len(self._cols) + + def _transactions_changed(self, row: typing.Union[None, int] = None): + + old_rows, old_cols = self._rows, self._cols + self._update_labels() + if old_rows != self._rows or old_cols != self._cols: + self.layoutChanged.emit() + if row is not None: + self.dataChanged.emit( + self.index(row, 0), self.index(row, len(self._cols) - 1) + ) + self._parent_changed() + + def _update_labels(self): + + self._rows = self._transactions.table_rows + self._cols = self._transactions.table_columns diff --git a/src/abgleich/gui/wizard.py b/src/abgleich/gui/wizard.py new file mode 100644 index 0000000..44d0476 --- /dev/null +++ b/src/abgleich/gui/wizard.py @@ -0,0 +1,241 @@ +# -*- coding: utf-8 -*- + +""" + +ABGLEICH +zfs sync tool +https://github.com/pleiszenburg/abgleich + + src/abgleich/gui/wizard.py: wizard gui + + Copyright (C) 2019-2020 Sebastian M. Ernst + + +The contents of this file are subject to the GNU Lesser General Public License +Version 2.1 ("LGPL" or "License"). You may not use this file except in +compliance with the License. You may obtain a copy of the License at +https://www.gnu.org/licenses/old-licenses/lgpl-2.1.txt +https://github.com/pleiszenburg/abgleich/blob/master/LICENSE + +Software distributed under the License is distributed on an "AS IS" basis, +WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License for the +specific language governing rights and limitations under the License. + + +""" + +# +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ +# IMPORT +# +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ + +import os + +from PyQt5.QtGui import QIcon +from PyQt5.QtWidgets import QApplication, QMessageBox +from typeguard import typechecked + +from .transaction import TransactionListModel +from .wizard_base import WizardUiBase +from ..core.abc import ConfigABC +from ..core.transaction import TransactionList +from ..core.i18n import t +from ..core.zpool import Zpool + +# +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ +# CLASS +# +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ + + +@typechecked +class WizardUi(WizardUiBase): + def __init__(self, config: ConfigABC): + + super().__init__() + self._config = config + + self.setWindowIcon( + QIcon(os.path.join(os.path.dirname(__file__), "..", "share", "icon.svg")) + ) + self.setWindowTitle(t("abgleich wizard")) + + self._ui["button_cancel"].setEnabled(False) + self._ui["button_continue"].setEnabled(False) + + self._ui["button_cancel"].clicked.connect(self._cancel_click) + self._ui["button_continue"].clicked.connect(self._continue_click) + + self._ui["button_cancel"].setText(t("Cancel")) + self._ui["button_continue"].setText("") + + self._continue = lambda: None + + self._transactions = TransactionList() + self._model = TransactionListModel(self._transactions, self._changed) + self._ui["table"].setModel(self._model) + + self._steps = [ + { + "init_button": t("Collect Snapshot Transactions"), + "prepare": self._prepare_snap, + "prepare_text": t("Collecting snapshot transactions ..."), + "run_text": t("Creating snapshots ..."), + "run_button": t("Execute Snapshot Transactions"), + "finish_text": t("Snapshots created."), + "finish_button": t("Collect Backup Transactions"), + }, + { + "prepare": lambda: self._prepare("backup"), + "prepare_text": t("Collecting backup transactions ..."), + "run_text": t("Transferring snapshots ..."), + "run_button": t("Execute Backup Transactions"), + "finish_text": t("Snapshots transferred."), + "finish_button": t("Collect Cleanup Transactions"), + }, + { + "prepare": lambda: self._prepare("cleanup"), + "prepare_text": t("Collecting cleanup transactions ..."), + "run_text": t("Removing old snapshots ..."), + "run_button": t("Execute Cleanup Transactions"), + "finish_text": t("Old snapshots removed."), + "finish_button": t("Close"), + }, + ] + + self._init_step(0) + + def _changed(self): + + self._ui["table"].resizeColumnsToContents() + QApplication.processEvents() + + def _continue_click(self): + + self._continue() + + def _cancel_click(self): + + self._quit() + + def _init_step(self, index: int): + + self._ui["button_cancel"].setEnabled(False) + self._ui["button_continue"].setEnabled(False) + self._ui["progress"].setValue(0) + self._ui["label"].setText(self._steps[index]["prepare_text"]) + self._transactions.clear() + + if index > 0: + self._prepare_step(index) + return + + self._continue = lambda: self._prepare_step(index) + self._ui["button_continue"].setText(self._steps[index]["init_button"]) + self._ui["button_cancel"].setEnabled(True) + self._ui["button_continue"].setEnabled(True) + + def _prepare_step(self, index: int): + + self._ui["button_cancel"].setEnabled(False) + self._ui["button_continue"].setEnabled(False) + QApplication.processEvents() + + if not self._steps[index]["prepare"](): + QMessageBox.warning(self, t("Warning"), t("Nothing to do.")) + self._finish_step(index) + return + + self._ui["label"].setText(self._steps[index]["run_text"]) + self._continue = lambda: self._run_step(index) + self._ui["button_continue"].setText(self._steps[index]["run_button"]) + self._ui["button_cancel"].setEnabled(True) + self._ui["button_continue"].setEnabled(True) + + def _run_step(self, index: int): + + self._ui["button_cancel"].setEnabled(False) + self._ui["button_continue"].setEnabled(False) + self._ui["progress"].setMaximum(len(self._transactions)) + QApplication.processEvents() + + for number, transaction in enumerate(self._transactions): + + assert not transaction.running + assert not transaction.complete + + transaction.run() + self._ui["progress"].setValue(number + 1) + QApplication.processEvents() + + assert not transaction.running + assert transaction.complete + + if transaction.error is not None: + QMessageBox.critical( + self, + t("Critical Error"), + t("Transaction failed!") + + "\n\n" + + "\n\n".join([str(item) for item in transaction.error.args]), + ) + self._quit() + return + + self._ui["label"].setText(self._steps[index]["finish_text"]) + self._continue = lambda: self._finish_step(index) + + if index + 1 == len(self._steps): + self._ui["button_cancel"].setVisible(False) + else: + self._ui["button_cancel"].setEnabled(True) + self._ui["button_continue"].setText(self._steps[index]["finish_button"]) + self._ui["button_continue"].setEnabled(True) + + def _finish_step(self, index: int): + + if index + 1 == len(self._steps): + self._quit() + return + + self._init_step(index + 1) + + def _prepare_snap(self): + + zpool = Zpool.from_config("source", config=self._config) + + gen = zpool.generate_snapshot_transactions() + length, _ = next(gen) + + self._ui["progress"].setMaximum(length) + QApplication.processEvents() + + for number, transaction in gen: + if transaction is not None: + self._transactions.append(transaction) + self._ui["progress"].setValue(number + 1) + QApplication.processEvents() + + return len(self._transactions) > 0 + + def _prepare(self, action: str): + + source_zpool = Zpool.from_config("source", config=self._config) + target_zpool = Zpool.from_config("target", config=self._config) + + gen = getattr(source_zpool, f"generate_{action:s}_transactions")(target_zpool) + length, _ = next(gen) + + self._ui["progress"].setMaximum(length) + QApplication.processEvents() + + for number, transactions in gen: + if transactions is not None: + self._transactions.extend(transactions) + self._ui["progress"].setValue(number + 1) + QApplication.processEvents() + + return len(self._transactions) > 0 + + def _quit(self): + + self._transactions.clear() + self.close() diff --git a/src/abgleich/gui/wizard_base.py b/src/abgleich/gui/wizard_base.py new file mode 100644 index 0000000..f15e1ab --- /dev/null +++ b/src/abgleich/gui/wizard_base.py @@ -0,0 +1,73 @@ +# -*- coding: utf-8 -*- + +""" + +ABGLEICH +zfs sync tool +https://github.com/pleiszenburg/abgleich + + src/abgleich/gui/wizard_base.py: wizard gui base + + Copyright (C) 2019-2020 Sebastian M. Ernst + + +The contents of this file are subject to the GNU Lesser General Public License +Version 2.1 ("LGPL" or "License"). You may not use this file except in +compliance with the License. You may obtain a copy of the License at +https://www.gnu.org/licenses/old-licenses/lgpl-2.1.txt +https://github.com/pleiszenburg/abgleich/blob/master/LICENSE + +Software distributed under the License is distributed on an "AS IS" basis, +WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License for the +specific language governing rights and limitations under the License. + + +""" + +# +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ +# IMPORT +# +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ + +from PyQt5.QtWidgets import ( + QDialog, + QHBoxLayout, + QLabel, + QProgressBar, + QPushButton, + QTableView, + QVBoxLayout, +) +from typeguard import typechecked + +# +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ +# CLASS +# +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ + + +@typechecked +class WizardUiBase(QDialog): + def __init__(self): + + super().__init__() # skip WizardUiBaseABC + + self._ui = { + "layout_0_v_root": QVBoxLayout(), # dialog + "layout_1_h_buttons": QHBoxLayout(), # for buttons + "label": QLabel(), + "table": QTableView(), + "progress": QProgressBar(), + "button_continue": QPushButton(), + "button_cancel": QPushButton(), + } + self.setLayout(self._ui["layout_0_v_root"]) + + self._ui["layout_0_v_root"].addWidget(self._ui["label"]) + self._ui["layout_0_v_root"].addWidget(self._ui["table"]) + self._ui["layout_0_v_root"].addWidget(self._ui["progress"]) + self._ui["layout_1_h_buttons"].addWidget(self._ui["button_continue"]) + self._ui["layout_1_h_buttons"].addWidget(self._ui["button_cancel"]) + self._ui["layout_0_v_root"].addLayout(self._ui["layout_1_h_buttons"]) + + self._ui["button_cancel"].setDefault(True) + + self.resize(1000, 700) diff --git a/src/abgleich/share/icon.svg b/src/abgleich/share/icon.svg new file mode 100644 index 0000000..4bdb0c0 --- /dev/null +++ b/src/abgleich/share/icon.svg @@ -0,0 +1,24 @@ + + + + + + + + + + + + + diff --git a/src/abgleich/share/translations.yaml b/src/abgleich/share/translations.yaml new file mode 100644 index 0000000..ae0b469 --- /dev/null +++ b/src/abgleich/share/translations.yaml @@ -0,0 +1,78 @@ +Aborted!: + de: Abgebrochen! +Cancel: + de: Abbrechen +Close: + de: Schließen +Collect Backup Transactions: + de: Sicherungstransaktionen sammeln +Collect Cleanup Transactions: + de: Säuberungstransaktionen sammeln +Collect Snapshot Transactions: + de: Schnappschuss-Transaktionen sammeln +Collecting backup transactions ...: + de: Sicherungstransaktionen sammeln ... +Collecting cleanup transactions ...: + de: Säuberungstransaktionen sammeln ... +Collecting snapshot transactions ...: + de: Schnappschuss-Transaktionen sammeln ... +Creating snapshots ...: + de: Schnappschüsse erstellen ... +Do you want to continue?: + de: Möchten Sie fortfahren? +Execute Backup Transactions: + de: Sicherungstransaktionen ausführen +Execute Cleanup Transactions: + de: Säuberungstransaktionen ausführen +Execute Snapshot Transactions: + de: Schnappschuss-Transaktionen ausführen +FAILED: + de: FEHLGESCHLAGEN +NAME: {} +OK: {} +Old snapshots removed.: + de: Alte Schnappschüsse entfernt. +REFER: + de: VERWEIST +Removing old snapshots ...: + de: Entferne alte Schnappschüsse ... +Snapshots created.: + de: Schnappschüsse erstellt. +Snapshots transferred.: + de: Schnappschüsse gesichert. +Transferring snapshots ...: + de: Sichere Schnappschüsse ... +USED: + de: BELEGT +abgleich wizard: + de: Abgleich Assistent +ancestor_name: + de: Vorfahr +cleanup_snapshot: + de: Säuberung +compressratio: + de: Kompressionsrate +dataset_subname: + de: Datensatz-Namensfragment +nothing to do: + de: nichts zu tun +snapshot: + de: Schnappschuss +snapshot_name: + de: Schnappschuss-Name +snapshot_subparent: + de: Datensatz-Namensfragment +source: + de: Quelle +target: + de: Ziel +transaction: + de: Transaktion +transfer_snapshot: + de: Sicherung +transfer_snapshot_incremental: + de: Inkrementelle Sicherung +type: + de: Typ +written: + de: Geschrieben