diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 0000000..2be131e --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,43 @@ +# Contributing + +I've added a good amount of doxygen style comments and structured the code in a way that +should make this easier. Happy to provide feedback on PRs. + +## Modular and extendable design + +Here are some jump-off points where you can start adding extensions: + +- additional [condition checks](src/chia_log/handlers/harvester_activity_handler.py) e.g. time since last eligible plot +- additional [parsers](src/chia_log/parsers) to monitor other parts of the log output (e.g. timelord) +- additional [log consumers](src/chia_log/log_consumer.py) to support watchdog over the network (NetworkLogConsumer) +- additional [notifiers](src/notifier) to support notifications over Slack, WhatsApp, E-mail, etc. + +## Further feature ideas + +- Handlers for automated recovery + - auto-restart chia processes if node cannot re-sync + - auto-restart xHCI devices to recover external HDDs + +## Formatting and linting + +Before submitting a PR make sure that your feature is covered with tests. + +Install dependencies for auto-formatting and linting: + +``` +pip3 install black flake8 mypy +``` + +Run formatting, type checking and linting: + +``` +black src tests && mypy src tests && flake8 src tests +``` + +Run all tests: + +``` +PUSHOVER_API_TOKEN= PUSHOVER_USER_KEY= python3 -m unittest +``` + +Check that all passes and there aren't any warnings. diff --git a/README.md b/README.md index ede1072..0ebb827 100644 --- a/README.md +++ b/README.md @@ -9,31 +9,61 @@ # Watchdog for your Chia farm -So you've become a [Chia](https://www.chia.net) farmer and are now looking to improve the uptime of your farm? How about -some automated monitoring that sends notifications directly to your phone? +So you've become a [Chia](https://www.chia.net) farmer and want to maximize the probability of getting a reward? +Chiadog helps with automated monitoring and sends you a mobile notification in case something appears to malfunction. -This tool parses the `debug.log` generated by the chia process and runs various checks to determine the health of your -farming operations. It can detect if your node has lost sync and the farmer is no longer participating in challenges, or -if one of your external HDDs disconnected and your harvester doesn't have access to the full amount of plots you have. +## Supported Notifications + +| Subsystem | Notification (example values) | Priority | +| ------------- | ------------- | ------| +| Harvester | No keep-alive events from harvester for the past 400 seconds. Your harvester appears to be offline! | HIGH | +| Harvester | The total plot count decreased from 100 to 40. | HIGH | +| Harvester | Harvester did not participate in any challenge for 120 seconds. This might indicate networking issues. | NORMAL | +| Harvester | Seeking plots took too long: 40 seconds! | NORMAL | +| Harvester | Found 1 proof(s)! | LOW | -## List of checked conditions: +## How it works? -- Time since last eligible farming event -- Non-decreasing number of total plots farmed -- Quick plot seeking time when responding to a challenge -- Whether a challenge proof was found +It parses the `debug.log` generated by the chia process and runs various checks to determine the health of your +farmer. Among others, it can detect if your node has lost sync and the farmer is no longer participating in challenges, or +if one of your external HDDs disconnected and your harvester doesn't have access to the full amount of plots you have. -## Smartphone Notifications +## Supported Integrations for Notifications -In case of deviations from the expected values, notifications with low, normal, or high priority can be triggered. -Currently the tool supports integration with [Pushover](https://pushover.net/) -which is available for both Android and iOS. High priority notifications can be configured from the Pushover app to +### Pushover + +[Pushover](https://pushover.net/) is available for both Android and iOS. High priority notifications can be configured from the Pushover app to overwrite any Silence or Do-Not-Disturb modes on your phone and sound a loud alarm at any time of the day to make you aware of any issues in a timely manner. +### Slack, WhatsApp, E-Mail, ...? + +These integrations can be easily added. Contributions are welcome! + # Getting started -**So far only tested on Ubuntu 20.04. Requires Python 3.7+.** +## Pre-requisites +- UNIX-based OS (Linux/Mac) - contributions to support Windows are welcome! +- Python 3.7+ +- Enabled `INFO` logs on your chia farmer + +### How to enable INFO logs on chia farmer? + +First configure the log level to `INFO`. This ensures that all logs necessary for chiadog +to operate are available under `~/.chia/mainnet/log/debug.log`. +``` +chia configure -log-level=INFO +``` +Then restart your farmer to apply the changes: +``` +chia start --restart farmer +``` +Check that logs are coming in: +``` +cat ~/.chia/mainnet/log/debug.log +``` + +## Running Chiadog 1. Clone the repository @@ -93,22 +123,6 @@ phone and the tests should complete with an `OK` status: You can also enable more verbose logging from `config.yaml` by changing `INFO` to `DEBUG`. Then you'll be able to see logs for every keep-alive event from the harvester. -# Extending & Contributing - -Contributions are welcome, I've added a good amount of doxygen style comments and structured the code in a way that -should make this easier. Happy to provide feedback on PRs. - -## Modular and extendable design - -Here are some jump-off points where you can start adding extensions: - -- additional [condition checks](src/chia_log/handlers/harvester_activity_handler.py) e.g. time since last eligible plot -- additional [parsers](src/chia_log/parsers) to monitor other parts of the log output (e.g. timelord) -- additional [log consumers](src/chia_log/log_consumer.py) to support watchdog over the network (NetworkLogConsumer) -- additional [notifiers](src/notifier) to support notifications over Slack, WhatsApp, E-mail, etc. - -## Further feature wishes +# Contributing -- Handlers for automated recovery - - auto-restart chia processes if node cannot re-sync - - auto-restart xHCI devices to recover external HDDs \ No newline at end of file +Contributions are always welcome! Please refer to [CONTRIBUTING](CONTRIBUTING.md) documentation. \ No newline at end of file diff --git a/src/chia_log/handlers/harvester_activity_handler.py b/src/chia_log/handlers/harvester_activity_handler.py index 461ff8f..31e596f 100644 --- a/src/chia_log/handlers/harvester_activity_handler.py +++ b/src/chia_log/handlers/harvester_activity_handler.py @@ -150,7 +150,7 @@ class FoundProofs(HarvesterConditionChecker): def check(self, obj: HarvesterActivityMessage) -> Optional[Event]: if obj.found_proofs_count > 0: - message = f"Found {obj.found_proofs_count} proofs!" + message = f"Found {obj.found_proofs_count} proof(s)!" logging.info(message) return Event( type=EventType.USER, priority=EventPriority.LOW, service=EventService.HARVESTER, message=message diff --git a/src/notifier/keep_alive_monitor.py b/src/notifier/keep_alive_monitor.py index 3a3199b..fa1111d 100644 --- a/src/notifier/keep_alive_monitor.py +++ b/src/notifier/keep_alive_monitor.py @@ -63,7 +63,10 @@ def check_last_keep_alive(self): for service in self._last_keep_alive.keys(): seconds_since_last = (datetime.now() - self._last_keep_alive[service]).seconds if seconds_since_last > self._last_keep_alive_threshold_seconds[service]: - message = f"No keep-alive events from harvester for the past {seconds_since_last} seconds" + message = ( + f"No keep-alive events from harvester for the past {seconds_since_last} seconds. " + f"Your harvester appears to be offline!" + ) logging.warning(message) events.append( Event(