diff --git a/.github/ISSUE_TEMPLATE/bug_report.md b/.github/ISSUE_TEMPLATE/bug_report.md new file mode 100644 index 00000000..7bc3e188 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/bug_report.md @@ -0,0 +1,38 @@ +--- +name: Bug report +about: Create a report to help us improve +title: "[BUG]" +labels: '' +assignees: w1ld3r + +--- + +**Describe the bug** +A clear and concise description of what the bug is. + +**To Reproduce** +Steps to reproduce the behavior: +1. git clone '...' +2. docker build -t dev/crypto-signals:latest . +3. docker run '....' +4. Error + +**Expected behavior** +A clear and concise description of what you expected to happen. + +**config.yml** +```yml +settings: +log_level: INFO +[...] +``` + +**Machine** + - OS name and version + - crypto-signal branch + - Docker version + +**Additional context** +Add any other context about the problem here. +For example if any modification of the code as been made + diff --git a/.github/ISSUE_TEMPLATE/feature_request.md b/.github/ISSUE_TEMPLATE/feature_request.md new file mode 100644 index 00000000..3f212ccc --- /dev/null +++ b/.github/ISSUE_TEMPLATE/feature_request.md @@ -0,0 +1,19 @@ +--- +name: Feature request +about: Suggest an idea for this project +title: "[FEATURE] ..." +labels: '' + +--- + +**Is your feature request related to a problem? Please describe.** +A clear and concise description of what the problem is. Ex. I'm always frustrated when [...] + +**Describe the solution you'd like** +A clear and concise description of what you want to happen. + +**Describe alternatives you've considered** +A clear and concise description of any alternative solutions or features you've considered. + +**Additional context** +Add any other context or screenshots about the feature request here. diff --git a/Dockerfile b/Dockerfile index 18118f1e..19cac2d5 100644 --- a/Dockerfile +++ b/Dockerfile @@ -1,4 +1,4 @@ -FROM python:3.8-buster +FROM python:latest # TA-lib is required by the python TA-lib wrapper. This provides analysis. COPY lib/ta-lib-0.4.0-src.tar.gz /tmp/ta-lib-0.4.0-src.tar.gz @@ -10,13 +10,11 @@ RUN cd /tmp && \ make && \ make install -ADD app/requirements-step-1.txt /app/requirements-step-1.txt -ADD app/requirements-step-2.txt /app/requirements-step-2.txt +COPY ./app /app + WORKDIR /app -# Pip doesn't install requirements sequentially. -# To ensure pre-reqs are installed in the correct -# order they have been split into two files +RUN pip install --upgrade pip RUN pip install -r requirements-step-1.txt RUN pip install -r requirements-step-2.txt diff --git a/README.md b/README.md index be368b93..db759897 100644 --- a/README.md +++ b/README.md @@ -19,11 +19,11 @@ Development branch to testing new features. This develop version has a lot of im ## Installing And Running -Because this is a development branch you need to build your custom Docker image. The commands listed below are intended to be run in a terminal. +The commands listed below are intended to be run in a terminal. Be sure you have git installed in your system. -1. Clone this repo `git clone https://github.com/CryptoSignal/crypto-signal.git` +1. Clone this repo `git clone https://github.com/w1ld3r/crypto-signal.git` 1. Enter to cripto-signal folder `cd crypto-signal` @@ -31,11 +31,7 @@ Be sure you have git installed in your system. 1. Create a config.yml file and put it into "app" folder. -1. Build your own Docker image, for example, `docker build -t dev/crypto-signals:latest .` - -1. For testing and debugging run docker with "-t" option `docker run --rm -ti -v $PWD/app:/app dev/crypto-signals:latest` - -1. For production run in daemon mode using "-d" option `docker run --rm -di -v $PWD/app:/app dev/crypto-signals:latest` +1. Build and run the docker container: `docker-compose up --build` ### Configuring config.yml @@ -125,7 +121,7 @@ exchanges: #### Show me the price! -If you want prices in your notification messages, you can use the "prices" variable. +If you want prices in your notification messages, you can use the "prices" variable or "price_value". ``` notifiers: @@ -138,6 +134,17 @@ notifiers: template: "[{{analysis.config.candle_period}}] {{market}} {{values}} Prices: [{{prices}}]" ``` +``` +notifiers: + telegram: + required: + token: 791615820:AAGFgGSumWUrb-CyXtGxzAuYaabababababababa + chat_id: 687950000 + optional: + parse_mode: html + template: "[{{analysis.config.candle_period}}] {{market}} {{values}} Price 15m low: [{{price_value['15m'].low}}]" +``` + By the way, to have this feature you need to configure "ohlcv" informant for each candle period of your indicators. ``` @@ -723,8 +730,7 @@ informants: - low - close candle_period: 4h - period_count: 14 - + period_count: 14 ``` Then you can use the "price_value" variable to have the values of prices and be able to do some operations on them. @@ -738,9 +744,9 @@ notifiers: optional: parse_mode: html template: "{{ market }} - BUY {{ price_value.close }} - SL: {{ decimal_format|format(price_value.low * 0.9) }} - TP: {{ decimal_format|format(price_value.close * 1.02) }} {{ decimal_format|format(price_value.close * 1.04) }} " + BUY {{ price_value['1h'].close }} + SL: {{ decimal_format|format(price_value['4h'].low * 0.9) }} + TP: {{ decimal_format|format(price_value['1h'].close * 1.02) }} {{ decimal_format|format(price_value['4h'].close * 1.04) }} " ``` The code for "decimal_format" and "format" is necessary to obtain the prices formatted with the corresponding zeros. diff --git a/app/analyzers/indicators/ichimoku.py b/app/analyzers/indicators/ichimoku.py index cd39a1a7..5e8c4524 100644 --- a/app/analyzers/indicators/ichimoku.py +++ b/app/analyzers/indicators/ichimoku.py @@ -76,8 +76,8 @@ def analyze(self, historical_data, tenkansen_period, kijunsen_period, senkou_spa last_time = dataframe.index[-1] timedelta = dataframe.index[1] - dataframe.index[0] newindex = pandas.date_range(last_time + timedelta, - freq=timedelta, - periods=cloud_displacement) + freq=timedelta, + periods=cloud_displacement) ichimoku_values = ichimoku_values.append( pandas.DataFrame(index=newindex)) # cloud offset @@ -133,4 +133,4 @@ def analyze(self, historical_data, tenkansen_period, kijunsen_period, senkou_spa except Exception as e: print('Error running ichimoku analysis: {}'.format(e)) - return ichimoku_values \ No newline at end of file + return ichimoku_values diff --git a/app/app.py b/app/app.py index 9b67c931..1dafc542 100644 --- a/app/app.py +++ b/app/app.py @@ -50,7 +50,7 @@ def main(): key: market_data[exchange][key] for key in chunk} notifier = Notifier( - config.notifiers, config.indicators, market_data_chunk) + config.notifiers, config.indicators, config.conditionals, market_data_chunk) behaviour = Behaviour(config, exchange_interface, notifier) workerName = "Worker-{}".format(num) diff --git a/app/behaviour.py b/app/behaviour.py index 5a05ac51..f0f64658 100644 --- a/app/behaviour.py +++ b/app/behaviour.py @@ -347,23 +347,25 @@ def _get_crossover_results(self, new_result): if not crossover_conf['enabled']: self.logger.debug("%s is disabled, skipping.", crossover) continue - - key_indicator = new_result[crossover_conf['key_indicator_type'] - ][crossover_conf['key_indicator']][crossover_conf['key_indicator_index']] - crossed_indicator = new_result[crossover_conf['crossed_indicator_type'] - ][crossover_conf['crossed_indicator']][crossover_conf['crossed_indicator_index']] - - crossover_conf['candle_period'] = crossover_conf['key_indicator'] + \ - str(crossover_conf['key_indicator_index']) - - dispatcher_args = { - 'key_indicator': key_indicator['result'], - 'key_signal': crossover_conf['key_signal'], - 'key_indicator_index': crossover_conf['key_indicator_index'], - 'crossed_indicator': crossed_indicator['result'], - 'crossed_signal': crossover_conf['crossed_signal'], - 'crossed_indicator_index': crossover_conf['crossed_indicator_index'] - } + try: + key_indicator = new_result[crossover_conf['key_indicator_type']][crossover_conf['key_indicator']][crossover_conf['key_indicator_index']] + crossed_indicator = new_result[crossover_conf['crossed_indicator_type']][crossover_conf['crossed_indicator']][crossover_conf['crossed_indicator_index']] + + crossover_conf['candle_period'] = crossover_conf['key_indicator'] + \ + str(crossover_conf['key_indicator_index']) + + dispatcher_args = { + 'key_indicator': key_indicator['result'], + 'key_signal': crossover_conf['key_signal'], + 'key_indicator_index': crossover_conf['key_indicator_index'], + 'crossed_indicator': crossed_indicator['result'], + 'crossed_signal': crossover_conf['crossed_signal'], + 'crossed_indicator_index': crossover_conf['crossed_indicator_index'] + } + except Exception as e: + self.logger.warning(e) + self.logger.warning(traceback.format_exc()) + continue results[crossover].append({ 'result': crossover_dispatcher[crossover](**dispatcher_args), diff --git a/app/conf.py b/app/conf.py index fcbd2403..788e3c1c 100644 --- a/app/conf.py +++ b/app/conf.py @@ -58,6 +58,11 @@ def __init__(self): self.exchanges = user_config['exchanges'] else: self.exchanges = dict() + + if 'conditionals' in user_config: + self.conditionals = user_config['conditionals'] + else: + self.conditionals = None for exchange in ccxt.exchanges: if exchange not in self.exchanges: diff --git a/app/defaults.yml b/app/defaults.yml index 2eaa1cad..ac5fda9a 100644 --- a/app/defaults.yml +++ b/app/defaults.yml @@ -36,8 +36,9 @@ notifiers: webhook: null optional: template: "{{exchange}}-{{market}}-{{indicator}}-{{indicator_number}} is {{status}}!{{ '\n' -}}" - gmail: + email: required: + smtp_server: null username: null password: null destination_emails: null diff --git a/app/exchange.py b/app/exchange.py index c4b3aacb..7cf8cbef 100644 --- a/app/exchange.py +++ b/app/exchange.py @@ -32,9 +32,11 @@ def __init__(self, exchange_config): # Loads the exchanges using ccxt. for exchange in exchange_config: if exchange_config[exchange]['required']['enabled']: - new_exchange = getattr(ccxt, exchange)({ - "enableRateLimit": True - }) + parameters = {'enableRateLimit': True} + if 'future' in exchange_config[exchange].keys(): + if exchange_config[exchange]['future'] == True: + parameters['options'] = {'defaultType': 'future'} + new_exchange = getattr(ccxt, exchange)(parameters) # sets up api permissions for user if given if new_exchange: diff --git a/app/notification.py b/app/notification.py index e7d3ab17..418c4387 100755 --- a/app/notification.py +++ b/app/notification.py @@ -4,15 +4,15 @@ import copy import json import os +import re import sys import traceback -from datetime import datetime +import datetime from time import sleep import matplotlib -matplotlib.use('Agg') -import matplotlib.pyplot as plt import matplotlib.dates as mdates +import matplotlib.pyplot as plt import matplotlib.ticker as mticker import numpy as np import pandas as pd @@ -29,19 +29,21 @@ from analyzers.indicators import candle_recognition, ichimoku from analyzers.utils import IndicatorUtils from notifiers.discord_client import DiscordNotifier -from notifiers.gmail_client import GmailNotifier +from notifiers.email_client import EmailNotifier from notifiers.slack_client import SlackNotifier from notifiers.stdout_client import StdoutNotifier from notifiers.telegram_client import TelegramNotifier from notifiers.twilio_client import TwilioNotifier from notifiers.webhook_client import WebhookNotifier +matplotlib.use('Agg') + class Notifier(IndicatorUtils): """Handles sending notifications via the configured notifiers """ - def __init__(self, notifier_config, indicator_config, market_data): + def __init__(self, notifier_config, indicator_config, conditional_config, market_data): """Initializes Notifier class Args: @@ -51,79 +53,97 @@ def __init__(self, notifier_config, indicator_config, market_data): self.logger = structlog.get_logger() self.notifier_config = notifier_config self.indicator_config = indicator_config + self.conditional_config = conditional_config self.market_data = market_data + self.alert_frequencies = {} self.last_analysis = dict() self.enable_charts = False self.all_historical_data = False self.timezone = None self.first_run = False + self.twilio_clients = {} + self.discord_clients = {} + self.slack_clients = {} + self.email_clients = {} + self.telegram_clients = {} + self.webhook_clients = {} + self.stdout_clients = {} enabled_notifiers = list() self.logger = structlog.get_logger() - self.twilio_configured = self._validate_required_config( - 'twilio', notifier_config) - if self.twilio_configured: - self.twilio_client = TwilioNotifier( - twilio_key=notifier_config['twilio']['required']['key'], - twilio_secret=notifier_config['twilio']['required']['secret'], - twilio_sender_number=notifier_config['twilio']['required']['sender_number'], - twilio_receiver_number=notifier_config['twilio']['required']['receiver_number'] - ) - enabled_notifiers.append('twilio') - - self.discord_configured = self._validate_required_config( - 'discord', notifier_config) - if self.discord_configured: - self.discord_client = DiscordNotifier( - webhook=notifier_config['discord']['required']['webhook'], - username=notifier_config['discord']['required']['username'], - avatar=notifier_config['discord']['optional']['avatar'] - ) - enabled_notifiers.append('discord') - - self.slack_configured = self._validate_required_config( - 'slack', notifier_config) - if self.slack_configured: - self.slack_client = SlackNotifier( - slack_webhook=notifier_config['slack']['required']['webhook'] - ) - enabled_notifiers.append('slack') - - self.gmail_configured = self._validate_required_config( - 'gmail', notifier_config) - if self.gmail_configured: - self.gmail_client = GmailNotifier( - username=notifier_config['gmail']['required']['username'], - password=notifier_config['gmail']['required']['password'], - destination_addresses=notifier_config['gmail']['required']['destination_emails'] - ) - enabled_notifiers.append('gmail') - - self.telegram_configured = self._validate_required_config( - 'telegram', notifier_config) - if self.telegram_configured: - self.telegram_client = TelegramNotifier( - token=notifier_config['telegram']['required']['token'], - chat_id=notifier_config['telegram']['required']['chat_id'], - parse_mode=notifier_config['telegram']['optional']['parse_mode'] - ) - enabled_notifiers.append('telegram') - - self.webhook_configured = self._validate_required_config( - 'webhook', notifier_config) - if self.webhook_configured: - self.webhook_client = WebhookNotifier( - url=notifier_config['webhook']['required']['url'], - username=notifier_config['webhook']['optional']['username'], - password=notifier_config['webhook']['optional']['password'] - ) - enabled_notifiers.append('webhook') - - self.stdout_configured = self._validate_required_config( - 'stdout', notifier_config) - if self.stdout_configured: - self.stdout_client = StdoutNotifier() - enabled_notifiers.append('stdout') + for notifier in notifier_config.keys(): + if notifier.startswith('twilio'): + self.twilio_configured = self._validate_required_config( + notifier, notifier_config) + if self.twilio_configured: + self.twilio_clients[notifier] = TwilioNotifier( + twilio_key=notifier_config[notifier]['required']['key'], + twilio_secret=notifier_config[notifier]['required']['secret'], + twilio_sender_number=notifier_config[notifier]['required']['sender_number'], + twilio_receiver_number=notifier_config[notifier]['required']['receiver_number'] + ) + enabled_notifiers.append(notifier) + + if notifier.startswith('discord'): + self.discord_configured = self._validate_required_config( + notifier, notifier_config) + if self.discord_configured: + self.discord_clients[notifier] = DiscordNotifier( + webhook=notifier_config[notifier]['required']['webhook'], + username=notifier_config[notifier]['required']['username'], + avatar=notifier_config[notifier]['optional']['avatar'] + ) + enabled_notifiers.append(notifier) + + if notifier.startswith('slack'): + self.slack_configured = self._validate_required_config( + notifier, notifier_config) + if self.slack_configured: + self.slack_clients[notifier] = SlackNotifier( + slack_webhook=notifier_config[notifier]['required']['webhook'] + ) + enabled_notifiers.append(notifier) + + if notifier.startswith('email'): + self.email_configured = self._validate_required_config( + notifier, notifier_config) + if self.email_configured: + self.email_clients[notifier] = EmailNotifier( + smtp_server=notifier_config[notifier]['required']['smtp_server'], + username=notifier_config[notifier]['required']['username'], + password=notifier_config[notifier]['required']['password'], + destination_addresses=notifier_config[notifier]['required']['destination_emails'] + ) + enabled_notifiers.append(notifier) + + if notifier.startswith('telegram'): + self.telegram_configured = self._validate_required_config( + notifier, notifier_config) + if self.telegram_configured: + self.telegram_clients[notifier] = TelegramNotifier( + token=notifier_config[notifier]['required']['token'], + chat_id=notifier_config[notifier]['required']['chat_id'], + parse_mode=notifier_config[notifier]['optional']['parse_mode'] + ) + enabled_notifiers.append(notifier) + + if notifier.startswith('webhook'): + self.webhook_configured = self._validate_required_config( + notifier, notifier_config) + if self.webhook_configured: + self.webhook_clients[notifier] = WebhookNotifier( + url=notifier_config[notifier]['required']['url'], + username=notifier_config[notifier]['optional']['username'], + password=notifier_config[notifier]['optional']['password'] + ) + enabled_notifiers.append(notifier) + + if notifier.startswith('stdout'): + self.stdout_configured = self._validate_required_config( + notifier, notifier_config) + if self.stdout_configured: + self.stdout_clients[notifier] = StdoutNotifier() + enabled_notifiers.append(notifier) self.logger.info('enabled notifers: %s', enabled_notifiers) @@ -147,13 +167,78 @@ def notify_all(self, new_analysis): for market_pair in messages[exchange]: _messages = messages[exchange][market_pair] - for candle_period in _messages: - if not isinstance(_messages[candle_period], list) or len(_messages[candle_period]) == 0: - continue + if self.conditional_config: + self.notify_conditional(exchange, market_pair, _messages) + else: + for candle_period in _messages: + if not isinstance(_messages[candle_period], list) or len(_messages[candle_period]) == 0: + continue - self.notify_all_messages( - exchange, market_pair, candle_period, _messages[candle_period]) - sleep(4) + self.notify_all_messages( + exchange, market_pair, candle_period, _messages[candle_period]) + sleep(4) + + if self.first_run: + self.first_run = False + + def notify_conditional(self, exchange, market_pair, messages): + status = ['hot', 'cold'] + + for condition in self.conditional_config: + c_nb_conditions = 0 + c_nb_once_muted = 0 + c_nb_new_status = 0 + nb_conditions = 0 + should_alert = False + new_message = {} + new_message['values'] = [] + new_message['indicator'] = [] + new_message['price_value'] = {} + + for stat in list(set(status) & set(condition.keys())): + nb_conditions += len(condition[stat]) + + for candle_period in messages: + if messages[candle_period]: + new_message['exchange'] = messages[candle_period][0]['exchange'] + new_message['market'] = messages[candle_period][0]['market'] + new_message['base_currency'] = messages[candle_period][0]['base_currency'] + new_message['quote_currency'] = messages[candle_period][0]['quote_currency'] + new_message['prices'] = messages[candle_period][0]['prices'] + new_message['price_value'][candle_period] = messages[candle_period][0]['price_value'] + new_message['decimal_format'] = messages[candle_period][0]['decimal_format'] + for msg in messages[candle_period]: + alert_frequency = msg['analysis']['config']['alert_frequency'] + for stat in status: + if msg['status'] == stat and stat in condition.keys(): + for indicator in condition[stat]: + if msg['indicator'] in indicator.keys(): + if indicator[msg['indicator']] == msg['indicator_number']: + new_message['values'].append( + msg['values']) + new_message['indicator'].append( + msg['indicator']) + c_nb_conditions += 1 + if alert_frequency != 'once': + key = ''.join([msg['market'], list( + msg['values'])[0], candle_period]) + should_alert += self.should_i_alert( + key, alert_frequency) + if msg['status'] == msg['last_status'] and alert_frequency == 'once' and not self.first_run: + c_nb_once_muted += 1 + if msg['status'] != msg['last_status']: + c_nb_new_status += 1 + + if c_nb_conditions == nb_conditions and c_nb_conditions and should_alert: + if c_nb_once_muted and not c_nb_new_status: + self.logger.info('Alert frecuency once. Dont alert. %s %s', + new_message['market'], new_message['indicator']) + else: + new_message['status'] = condition['label'] + self.notify_discord([new_message], None) + self.notify_webhook([new_message], None) + self.notify_telegram([new_message], None) + self.notify_stdout([new_message]) def notify_all_messages(self, exchange, market_pair, candle_period, messages): chart_file = None @@ -163,21 +248,21 @@ def notify_all_messages(self, exchange, market_pair, candle_period, messages): candles_data = self.all_historical_data[exchange][market_pair][candle_period] chart_file = self.create_chart( exchange, market_pair, candle_period, candles_data) - #self.logger.info('Chart file %s', chart_file) + # self.logger.info('Chart file %s', chart_file) except Exception as e: self.logger.info('Error creating chart for %s %s', market_pair, candle_period) self.logger.exception(e) # self.notify_slack(new_analysis) - self.notify_discord(messages) + self.notify_discord(messages, chart_file) self.notify_webhook(messages, chart_file) # self.notify_twilio(new_analysis) - # self.notify_gmail(new_analysis) + self.notify_email(messages) self.notify_telegram(messages, chart_file) self.notify_stdout(messages) - def notify_discord(self, messages): + def notify_discord(self, messages, chart_file): """Send a notification via the discord notifier Args: @@ -187,13 +272,24 @@ def notify_discord(self, messages): if not self.discord_configured: return - message_template = Template( - self.notifier_config['discord']['optional']['template']) + for notifier in self.discord_clients: + message_template = Template( + self.notifier_config[notifier]['optional']['template']) - for message in messages: - formatted_message = message_template.render(message) + formatted_messages = [message_template.render( + message) for message in messages] - self.discord_client.notify(formatted_message.strip()) + if self.enable_charts: + if chart_file: + self.discord_clients[notifier].send_chart_messages(chart_file, formatted_messages) + else: + self.logger.info( + 'Chart file %s doesnt exist, sending text message.', chart_file) + self.discord_clients[notifier].send_messages( + formatted_messages) + else: + self.discord_clients[notifier].send_messages( + formatted_messages) def notify_slack(self, new_analysis): """Send a notification via the slack notifier @@ -225,20 +321,22 @@ def notify_twilio(self, new_analysis): if message.strip(): self.twilio_client.notify(message) - def notify_gmail(self, new_analysis): - """Send a notification via the gmail notifier + def notify_email(self, new_analysis): + """Send a notification via the email notifier Args: new_analysis (dict): The new_analysis to send. """ + if not self.email_configured: + return - if self.gmail_configured: - message = self._indicator_message_templater( - new_analysis, - self.notifier_config['gmail']['optional']['template'] - ) - if message.strip(): - self.gmail_client.notify(message) + for notifier in self.email_clients: + message_template = Template( + self.notifier_config[notifier]['optional']['template']) + + for message in new_analysis: + formatted_message = message_template.render(message) + self.email_clients[notifier].notify(formatted_message.strip()) def notify_telegram(self, messages, chart_file): """Send notifications via the telegram notifier @@ -251,27 +349,27 @@ def notify_telegram(self, messages, chart_file): if not self.telegram_configured: return - message_template = Template( - self.notifier_config['telegram']['optional']['template']) + for notifier in self.telegram_clients: + message_template = Template( + self.notifier_config[notifier]['optional']['template']) - formatted_messages = [] + formatted_messages = [] - for message in messages: - formatted_messages.append(message_template.render(message)) + for message in messages: + formatted_messages.append(message_template.render(message)) - if self.enable_charts: - if chart_file and os.path.exists(chart_file): - try: - self.telegram_client.send_chart_messages( - open(chart_file, 'rb'), formatted_messages) - except (IOError, SyntaxError): - self.telegram_client.send_messages(formatted_messages) + if self.enable_charts: + if chart_file: + self.telegram_clients[notifier].send_chart_messages( + chart_file, formatted_messages) + else: + self.logger.info( + 'Chart file %s doesnt exist, sending text message.', chart_file) + self.telegram_clients[notifier].send_messages( + formatted_messages) else: - self.logger.info( - 'Chart file %s doesnt exist, sending text message.', chart_file) - self.telegram_client.send_messages(formatted_messages) - else: - self.telegram_client.send_messages(formatted_messages) + self.telegram_clients[notifier].send_messages( + formatted_messages) def notify_webhook(self, messages, chart_file): """Send notifications via a new webhook notifier @@ -284,7 +382,8 @@ def notify_webhook(self, messages, chart_file): if not self.webhook_configured: return - self.webhook_client.notify(messages, chart_file) + for notifier in self.webhook_clients: + self.webhook_clients[notifier].notify(messages, chart_file) def notify_stdout(self, messages): """Send a notification via the stdout notifier @@ -296,13 +395,14 @@ def notify_stdout(self, messages): if not self.stdout_configured: return - message_template = Template( - self.notifier_config['stdout']['optional']['template']) + for notifier in self.stdout_clients: + message_template = Template( + self.notifier_config[notifier]['optional']['template']) - for message in messages: - formatted_message = message_template.render(message) + for message in messages: + formatted_message = message_template.render(message) - self.stdout_client.notify(formatted_message.strip()) + self.stdout_clients[notifier].notify(formatted_message.strip()) def _validate_required_config(self, notifier, notifier_config): """Validate the required configuration items are present for a notifier. @@ -440,6 +540,38 @@ def _indicator_message_templater(self, new_analysis, template): self.last_analysis = {**self.last_analysis, **new_analysis} return new_message + def parse_alert_fequency(self, alert_frequency): + now = datetime.datetime.now() + matches = re.findall(r'\d+[dhms]', alert_frequency) + if not matches: + return + + for match in matches: + try: + value = int(match[:-1]) + except Exception as e: + self.logger.info('Unable to parse alert_frequency "%s"', value) + self.logger.debug(e) + continue + if match.endswith('m'): + now += datetime.timedelta(minutes=value) + elif match.endswith('h'): + now += datetime.timedelta(hours=value) + elif match.endswith('s'): + now += datetime.timedelta(seconds=value) + elif match.endswith('d'): + now += datetime.timedelta(days=value) + return now + + def should_i_alert(self, alert_frequency_key, alert_frequency): + if alert_frequency_key in self.alert_frequencies: + if self.alert_frequencies[alert_frequency_key] > datetime.datetime.now(): + return False + timedelta = self.parse_alert_fequency(alert_frequency) + if timedelta: + self.alert_frequencies[alert_frequency_key] = timedelta + return True + def get_indicator_messages(self, new_analysis): """Creates a message list from a user defined template @@ -454,9 +586,9 @@ def get_indicator_messages(self, new_analysis): self.last_analysis = new_analysis self.first_run = True - #self.logger.info('Is first run: {}'.format(self.first_run)) + # self.logger.info('Is first run: {}'.format(self.first_run)) - now = datetime.now(timezone(self.timezone)) + now = datetime.datetime.now(timezone(self.timezone)) creation_date = now.strftime("%Y-%m-%d %H:%M:%S") new_messages = dict() @@ -520,6 +652,7 @@ def get_indicator_messages(self, new_analysis): if isinstance(values[signal], float): values[signal] = format( values[signal], '.2f') + elif indicator_type == 'crossovers': latest_result = analysis['result'].iloc[-1] @@ -574,15 +707,14 @@ def get_indicator_messages(self, new_analysis): should_alert = True - # if self.first_run: - #self.logger.info('Alert once for %s %s %s', market_pair, indicator, candle_period) - - if not self.first_run: - if analysis['config']['alert_frequency'] == 'once': - if last_status == status: - self.logger.info('Alert frecuency once. Dont alert. %s %s %s', - market_pair, indicator, candle_period) - should_alert = False + if not self.first_run and not self.conditional_config: + if analysis['config']['alert_frequency'] == 'once' and last_status == status: + self.logger.info('Alert frecuency once. Dont alert. %s %s %s', + market_pair, indicator, candle_period) + should_alert = False + else: + should_alert = self.should_i_alert(''.join( + [market_pair, indicator, candle_period]), analysis['config']['alert_frequency']) if not analysis['config']['alert_enabled']: should_alert = False @@ -593,8 +725,10 @@ def get_indicator_messages(self, new_analysis): should_alert = False if should_alert: - base_currency, quote_currency = market_pair.split( - '/') + base_currency = market_pair.split('/') + quote_currency = '' + if len(base_currency) == 2: + base_currency, quote_currency = base_currency precision = self.market_data[exchange][market_pair]['precision'] decimal_format = '.{}f'.format( precision['price']) @@ -610,7 +744,7 @@ def get_indicator_messages(self, new_analysis): value = format( value, decimal_format) - prices = '{} {}: {}' . format( + prices = '{} {}: {}'.format( prices, key.title(), value) decimal_format = '%' + decimal_format @@ -624,7 +758,7 @@ def get_indicator_messages(self, new_analysis): new_message = message_template.render( values=values, exchange=exchange, market=market_pair, base_currency=base_currency, quote_currency=quote_currency, indicator=indicator, indicator_number=index, - analysis=analysis, status=status, last_status=last_status, + analysis=analysis, status=status, last_status=last_status, prices=prices, lrsi=lrsi, creation_date=creation_date, indicator_label=indicator_label) """ @@ -645,9 +779,6 @@ def get_indicator_messages(self, new_analysis): # Merge changes from new analysis into last analysis self.last_analysis = {**self.last_analysis, **new_analysis} - if self.first_run: - self.first_run = False - return new_messages def set_timezone(self, timezone): @@ -682,13 +813,14 @@ def create_charts(self, messages): except Exception as e: self.logger.info( 'Error creating chart for %s %s', market_pair, candle_period) - self.logger.exception(e) + self.logger.exception(e) + raise def create_chart(self, exchange, market_pair, candle_period, candles_data): - #self.logger.info("Beginning creation of charts: {} - {} - {}".format(exchange, market_pair, candle_period)) + # self.logger.info("Beginning creation of charts: {} - {} - {}".format(exchange, market_pair, candle_period)) - now = datetime.now(timezone(self.timezone)) + now = datetime.datetime.now(timezone(self.timezone)) creation_date = now.strftime("%Y-%m-%d %H:%M:%S") df = self.convert_to_dataframe(candles_data) diff --git a/app/notifiers/discord_client.py b/app/notifiers/discord_client.py index 9705c788..4a1ece91 100644 --- a/app/notifiers/discord_client.py +++ b/app/notifiers/discord_client.py @@ -1,12 +1,15 @@ - """Notify a user via discord """ import structlog -from webcord import Webhook +from discord_webhook import DiscordWebhook as Webhook + +from notifiers.utils import NotifierUtils +__max_message_size__ = 2000 -class DiscordNotifier(): + +class DiscordNotifier(NotifierUtils): """Class for handling Discord notifications """ @@ -18,16 +21,43 @@ def __init__(self, webhook, username, avatar=None): username (str): Display name for the discord bot. avatar (str, optional): Defaults to None. Url of an image to use as an avatar. """ - self.logger = structlog.get_logger() self.discord_username = username - self.discord_client = Webhook(webhook, avatar_url=avatar) + self.discord_client = Webhook( + url=webhook, username=username, avatar_url=avatar, rate_limit_retry=True) - def notify(self, message): + def notify(self, message: str): """Sends the message. Args: message (str): The message to send. """ - - self.discord_client.send_message(message, self.discord_username) + message_chunks = self.chunk_message( + message=message, max_message_size=__max_message_size__) + for message_chunk in message_chunks: + try: + self.discord_client.set_content(message_chunk) + self.discord_client.execute() + except Exception as e: + self.logger.info('Unable to send message using Discord !') + self.logger.debug(e) + + def send_chart_messages(self, photo_url: str, messages=[]): + """Send image chart + Args: + photo_url (str): The photo url to send. + """ + try: + self.discord_client.set_content('') + with open(photo_url, 'rb') as f: + self.discord_client.add_file(file=f.read(), filename=f.name) + self.discord_client.execute(remove_files=True) + except Exception as e: + self.logger.info('Unable to send chart messages using Discord !') + self.logger.debug(e) + self.send_messages(messages) + + def send_messages(self, messages=[]): + if messages: + for message in messages: + self.notify(message) diff --git a/app/notifiers/gmail_client.py b/app/notifiers/email_client.py similarity index 73% rename from app/notifiers/gmail_client.py rename to app/notifiers/email_client.py index bd7f6fda..99376eb5 100644 --- a/app/notifiers/gmail_client.py +++ b/app/notifiers/email_client.py @@ -1,4 +1,4 @@ -"""Notify a user via Gmail +"""Notify a user via Email """ import smtplib @@ -9,21 +9,22 @@ from notifiers.utils import NotifierUtils -class GmailNotifier(NotifierUtils): - """Class for handling gmail notifications +class EmailNotifier(NotifierUtils): + """Class for handling email notifications """ - def __init__(self, username, password, destination_addresses): - """Initialize GmailNotifier class + def __init__(self, smtp_server, username, password, destination_addresses): + """Initialize EmailNotifier class Args: - username (str): Username of the gmail account to use for sending message. - password (str): Password of the gmail account to use for sending message. + smtp_server (str): Smtp server address in form host:port + username (str): Username of the email account to use for sending message. + password (str): Password of the email account to use for sending message. destination_addresses (list): A list of email addresses to notify. """ self.logger = structlog.get_logger() - self.smtp_server = 'smtp.gmail.com:587' + self.smtp_server = smtp_server self.username = username self.password = password self.destination_addresses = ','.join(destination_addresses) diff --git a/app/notifiers/telegram_client.py b/app/notifiers/telegram_client.py index b98448de..1436aaab 100644 --- a/app/notifiers/telegram_client.py +++ b/app/notifiers/telegram_client.py @@ -11,6 +11,12 @@ from notifiers.utils import NotifierUtils +__con_pool_size__ = 10 +__connect_timeout__ = 40 +__stop_after_attempt__ = 3 +__wait_fixed__ = 5 +__max_message_size__ = 4096 + class TelegramNotifier(NotifierUtils): """Used to notify user of events via telegram. @@ -23,53 +29,54 @@ def __init__(self, token, chat_id, parse_mode): token (str): The telegram API token. chat_id (str): The chat ID you want the bot to send messages to. """ - self.logger = structlog.get_logger() self.bot = telegram.Bot(token=token, request=Request( - con_pool_size=10, connect_timeout=40)) + con_pool_size=__con_pool_size__, connect_timeout=__connect_timeout__)) self.chat_id = chat_id self.parse_mode = parse_mode @retry( retry=retry_if_exception_type(telegram.error.TimedOut), - stop=stop_after_attempt(3), - wait=wait_fixed(5) + stop=stop_after_attempt(__stop_after_attempt__), + wait=wait_fixed(__wait_fixed__) ) - def notify(self, message): + def notify(self, message: str): """Send the notification. Args: message (str): The message to send. """ - - max_message_size = 4096 message_chunks = self.chunk_message( - message=message, max_message_size=max_message_size) - # print(message_chunks) - # exit() + message=message, max_message_size=__max_message_size__) for message_chunk in message_chunks: - self.bot.send_message( - chat_id=self.chat_id, text=message_chunk, parse_mode=self.parse_mode) + try: + self.bot.send_message( + chat_id=self.chat_id, text=message_chunk, parse_mode=self.parse_mode) + except Exception as e: + self.logger.info('Unable to send message using Telegram !') + self.logger.debug(e) @retry( retry=retry_if_exception_type(telegram.error.TimedOut), - stop=stop_after_attempt(6), - wait=wait_fixed(5) + stop=stop_after_attempt(__stop_after_attempt__), + wait=wait_fixed(__wait_fixed__) ) - def send_chart_messages(self, photo_url, messages=[]): + def send_chart_messages(self, photo_url: str, messages=[]): """Send image chart Args: photo_url (str): The photo url to send. """ - - self.bot.send_photo(chat_id=self.chat_id, photo=photo_url, timeout=40) - - if len(messages) > 0: - for message in messages: - self.notify(message) + try: + with open(photo_url, 'rb') as f: + self.bot.send_photo(chat_id=self.chat_id, + photo=f.read(), timeout=__connect_timeout__) + except Exception as e: + self.logger.info('Unable to send chart messages using Telegram !') + self.logger.debug(e) + self.send_messages(messages) def send_messages(self, messages=[]): - if len(messages) > 0: + if messages: for message in messages: self.notify(message) diff --git a/app/notifiers/utils.py b/app/notifiers/utils.py index 27167bea..c2f1dea3 100644 --- a/app/notifiers/utils.py +++ b/app/notifiers/utils.py @@ -38,4 +38,4 @@ def chunk_message(self, message, max_message_size): else: chunked_message.append(message) - return chunked_message + return chunked_message \ No newline at end of file diff --git a/app/requirements-step-2.txt b/app/requirements-step-2.txt index 42ef2b6c..625ab94e 100644 --- a/app/requirements-step-2.txt +++ b/app/requirements-step-2.txt @@ -9,10 +9,11 @@ tabulate>=0.8.2 slackweb>=1.0.5 tenacity>=4.8.0 python-telegram-bot>=10.0.1 -webcord>=0.2 +discord-webhook>=0.15.0 jinja2>=2.10 requests>=2.20.0 PyYAML>=5.1 -tulipy>=0.2.1 +newtulipy matplotlib>=3.0.1 scipy>=1.1.0 +numpy>=1.22.2 # not directly required, pinned by Snyk to avoid a vulnerability diff --git a/docker-compose.dev.yml b/docker-compose.dev.yml deleted file mode 100644 index 6d27ac8b..00000000 --- a/docker-compose.dev.yml +++ /dev/null @@ -1,8 +0,0 @@ -version: '3' - -services: - app: - image: shadowreaver/crypto-signal:latest - volumes: - - ./app:/app - - ./config.yml:/app/config.yml diff --git a/docker-compose.yml b/docker-compose.yml index 54f30bb4..1502221a 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -2,6 +2,4 @@ version: '3' services: app: - image: shadowreaver/crypto-signal:master - volumes: - - ./config.yml:/app/config.yml + build: . diff --git a/docs/config.md b/docs/config.md index 24a1facd..29c45df5 100644 --- a/docs/config.md +++ b/docs/config.md @@ -7,7 +7,9 @@ 5) Indicators 6) Informants 7) Crossovers -8) Examples +8) Conditionals +9) Advanced Settings +10) Examples # 1) Configuration file structure The configuration file is YAML formatted and consists of the following top level keys. @@ -145,20 +147,25 @@ notifiers: template: "{{exchange}}-{{market}}-{{indicator}}-{{indicator_number}} is {{status}}!{{ '\n' -}}" ``` -## Gmail +## Email +**smtp_server**\ +default: None\ +necessity: required for Email\ +description: Your smtp server hostname + **username**\ default: None\ -necessity: required for Gmail\ -description: Your gmail username which is required for sending emails. +necessity: required for Email\ +description: Your email username which is required for sending emails. **password**\ default: None\ -necessity: required for Gmail\ -description: Your gmail password which is required for sending emails. +necessity: required for Email\ +description: Your email password which is required for sending emails. **destination_emails**\ default: None\ -necessity: required for Gmail\ +necessity: required for Email\ description: The email addresses to receive the emails that are sent. **template**\ @@ -166,13 +173,14 @@ default: {{exchange}}-{{market}}-{{analyzer}}-{{analyzer_number}} is {{status}}! necessity: optional\ description: See the notifier templating section. -An example of notifier settings for gmail +An example of notifier settings for email ```yml notifiers: - gmail: + email: required: - username: my_user@gmail.com + smtp_server: smtp.gmail.com:587 + username: example@gmail.com password: abcd1234 destination_emails: - my_user@gmail.com @@ -299,6 +307,28 @@ notifiers: template: "{{exchange}}-{{market}}-{{indicator}}-{{indicator_number}} is {{status}}!{{ '\n' -}}" ``` +## Mutiple Notifiers +The same type of notifier can be used multiple times like bellow: +```yml +notifiers: + telegram_00: + required: + token: XXX + chat_id: YYY + optional: + parse_mode: html + template: "[{{market}}] {{indicator}} {{status}} {{values}} {{ '\n' -}}" + telegram_01: + required: + token: AAA + chat_id: BBB + optional: + parse_mode: html + template: "[{{market}}] {{prices}} {{ '\n' -}}" +``` + +Be careful of the request rate to external services. + ## Notifier Templating The notifier templates are built with a templating language called [Jinja2](http://jinja.pocoo.org/docs/2.10/templates/) and anything that is a valid Jinja message is valid for crypto-signal. The options available are as follows: @@ -316,7 +346,7 @@ The notifier templates are built with a templating language called [Jinja2](http - analysis.result.is_cold - The raw boolean value of if the indicator is cold. - analysis.config.enabled - The raw config item of if this indicator is enabled. If you receive a message with a value other than True something has gone horribly wrong. - analysis.config.alert_enabled - The raw config item of if this indicator alert is enabled. If you receive a message with a value other than True something has gone horribly wrong. -- analysis.config.alert_frequency - The raw config item of whether this alert is always sent or if it is only sent once per status change. +- analysis.config.alert_frequency - The raw config item of whether this alert is always sent or if it is only sent once per status change. Can also define the sleep time of an alert. - analysis.config.hot - The raw config item of what the configured hot threshold is. - analysis.config.cold - The raw config item of what the configured cold threshold is. - analysis.config.candle_period - The raw config item of what time period of candles to gather. @@ -343,6 +373,19 @@ default: True\ necessity: optional\ description: Valid values are true or false. Whether to send alerts for this particular indicator. +**alert_frequency**\ +default: always\ +necessity: optional\ +description: Valid values are always, once or time period in the format described below. Whether to send alerts or frequency of alerts for this particular indicator. +time period format: + - `d` for number of days + - `h` for number of hours + - `m` for number of minutes + - `s` for number of seconds + ex 1: `1d12h20m10s` + ex 2: `15m` +For conditional mode, if only one indicator of the condition has its alert frequency valid, the alert will be sent. (Should it stays like that ? Open an issue if you want to change that behaviour) + **signal**\ default: A string\ necessity: optional\ @@ -508,8 +551,149 @@ crossovers: crossed_signal: sma ``` +# 8) Conditionals + +It's allowing you to receive notifications, only if one or more conditions are respected. + +Use case examples: +- Receive a buy notification if rsi is cold and bollinger is hot and aroon is cold. +- Receive a sell notification if 1d rsi is hot and 1h rsi is hot and bollinger is cold and aroon is hot. + +**You will not receive notifications if all conditions, of one conditionnal, are not met.** + +## Example + +```yml +settings: + log_level: INFO + update_interval: 120 + start_worker_interval: 2 + market_data_chunk_size: 1 + timezone: Europe/Paris + +exchanges: + kraken: + required: + enabled: true + all_pairs: + - USD + +indicators: + rsi: + - enabled: true + alert_enabled: true + alert_frequency: always + signal: + - rsi + hot: 30 + cold: 70 + candle_period: 1h + period_count: 14 + - enabled: true + alert_enabled: true + alert_frequency: always + signal: + - rsi + hot: 40 + cold: 60 + candle_period: 1d + period_count: 14 + bollinger: + - enabled: true + candle_period: 1h + alert_enabled: true + alert_frequency: always + period_count: 25 + std_dev: 2.5 + signal: + - low_band + - close + - up_band + mute_cold: false + - enabled: true + candle_period: 1d + alert_enabled: true + alert_frequency: always + period_count: 25 + std_dev: 2.5 + signal: + - low_band + - close + - up_band + mute_cold: false + aroon_oscillator: + - enabled: true + alert_enabled: true + alert_frequency: always + sma_vol_period: 50 + period_count: 14 + signal: + - aroon + candle_period: 1h + +conditionals: + - label: "Signal to buy" + hot: + - rsi: 0 + - rsi: 1 + cold: + - bollinger: 0 + - label: "Signal to buy" + hot: + - rsi: 1 + - label: "Signal to sell" + cold: + - rsi: 1 + - rsi: 0 + hot: + - aroon_oscillator: 0 + +notifiers: + telegram: + required: + token: X + chat_id: Y + optional: + parse_mode: html + template: "[{{market}}] {{indicator}} {{status}} {{values}} {{ '\n' -}}" +``` -# 8) Examples +## Template value available + - values + - indicator + - price_value + - exchange + - market + - base_currency + - quote_currency + - status + - price_value + - decimal_format + + The `status` will be the string set in `label`. + +# 9) Advanced Settings + ## `start_worker_interval` + `start_worker_interval` allows to define the number of the seconds between each start of a worker (use of multi processing to manage chunk of pairs). + It's usefull to manage time between requests to exchange servers. + ```yml + settings: + [...] + start_worker_interval: 2 + [...] + ``` + + ## `market_data_chunk_size` + `market_data_chunk_size` allows to define the number of pairs a worker will work on. + Lower the chunk is, faster the worker end his work. But, lower the chunk is, more workers will be required. + ```yml + settings: + [...] + market_data_chunk_size: 1 + [...] + ``` + +# 10) Examples Putting it all together an example config.yml might look like the config below if you want to use the default settings with bittrex ```yml