Skip to content

Commit

Permalink
Merge pull request #53 from semiotic-ai/overhaul_and_cleanup
Browse files Browse the repository at this point in the history 
Overhaul args, cleanup docs, code
  • Loading branch information
@aasseman
aasseman authored Jan 19, 2023
2 parents 963987d + 453a816 commit 24324e7
Show file tree
Hide file tree
Showing 9 changed files with 138 additions and 259 deletions.
10 changes: 0 additions & 10 deletions CHANGELOG
View file

This file was deleted.

105 changes: 56 additions & 49 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -21,22 +21,22 @@ An [Agora](https://github.com/graphprotocol/agora) cost model automation tool fo

Just build the container!

```sh
```console
docker build -t autoagora .
```

### Usage

For AutoAgora to function correctly, you will also need to set up:
For AutoAgora's relative query costs generator, you will also need to set up:

- [AutoAgora indexer-service](https://github.com/semiotic-ai/autoagora-indexer-service)
- [AutoAgora Processor](https://github.com/semiotic-ai/autoagora-processor)

AutoAgora will continously:
AutoAgora will continuously:

- Watch for the indexer's current allocations by querying the `indexer-agent`'s management GraphQL endpoint.
- Analyze the query logs stored in a PostgreSQL database -- logs that were previously processed by the `AutoAgora
indexer-service` wrapper and the `AutoAgora Processor`.
- Analyze the query logs stored in a PostgreSQL database (relative query costs generator) -- logs that were previously
processed by the `AutoAgora indexer-service` wrapper and the `AutoAgora Processor`.
- Gather query metrics from the `indexer-service`'s prometheus metrics endpoint.
- Update the allocated subgraph's cost models by sending mutations to the `indexer-agent`'s management GraphQL endpoint.

Expand All @@ -45,25 +45,18 @@ Therefore, only a single instance of `AutoAgora` should be running against an `i
Configuration:

```txt
usage: autoagora [-h] [--experimental-model-builder]
[--log-level {DEBUG,INFO,WARNING,ERROR,CRITICAL}] [--json-logs JSON_LOGS]
--indexer-agent-mgmt-endpoint INDEXER_AGENT_MGMT_ENDPOINT
--indexer-service-metrics-endpoint INDEXER_SERVICE_METRICS_ENDPOINT
[--observation-duration OBSERVATION_DURATION]
[--exclude-subgraphs EXCLUDE_SUBGRAPHS]
[--agora-models-refresh-interval AGORA_MODELS_REFRESH_INTERVAL]
[--logs-postgres-host LOGS_POSTGRES_HOST]
[--logs-postgres-port LOGS_POSTGRES_PORT]
[--logs-postgres-database LOGS_POSTGRES_DATABASE]
[--logs-postgres-username LOGS_POSTGRES_USERNAME]
[--logs-postgres-password LOGS_POSTGRES_PASSWORD]
usage: autoagora [-h] [--log-level {DEBUG,INFO,WARNING,ERROR,CRITICAL}] [--json-logs JSON_LOGS]
--postgres-host POSTGRES_HOST [--postgres-port POSTGRES_PORT]
[--postgres-database POSTGRES_DATABASE] --postgres-username POSTGRES_USERNAME
--postgres-password POSTGRES_PASSWORD --indexer-agent-mgmt-endpoint
INDEXER_AGENT_MGMT_ENDPOINT --indexer-service-metrics-endpoint
INDEXER_SERVICE_METRICS_ENDPOINT
[--qps-observation-duration QPS_OBSERVATION_DURATION] [--relative-query-costs]
[--relative-query-costs-exclude-subgraphs RELATIVE_QUERY_COSTS_EXCLUDE_SUBGRAPHS]
[--relative-query-costs-refresh-interval RELATIVE_QUERY_COSTS_REFRESH_INTERVAL]
optional arguments:
-h, --help show this help message and exit
--experimental-model-builder
Activates the relative query cost discovery. Otherwise only builds a
default query pricing model with automated market price discovery. [env
var: EXPERIMENTAL_MODEL_BUILDER] (default: False)
--log-level {DEBUG,INFO,WARNING,ERROR,CRITICAL}
[env var: LOG_LEVEL] (default: WARNING)
--json-logs JSON_LOGS
Expand All @@ -75,33 +68,42 @@ optional arguments:
--indexer-service-metrics-endpoint INDEXER_SERVICE_METRICS_ENDPOINT
HTTP endpoint for the indexer-service metrics. [env var:
INDEXER_SERVICE_METRICS_ENDPOINT] (default: None)
--observation-duration OBSERVATION_DURATION
--qps-observation-duration QPS_OBSERVATION_DURATION
Duration of the measurement period of the query-per-second after a price
multiplier update. [env var: MEASUREMENT_PERIOD] (default: 60)
Model Builder Options:
--exclude-subgraphs EXCLUDE_SUBGRAPHS
Comma delimited list of subgraphs (ipfs hash) to exclude from model
updates. [env var: EXCLUDE_SUBGRAPHS] (default: None)
--agora-models-refresh-interval AGORA_MODELS_REFRESH_INTERVAL
Interval in seconds between rebuilds of the Agora models. [env var:
AGORA_MODELS_REFRESH_INTERVAL] (default: 3600)
--logs-postgres-host LOGS_POSTGRES_HOST
Host of the postgres instance storing the logs. [env var:
LOGS_POSTGRES_HOST] (default: None)
--logs-postgres-port LOGS_POSTGRES_PORT
Port of the postgres instance storing the logs. [env var:
LOGS_POSTGRES_PORT] (default: 5432)
--logs-postgres-database LOGS_POSTGRES_DATABASE
Name of the logs database. [env var: LOGS_POSTGRES_DATABASE] (default:
None)
--logs-postgres-username LOGS_POSTGRES_USERNAME
Username for the logs database. [env var: LOGS_POSTGRES_USERNAME]
(default: None)
--logs-postgres-password LOGS_POSTGRES_PASSWORD
Password for the logs database. [env var: LOGS_POSTGRES_PASSWORD]
(default: None)
multiplier update. [env var: QPS_OBSERVATION_DURATION] (default: 60)
Database settings:
Must be the same database as AutoAgora Processor's if the relative costs models generator is
enabled.
--postgres-host POSTGRES_HOST
Host of the postgres instance to be used by AutoAgora. [env var:
POSTGRES_HOST] (default: None)
--postgres-port POSTGRES_PORT
Port of the postgres instance to be used by AutoAgora. [env var:
POSTGRES_PORT] (default: 5432)
--postgres-database POSTGRES_DATABASE
Name of the database to be used by AutoAgora. [env var: POSTGRES_DATABASE]
(default: autoagora)
--postgres-username POSTGRES_USERNAME
Username for the database to be used by AutoAgora. [env var:
POSTGRES_USERNAME] (default: None)
--postgres-password POSTGRES_PASSWORD
Password for the database to be used by AutoAgora. [env var:
POSTGRES_PASSWORD] (default: None)
Relative query costs generator settings:
--relative-query-costs
(EXPERIMENTAL) Enables the relative query cost generator. Otherwise only
builds a default query pricing model with automated market price
discovery. [env var: RELATIVE_QUERY_COSTS] (default: False)
--relative-query-costs-exclude-subgraphs RELATIVE_QUERY_COSTS_EXCLUDE_SUBGRAPHS
Comma delimited list of subgraphs (ipfs hash) to exclude from the relative
query costs model generator. [env var:
RELATIVE_QUERY_COSTS_EXCLUDE_SUBGRAPHS] (default: None)
--relative-query-costs-refresh-interval RELATIVE_QUERY_COSTS_REFRESH_INTERVAL
(Seconds) Interval between rebuilds of the relative query costs models.
[env var: RELATIVE_QUERY_COSTS_REFRESH_INTERVAL] (default: 3600)
If an arg is specified in more than one place, then commandline values override environment
variables which override defaults.
Expand All @@ -124,13 +126,18 @@ bandit_stddev{subgraph="QmRDGLp6BHwiH9HAE2NYEE3f7LrKuRqziHBv76trT4etgU"} 1.84346
bandit_mean{subgraph="QmRDGLp6BHwiH9HAE2NYEE3f7LrKuRqziHBv76trT4etgU"} 3.653126148672616e-05
```

Where "bandit" refers to the reinforcement learning method (Continuum-armed bandit) used to track the market price for each subgraph.
Where "bandit" refers to the reinforcement learning method (Continuum-armed bandit) used to track the market price for
each subgraph.

## Developer's guide

If you would like to contribute, please consult [`CONTRIBUTING.md`](CONTRIBUTING.md).

### Installation directly from the source code

To install AutoAgora directly from the source code please clone the repository and install package in the virtual environment using `poetry`:
To install AutoAgora directly from the source code please clone the repository and install package in the virtual
environment using `poetry`:

```console
git clone https://github.com/semiotic-ai/autoagora.git
cd autoagora
Expand Down
148 changes: 70 additions & 78 deletions autoagora/config.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -26,41 +26,23 @@ def init_config(argv: Optional[Sequence[str]] = None):
Argument values are added to the global `args` namespace object declared in this
module.
"""
# 2 arg parsers to make it possible to fetch the `experimental_model_builder` option
# early without breaking anything.
argsparser_experimental_model_builder = configargparse.ArgumentParser(
formatter_class=argparse.ArgumentDefaultsHelpFormatter, add_help=False
)
argsparser_experimental_model_builder.add_argument(
"--experimental-model-builder",
env_var="EXPERIMENTAL_MODEL_BUILDER",
action="store_true",
help="Activates the relative query cost discovery. Otherwise only builds a "
"default query pricing model with automated market price discovery.",
)
# Get the value of `experimental-model-builder` early
_, argv = argsparser_experimental_model_builder.parse_known_args(
argv, namespace=args
)

# 2nd arg parser
argsparser = configargparse.ArgumentParser(
formatter_class=argparse.ArgumentDefaultsHelpFormatter,
parents=[argsparser_experimental_model_builder],
argparser = configargparse.ArgumentParser(
formatter_class=argparse.ArgumentDefaultsHelpFormatter
)

#
# General arguments
#
argsparser.add_argument(
argparser.add_argument(
"--log-level",
env_var="LOG_LEVEL",
type=str,
choices=("DEBUG", "INFO", "WARNING", "ERROR", "CRITICAL"),
default="WARNING",
required=False,
)
argsparser.add_argument(
argparser.add_argument(
"--json-logs",
env_var="JSON_LOGS",
type=bool,
Expand All @@ -69,10 +51,52 @@ def init_config(argv: Optional[Sequence[str]] = None):
help="Output logs in JSON format. Compatible with GKE.",
)

#
# AutoAgora DB
#
argparser_database_group = argparser.add_argument_group(
"Database settings",
description="Must be the same database as AutoAgora Processor's if the "
"relative costs models generator is enabled.",
)
argparser_database_group.add_argument(
"--postgres-host",
env_var="POSTGRES_HOST",
required=True,
help="Host of the postgres instance to be used by AutoAgora.",
)
argparser_database_group.add_argument(
"--postgres-port",
env_var="POSTGRES_PORT",
required=False,
type=int,
default=5432,
help="Port of the postgres instance to be used by AutoAgora.",
)
argparser_database_group.add_argument(
"--postgres-database",
env_var="POSTGRES_DATABASE",
required=False,
default="autoagora",
help="Name of the database to be used by AutoAgora.",
)
argparser_database_group.add_argument(
"--postgres-username",
env_var="POSTGRES_USERNAME",
required=True,
help="Username for the database to be used by AutoAgora.",
)
argparser_database_group.add_argument(
"--postgres-password",
env_var="POSTGRES_PASSWORD",
required=True,
help="Password for the database to be used by AutoAgora.",
)

#
# Indexer utils
#
argsparser.add_argument(
argparser.add_argument(
"--indexer-agent-mgmt-endpoint",
env_var="INDEXER_AGENT_MGMT_ENDPOINT",
required=True,
Expand All @@ -82,8 +106,7 @@ def init_config(argv: Optional[Sequence[str]] = None):
#
# Query volume metrics
#

argsparser.add_argument(
argparser.add_argument(
"--indexer-service-metrics-endpoint",
env_var="INDEXER_SERVICE_METRICS_ENDPOINT",
required=True,
Expand All @@ -93,9 +116,9 @@ def init_config(argv: Optional[Sequence[str]] = None):
#
# Price multiplier (Absolute price)
#
argsparser.add_argument(
"--observation-duration",
env_var="MEASUREMENT_PERIOD",
argparser.add_argument(
"--qps-observation-duration",
env_var="QPS_OBSERVATION_DURATION",
required=False,
type=int,
default=60,
Expand All @@ -106,64 +129,33 @@ def init_config(argv: Optional[Sequence[str]] = None):
#
# Optional model builder (Relative query costs)
#
model_builder_group = argsparser.add_argument_group(
title="Model Builder Options", description=""
argparser_relative_query_costs = argparser.add_argument_group(
"Relative query costs generator settings"
)
model_builder_group.add_argument(
"--exclude-subgraphs",
env_var="EXCLUDE_SUBGRAPHS",
required=False,
help="Comma delimited list of subgraphs (ipfs hash) to exclude from model "
"updates.",
argparser_relative_query_costs.add_argument(
"--relative-query-costs",
env_var="RELATIVE_QUERY_COSTS",
action="store_true",
help="(EXPERIMENTAL) Enables the relative query cost generator. Otherwise only "
"builds a default query pricing model with automated market price discovery.",
)
model_builder_group.add_argument(
"--agora-models-refresh-interval",
env_var="AGORA_MODELS_REFRESH_INTERVAL",
argparser_relative_query_costs.add_argument(
"--relative-query-costs-exclude-subgraphs",
env_var="RELATIVE_QUERY_COSTS_EXCLUDE_SUBGRAPHS",
required=False,
type=int,
default=3600,
help="Interval in seconds between rebuilds of the Agora models.",
)

#
# Logs DB
#

# Needed only if the model builder is turned on
model_builder_group.add_argument(
"--logs-postgres-host",
env_var="LOGS_POSTGRES_HOST",
required=args.experimental_model_builder,
help="Host of the postgres instance storing the logs.",
help="Comma delimited list of subgraphs (ipfs hash) to exclude from the "
"relative query costs model generator.",
)
model_builder_group.add_argument(
"--logs-postgres-port",
env_var="LOGS_POSTGRES_PORT",
argparser_relative_query_costs.add_argument(
"--relative-query-costs-refresh-interval",
env_var="RELATIVE_QUERY_COSTS_REFRESH_INTERVAL",
required=False,
type=int,
default=5432,
help="Port of the postgres instance storing the logs.",
)
model_builder_group.add_argument(
"--logs-postgres-database",
env_var="LOGS_POSTGRES_DATABASE",
required=args.experimental_model_builder,
help="Name of the logs database.",
)
model_builder_group.add_argument(
"--logs-postgres-username",
env_var="LOGS_POSTGRES_USERNAME",
required=args.experimental_model_builder,
help="Username for the logs database.",
)
model_builder_group.add_argument(
"--logs-postgres-password",
env_var="LOGS_POSTGRES_PASSWORD",
required=args.experimental_model_builder,
help="Password for the logs database.",
default=3600,
help="(Seconds) Interval between rebuilds of the relative query costs models.",
)

argsparser.parse_args(args=argv, namespace=args)
argparser.parse_args(args=argv, namespace=args)

# Set the logs formatting
if args.json_logs:
Expand Down
Loading

0 comments on commit 24324e7

Please sign in to comment.