Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

geoipupdate supports environment variables for config options #243

Merged
merged 25 commits into from
Jul 10, 2023
Merged

geoipupdate supports environment variables for config options #243

Show file tree
Hide file tree
Changes from 19 commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
67b1360
Refactor config code to better accommodate multiple config sources
ugexe Jun 20, 2023
afa6092
Add environment variable configuration options
ugexe Jun 20, 2023
37011a6
Use environment variables in docker entry instead of config file
ugexe Jun 20, 2023
acb13ed
Update docs to reference enviroment variable configuration
ugexe Jun 21, 2023
20ab1a6
Remove environment config specific documentation
ugexe Jun 23, 2023
98b41b3
Fix typos
ugexe Jun 23, 2023
0b89f05
Remove confusing message
ugexe Jun 23, 2023
0352d5f
Simplify some boolean config assignment
ugexe Jun 23, 2023
cbeaef4
Use t.TempDir() instead of os.CreateTemp()
ugexe Jun 23, 2023
f134315
Mention environment config options in changelog
ugexe Jun 23, 2023
928b738
Update expected value in documentation
ugexe Jun 23, 2023
34a4203
Support $GEOIPUPDATE_CONF_FILE in geoipupdate
ugexe Jun 23, 2023
443d615
Update error to reflect non-file config errors
ugexe Jun 23, 2023
62070e2
Use GEOIPUPDATE_DB_DIR instead of --database-directory
ugexe Jun 23, 2023
8f864e5
Make config file path optional
ugexe Jun 28, 2023
2fd246e
Use original truth criteria when setting Verbose
ugexe Jun 28, 2023
39a4103
Fix typos and spacing
ugexe Jun 28, 2023
d1abad8
Add related environment variables in CLI docs
ugexe Jun 28, 2023
6e8aee6
Mention breaking changes in changelog
ugexe Jun 28, 2023
3b9b147
Use non-pointer
ugexe Jul 6, 2023
66f47bb
Add _ACCOUNT_ID_FILE and _LICENSE_KEY_FILE env vars
ugexe Jul 6, 2023
f0334b4
Remove leftover reference to non-existant GeoIP.env
ugexe Jul 6, 2023
08dce57
Make configuration error message more generic
ugexe Jul 7, 2023
ad02665
Avoid leaking account id in error message
ugexe Jul 10, 2023
007a68c
Tidy whitespace, fix typo
ugexe Jul 10, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 26 additions & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,31 @@
# CHANGELOG

## 6.0.0

* `geoipupdate` now supports configuration via environment variables. Any
configuration set this way will override any value from the config file,
but still be overridden by any associated command line option (if any).
The following new environment variables are supported:

* `GEOIPUPDATE_ACCOUNT_ID`
* `GEOIPUPDATE_CONF_FILE`
* `GEOIPUPDATE_DB_DIR`
* `GEOIPUPDATE_EDITION_IDS`
* `GEOIPUPDATE_HOST`
* `GEOIPUPDATE_LICENSE_KEY`
ugexe marked this conversation as resolved.
Show resolved Hide resolved
* `GEOIPUPDATE_LOCK_FILE`
* `GEOIPUPDATE_PARALLELISM`
* `GEOIPUPDATE_PRESERVE_FILE_TIMES`
* `GEOIPUPDATE_PROXY`
* `GEOIPUPDATE_PROXY_USER_PASSWORD`
* `GEOIPUPDATE_RETRY_FOR`
* `GEOIPUPDATE_VERBOSE`
horgh marked this conversation as resolved.
Show resolved Hide resolved

* Changed the signature of `NewConfig` in `pkg/geoipupdate` to no longer accept
a positional config file path argument, which can now be passed in using the
option from `WithConfigFile` along with the other optional parameters.
* `geoipupdate` and `NewConfig` no longer require a config file to exist.

## 5.1.1 (2023-05-08)

* Based on feedback, the change to use a non-root user in 5.1.0
Expand Down
12 changes: 6 additions & 6 deletions cmd/geoipupdate/args.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,10 +19,15 @@ type Args struct {
}

func getArgs() *Args {
confFileDefault := vars.DefaultConfigFile
ugexe marked this conversation as resolved.
Show resolved Hide resolved
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I want to configure based on env vars rather than a config file, will that work? I'm thinking that there will always be a non-empty config file specified given the default so we will always try to use it. In the case where I'm wanting to use env vars only, would that error? Not sure how to solve this if so. I guess we'd have to set GEOIPUPDATE_CONF_FILE="" maybe?

We also should think about how docker users will work given they may have been passing that env var already.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, GEOIPUPDATE_CONFIG_FILE="" would have to be set to avoid reading the config file altogether. To avoid using any of the values of that config file one would have to supply each option as an environment variable or command line flag -- the file would still get read but the values should be overridden.

The only other thing I can think of is convoluted and doesn't really solve it ideally either. Something like create the config using flags first, then environment variables, and then the config file. Each time it would only update fields that haven't yet been updated. When it reaches the time to set config from the config file, it could check if the current config has any fields that are zero values and if not avoid reading the file altogether. But that would mean every optional value would need to be set explicitly, so I don't think thats what we want to do.

if value, ok := os.LookupEnv("GEOIPUPDATE_CONF_FILE"); ok {
confFileDefault = value
}

configFile := flag.StringP(
"config-file",
"f",
vars.DefaultConfigFile,
confFileDefault,
"Configuration file",
)
databaseDirectory := flag.StringP(
Expand All @@ -49,11 +54,6 @@ func getArgs() *Args {
os.Exit(0)
}

if *configFile == "" {
log.Printf("You must provide a configuration file.")
printUsage()
}

if *parallelism < 0 {
log.Printf("Parallelism must be a positive number")
printUsage()
Expand Down
4 changes: 2 additions & 2 deletions cmd/geoipupdate/main.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,14 +41,14 @@ func main() {
}

config, err := geoipupdate.NewConfig(
args.ConfigFile,
geoipupdate.WithConfigFile(args.ConfigFile),
geoipupdate.WithDatabaseDirectory(args.DatabaseDirectory),
geoipupdate.WithParallelism(args.Parallelism),
geoipupdate.WithVerbose(args.Verbose),
geoipupdate.WithOutput(args.Output),
)
if err != nil {
fatalLogger(fmt.Sprintf("error loading configuration file %s", args.ConfigFile), err)
fatalLogger(fmt.Sprintf("error loading configuration %s", args.ConfigFile), err)
ugexe marked this conversation as resolved.
Show resolved Hide resolved
}

if config.Verbose {
Expand Down
45 changes: 29 additions & 16 deletions doc/GeoIP.conf.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,66 +17,79 @@ sensitive.

`AccountID`

: Your MaxMind account ID. This was formerly known as `UserId`.
: Your MaxMind account ID. This was formerly known as `UserId`. This can be
overridden at run time by the `GEOIPUPDATE_ACCOUNT_ID` environment
variable.

`LicenseKey`

: Your case-sensitive MaxMind license key.
: Your case-sensitive MaxMind license key. This can be overridden at run time
by the `GEOIPUPDATE_LICENSE_KEY` environment variable.

`EditionIDs`

: List of space-separated database edition IDs. Edition IDs may consist
of letters, digits, and dashes. For example, `GeoIP2-City` would
download the GeoIP2 City database (`GeoIP2-City`). Note: this was
formerly called `ProductIds`.
download the GeoIP2 City database (`GeoIP2-City`). This can be overridden
at run time by the `GEOIPUPDATE_EDITION_IDS` environment variable. Note:
this was formerly called `ProductIds`.

## Optional settings:

`DatabaseDirectory`

: The directory to store the database files. If not set, the default is
DATADIR. This can be overridden at run time by the `-d` command line
argument.
DATADIR. This can be overridden at run time by the `GEOIPUPDATE_DB_DIR`
environment variable or the `-d` command line argument.

`Host`

: The host name of the server to use. The default is `updates.maxmind.com`.
This can be overridden at run time by the `GEOIPUPDATE_HOST` environment
variable.

`Proxy`

: The proxy host name or IP address. You may optionally specify
a port number, e.g., `127.0.0.1:8888`. If no port number is specified,
1080 will be used.
: The proxy host name or IP address. You may optionally specify a port
number, e.g., `127.0.0.1:8888`. If no port number is specified, 1080
will be used. This can be overridden at run time by the
`GEOIPUPDATE_PROXY` environment variable.

`ProxyUserPassword`

: The proxy user name and password, separated by a colon. For instance,
`username:password`.
`username:password`. This can be overridden at run time by the
`GEOIPUPDATE_PROXY_USER_PASSWORD` environment variable.

`PreserveFileTimes`

: Whether to preserve modification times of files downloaded from the
server. This option is either `0` or `1`. The default is `0`.
server. This option is either `0` or `1`. The default is `0`. This
can be overridden at run time by the `GEOIPUPDATE_PRESERVE_FILE_TIMES`
environment variable.

`LockFile`

: The lock file to use. This ensures only one `geoipupdate` process can run
at a time. Note: Once created, this lockfile is not removed from the
filesystem. The default is `.geoipupdate.lock` under the
`DatabaseDirectory`.
`DatabaseDirectory`. This can be overridden at run time by the
`GEOIPUPDATE_LOCK_FILE` environment variable.

`RetryFor`

: The amount of time to retry for when errors during HTTP transactions are
encountered. It can be specified as a (possibly fractional) decimal number
followed by a unit suffix. Valid time units are `ns`, `us` (or `µs`), `ms`,
`s`, `m`, `h`. The default is `5m` (5 minutes).
`s`, `m`, `h`. The default is `5m` (5 minutes). This can be overridden at
run time by the `GEOIPUPDATE_RETRY_FOR` environment variable.

`Parallelism`

: The maximum number of parallel database downloads. The default is
1, which means that databases will be downloaded sequentially. This can be
overriden at runtime by the `--parallelism` command line argument.
: The maximum number of parallel database downloads. The default is
1, which means that databases will be downloaded sequentially. This can be
overridden at run time by the `GEOIPUPDATE_PARALLELISM` environment
variable or the `--parallelism` command line argument.

## Deprecated settings:

Expand Down
4 changes: 2 additions & 2 deletions doc/docker.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,8 +34,8 @@ The following are optional:
default is `0`.
* `GEOIPUPDATE_VERBOSE` - Enable verbose mode. Prints out the steps that
`geoipupdate` takes. Set to **anything** (e.g., `1`) to enable.
* `GEOIPUPDATE_CONF_FILE` - The path where the configuration file will be
written. The default is `/etc/GeoIP.conf`.
* `GEOIPUPDATE_CONF_FILE` - The path of a configuration file to be used by
`geoipupdate`.
* `GEOIPUPDATE_DB_DIR` - The directory where geoipupdate will download the
databases. The default is `/usr/share/GeoIP`.

Expand Down
16 changes: 10 additions & 6 deletions doc/geoipupdate.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,12 +21,14 @@ open.
`-d`, `--database-directory`

: Install databases to a custom directory. This is optional. If provided, it
overrides any `DatabaseDirectory` set in the configuration file.
overrides the `DatabaseDirectory` value from the configuration file and the
`GEOIPUPDATE_DB_DIR` environment variable.

`-f`, `--config-file`

: The configuration file to use. See `GeoIP.conf` and its documentation for
more information. This is optional. It defaults to CONFFILE.
more information. This is optional. It defaults to the environment variable
`GEOIPUPDATE_CONF_FILE` if it is set, or CONFFILE otherwise.

`--parallelism`

Expand All @@ -47,7 +49,8 @@ open.

`-v`, `--verbose`

: Enable verbose mode. Prints out the steps that `geoipupdate` takes.
: Enable verbose mode. Prints out the steps that `geoipupdate` takes. If
provided, it overrides any `GEOIPUPDATE_VERBOSE` environment variable.

`-o`, `--output`

Expand All @@ -72,9 +75,10 @@ runs `geoipupdate` on each Wednesday at noon:
# end of crontab


To use with a proxy server, update your `GeoIP.conf` file as specified
in the `GeoIP.conf` man page or set the `http_proxy` environment
variable.
To use with a proxy server, update your `GeoIP.conf` file as specified in
the `GeoIP.conf` man page or set the `GEOIPUPDATE_PROXY` environment variable
as specified in the `GeoIP.env` man page. Alternatively, set the `http_proxy`
ugexe marked this conversation as resolved.
Show resolved Hide resolved
environment variable.

# BUGS

Expand Down
39 changes: 3 additions & 36 deletions docker/entry.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,19 +14,14 @@ term_handler() {
trap 'kill ${!}; term_handler' SIGTERM

pid=0
conf_file=/var/lib/geoipupdate/GeoIP.conf
database_dir=/usr/share/GeoIP
log_dir="/var/lib/geoipupdate"
log_file="$log_dir/.healthcheck"
flags="--output"
frequency=$((GEOIPUPDATE_FREQUENCY * 60 * 60))

if ! [ -z "$GEOIPUPDATE_CONF_FILE" ]; then
conf_file=$GEOIPUPDATE_CONF_FILE
fi

if ! [ -z "$GEOIPUPDATE_DB_DIR" ]; then
database_dir=$GEOIPUPDATE_DB_DIR
if [ -z "$GEOIPUPDATE_DB_DIR" ]; then
GEOIPUPDATE_DB_DIR="$database_dir"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doesn't this need to be exported?

fi

if [ ! -z "$GEOIPUPDATE_ACCOUNT_ID_FILE" ]; then
Expand All @@ -42,39 +37,11 @@ if [ -z "$GEOIPUPDATE_ACCOUNT_ID" ] || [ -z "$GEOIPUPDATE_LICENSE_KEY" ] || [ -
exit 1
fi

# Create configuration file
echo "# STATE: Creating configuration file at $conf_file"
cat <<EOF > "$conf_file"
AccountID $GEOIPUPDATE_ACCOUNT_ID
LicenseKey $GEOIPUPDATE_LICENSE_KEY
EditionIDs $GEOIPUPDATE_EDITION_IDS
EOF

if [ ! -z "$GEOIPUPDATE_HOST" ]; then
echo "Host $GEOIPUPDATE_HOST" >> "$conf_file"
fi

if [ ! -z "$GEOIPUPDATE_PROXY" ]; then
echo "Proxy $GEOIPUPDATE_PROXY" >> "$conf_file"
fi

if [ ! -z "$GEOIPUPDATE_PROXY_USER_PASSWORD" ]; then
echo "ProxyUserPassword $GEOIPUPDATE_PROXY_USER_PASSWORD" >> "$conf_file"
fi

if [ ! -z "$GEOIPUPDATE_PRESERVE_FILE_TIMES" ]; then
echo "PreserveFileTimes $GEOIPUPDATE_PRESERVE_FILE_TIMES" >> "$conf_file"
fi

if [ "$GEOIPUPDATE_VERBOSE" ]; then
flags="$flags -v"
fi

mkdir -p $log_dir

while true; do
echo "# STATE: Running geoipupdate"
/usr/bin/geoipupdate -d "$database_dir" -f "$conf_file" $flags 1>$log_file
/usr/bin/geoipupdate $flags 1>$log_file
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This didn't work as apparently we still need to provide a GeoIP.conf file.

if [ "$frequency" -eq 0 ]; then
break
fi
Expand Down
Loading