Initial commit

.gitignore

@@ -0,0 +1,160 @@
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[cod]
+*$py.class
+
+# C extensions
+*.so
+
+# Distribution / packaging
+.Python
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+share/python-wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+.python-version
+MANIFEST
+
+# PyInstaller
+#  Usually these files are written by a python script from a template before
+#  PyInstaller builds the exe, so as to inject date/other infos into it.
+*.manifest
+*.spec
+
+# Installer logs
+pip-log.txt
+pip-delete-this-directory.txt
+
+# Unit test / coverage reports
+htmlcov/
+.tox/
+.nox/
+.coverage
+.coverage.*
+.cache
+nosetests.xml
+coverage.xml
+*.cover
+*.py,cover
+.hypothesis/
+.pytest_cache/
+cover/
+
+# Translations
+*.mo
+*.pot
+
+# Django stuff:
+*.log
+local_settings.py
+db.sqlite3
+db.sqlite3-journal
+
+# Flask stuff:
+instance/
+.webassets-cache
+
+# Scrapy stuff:
+.scrapy
+
+# Sphinx documentation
+docs/_build/
+
+# PyBuilder
+.pybuilder/
+target/
+
+# Jupyter Notebook
+.ipynb_checkpoints
+
+# IPython
+profile_default/
+ipython_config.py
+
+# pyenv
+#   For a library or package, you might want to ignore these files since the
+#   code is intended to run in multiple environments; otherwise, check them in:
+.python-version
+
+# pipenv
+#   According to pypa/pipenv#598, it is recommended to include Pipfile.lock in
+#   version control. However, in case of collaboration, if having
+#   platform-specific dependencies or dependencies having no cross-platform
+#   support, pipenv may install dependencies that don't work, or not install
+#   all needed dependencies.
+#Pipfile.lock
+
+# poetry
+#   Similar to Pipfile.lock, it is generally recommended to include poetry.lock
+#   in version control. This is especially recommended for binary packages to
+#   ensure reproducibility, and is more commonly ignored for libraries.
+#   https://python-poetry.org/docs/basic-usage/#commit-your-poetrylock-file-to-version-control
+#poetry.lock
+
+# PEP 582; used by e.g. github.com/David-OConnor/pyflow
+__pypackages__/
+
+# Celery stuff
+celerybeat-schedule
+celerybeat.pid
+
+# SageMath parsed files
+*.sage.py
+
+# Environments
+.env
+.venv/
+env/
+venv/
+ENV/
+env.bak/
+venv.bak/
+
+# Spyder project settings
+.spyderproject
+.spyproject
+
+# Rope project settings
+.ropeproject
+
+# mkdocs documentation
+/site
+
+# mypy
+.mypy_cache/
+.dmypy.json
+dmypy.json
+
+# Pyre type checker
+.pyre/
+
+# pytype static type analyzer
+.pytype/
+
+# Cython debug symbols
+cython_debug/
+
+# PyCharm
+#  JetBrains specific template is maintainted in a separate JetBrains.gitignore
+#  that can be found at
+#  https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore and
+#  can be added to the global gitignore or merged into this file.  For a more
+#  nuclear option (not recommended) you can uncomment the following to ignore
+#  the entire idea folder.
+#.idea/
+
+# Custom exclusions
+data/
+.init/

Makefile

@@ -0,0 +1,69 @@
+all: help
+
+# --------------------------------------------
+
+.PHONY: setup
+setup: ## setup project with runtime dependencies
+ifeq (,$(wildcard .init/setup))
+	@(which poetry > /dev/null 2>&1) || \
+	(echo "banip requires poetry. See README for instructions."; exit 1)
+	@if [ ! -d "./data" ]; then \
+		mkdir data; \
+	fi
+	mkdir .init
+	touch .init/setup
+	poetry install --only=main
+else
+	@echo "Initial setup is already complete. If you are having issues, run:"
+	@echo
+	@echo "make reset"
+	@echo "make setup"
+	@echo
+endif
+
+# --------------------------------------------
+
+.PHONY: dev
+dev: ## add development dependencies (run make setup first)
+ifneq (,$(wildcard .init/setup))
+	poetry install
+	@touch .init/dev
+else
+	@echo "Please run \"make setup\" first"
+endif
+
+# --------------------------------------------
+
+.PHONY: update
+update: ## update banip code and dependencies
+	@echo Updating pymids
+	git pull
+	@echo Updating dependencies
+ifeq (,$(wildcard .init/dev))
+	poetry update --only=main
+else
+	poetry update
+endif
+
+# --------------------------------------------
+
+.PHONY: reset
+reset: clean ## remove venv, artifacts, and init directory
+	@echo Resetting project state
+	rm -rf .init .mypy_cache .venv
+
+# --------------------------------------------
+
+.PHONY: clean
+clean: ## cleanup python build artifacts
+	@echo Cleaning python build artifacts
+	@find . -type d -name __pycache__ -exec rm -rf {} \; -prune
+
+# --------------------------------------------
+
+.PHONY: help
+help: ## show help
+	@echo Please specify a target. Choices are:
+	@grep -E '^[a-zA-Z_-]+:.*?## .*$$' $(MAKEFILE_LIST) | sort | awk \
+	'BEGIN {FS = ":.*?## "}; \
+	{printf "\033[36m%-30s\033[0m %s\n", $$1, $$2}'

README.md

@@ -0,0 +1,138 @@
+# banip
+
+This tool will create a customized list of IP addresses that are
+cross-referenced between two sources:
+
+1. A global (worldwide) list of identified blacklisted IPs.
+2. A list of the IP subnets associated with each country.
+
+The result is a customized list of IP blacklisted addresses based on
+countries that you select.
+
+## Why not just use the source list of all blacklisted IPs?
+
+You could, but where's the fun in that?
+
+Using the second list above, I've configured my HAProxy server to drop
+IP connections from all countries except a few that I've whitelisted.
+What was missing was the ability to create a customized blacklist of bad
+actors, even if they're coming from countries that I whitelisted. This
+tool accomplishes that.
+
+## Requirements
+
+### poetry
+
+banip requires [poetry][def2] for dependency management. Poetry is well
+behaved and if you're a Python developer you should check it out. It
+installs itself in a virtual environment, uninstalls cleanly and easily,
+and doesn't require `sudo` for installation. Visit the [poetry
+site][def2] and install it using your preferred methods, with the
+instructions for your operating system.
+
+### gitignore (optional)
+
+If you want to fork and develop this repo, I've included a file called
+`global-gitignore.txt` which is a copy of the `.gitignore` I placed in
+my home directory and configured globally for all my development
+projects. The `global-gitignore.txt` file reflects my development setup
+(for example using tools like vscode), but yours may be different. Just
+cherry-pick any necessary elements from `global-gitignore.txt` for your
+own use.
+
+*Details on gitignore files are available on [GitHub][def3].*
+
+### List of subnets for all countries
+
+Download the list from [this site][def4].
+
+### List of blacklisted IPs
+
+Clone the [ipsum repository][def5] to a location of your choosing (let's
+say your home directory `~`). You'll need to copy a file from it later.
+
+### make
+
+You'll need the linux [make][def6] utility installed (*it probably
+already is*).
+
+## Setup
+
+Clone this repository. Let's assume your also clone it your home
+directory (`~`)
+
+Change to the repository (`cd ~/banip`) and run the command below:
+
+```shell
+make setup
+```
+
+Copy the following files as indicated below.
+
+### Country subnets
+
+```shell
+cp .../haproxy_geo_ip.txt ./data/haproxy_geo_ip.txt
+```
+
+### Blacklisted IPs
+
+```shell
+cp ~/ipsum/ipsum.txt ./data/ipsum.txt
+```
+
+### Target countries
+
+```shell
+cp sample-targets.txt ./data/targets.txt
+```
+
+Modify `./data/targets.txt` to select your desired target countries. The
+comments in the file will guide you.
+
+### Custom bans
+
+```shell
+cp sample-custom_bans.txt ./data/custom_bans.txt
+```
+
+These will be specific IP address or subnets (one per line, in
+[CIDR][def] format) that you want to block. Some of your IPs may be
+found when you run the tool, so this file (`custom_bans.txt`) will be
+overwritten to remove the duplicates. The contents of the de-duplicated
+file will be appended to the list generated when you run the program.
+
+*Note: If you're concerned about keeping your original list of custom
+bans, save a copy of it somewhere outside the repository.*
+
+## Running
+
+After copying/tweaking all the required files, start with this command
+to learn how to build your custom blacklist:
+
+```shell
+banip -h
+```
+
+## Updating
+
+The source lists of blacklisted IPs and country subnets are updated by
+their authors daily (sometimes twice daily). When you're ready to update
+your custom blacklist, start with this:
+
+```shell
+cp ~/ipsum
+git pull
+```
+
+Next, download a new copy of `haproxy_geo_ip.txt` as discussed above.
+Put new copies of `ipsum.txt` and `haproxy_geo_ip.txt` in `./data`.
+Tweak `./data/targets.txt` and `./data/custom_bans.txt` to your liking, and
+run `banip` again.
+
+[def]: https://aws.amazon.com/what-is/cidr/#:~:text=CIDR%20notation%20represents%20an%20IP,as%20192.168.1.0%2F22.
+[def2]: https://python-poetry.org/
+[def3]: https://docs.github.com/en/get-started/getting-started-with-git/ignoring-files
+[def4]: https://wetmore.ca/ip/
+[def5]: https://github.com/stamparm/ipsum
+[def6]: https://man7.org/linux/man-pages/man1/make.1p.html