Skip to content

Commit

Permalink
Changelog (#34)
Browse files Browse the repository at this point in the history 
* Improved version number management.

* README linting.

* Fixed #32

* Updated CHANGELOG

* Finished CHANGELOG migration.

* CHANGELOG updates.

* Dependency Updates.

* Fixed help message in stats.py.

* Fixed help message in build.py.

* Fixed help message for check.py

* CHANGELOG update.

* Merge remote-tracking branch 'origin/main' into changelog
  • Loading branch information
@geozeke
geozeke authored Jan 11, 2025
1 parent edda7d4 commit f223785
Show file tree
Hide file tree
Showing 13 changed files with 392 additions and 100 deletions.
139 changes: 139 additions & 0 deletions CHANGELOG.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,139 @@
# Changelog

<!--------------------------------------------------------------------->

## [1.2.0][1.2.0] - 2025-01-11

### Changed

* Implement [tomli compatability layer][tomli].
* Optimize version numbering.

### Added

* Add statistics for a given country. ([#32][issue32])
* Establish and maintain a proper changelog.

### Fixed

* Lint documentation.
* Display properly sorted options when getting help.

<!--------------------------------------------------------------------->

## [1.1.3][1.1.3] - 2024-12-19

### Added

* Add the ability to compact ipsum entries into /24 subnets.

### Fixed

* Improve input validation.
* Lint documentation.
* Refactor and optimize code.

<!--------------------------------------------------------------------->

## [1.1.2][1.1.2] - 2024-12-02

### Changed

* Tune binary search algorithm.
* Enhance `banip check` with prettier output using [rich][rich].

### Fixed

* Lint documentation.

<!--------------------------------------------------------------------->

## [1.1.1][1.1.1] - 2024-11-15

### Changed

* Get better output with the [rich][rich] library.

<!--------------------------------------------------------------------->

## [1.1.0][1.1.0] - 2024-10-23

### Changed

* Refine display of final metrics.

### Fixed

* Refactor code for better maintainability.

<!--------------------------------------------------------------------->

## [1.0.2][1.0.2] - 2024-08-09

### Changed

* Optimize binary search. ([#13][issue13])
* Bumped tqdm library to v4.66.5.

### Added

* Add functionality to remove IP addresses captured in a subnets.
* Add license and acknowledgements for the tqmd library

### Fixed

* Fix calculation error in summary metrics.
* Choose Better Variable Names. ([#14][issue14])
* Improve IP checking to include membership in subnets. ([#15][issue15])

<!--------------------------------------------------------------------->

## [1.0.1][1.0.1] - 2024-07-30

### Added

* Add separate command line option to display version information.
* Add additional help indicators to the subcommands.

### Fixed

* Refactor code for better maintainability.

<!--------------------------------------------------------------------->

## [1.0.0][1.0.0] - 2024-03-15

### Changed

* Update documentation with page anchors.

### Added

* Introduce a plugin architecture.

### Fixed

* Fixed uncaught exception.
* Lint documentation.

<!--------------------------------------------------------------------->

## [0.1.0][0.1.0] - 2024-05-12

_Initial Release._

[0.1.0]: https://github.com/geozeke/banip/releases/tag/v0.1.0
[1.0.0]: https://github.com/geozeke/banip/releases/tag/V1.0.0
[1.0.1]: https://github.com/geozeke/banip/releases/tag/v1.0.1
[1.0.2]: https://github.com/geozeke/banip/releases/tag/v1.0.2
[issue13]: https://github.com/geozeke/banip/issues/13
[issue14]: https://github.com/geozeke/banip/issues/14
[issue15]: https://github.com/geozeke/banip/issues/15
[1.1.0]: https://github.com/geozeke/banip/releases/tag/v1.1.0
[1.1.1]: https://github.com/geozeke/banip/releases/tag/v1.1.1
[rich]: https://github.com/Textualize/rich
[1.1.2]: https://github.com/geozeke/banip/releases/tag/v1.1.2
[1.1.3]: https://github.com/geozeke/banip/releases/tag/v1.1.3
[tomli]: https://pypi.org/project/tomli/
[1.2.0]: https://github.com/geozeke/banip/releases/tag/v1.2.0
[issue32]: https://github.com/geozeke/banip/issues/32
87 changes: 43 additions & 44 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ alt = "Dinobox logo" width="120"/>
This tool will create a customized list of IP addresses that are
cross-referenced between two sources:

1. [This list][def10] of worldwide identified blacklisted IPs.
1. [This list][ipsum] of worldwide identified blacklisted IPs.
2. A list of the IP subnets associated with each country.

The result is a customized blacklist of IP addresses based on
Expand All @@ -19,11 +19,11 @@ You could, but where's the fun in that?

You may want to create a list of bad actors for specific countries. The
global list contains several hundred thousand entries, and you may need
more targeted list for testing or deployment in production. For example,
I've configured my HAProxy server to drop IP connections from all
countries except those that I've whitelisted. I also want the ability to
create a customized IP list to block any bad actors from those
whitelisted countries. This tool accomplishes that.
a more targeted list for testing or deployment in production. For
example, I've configured my HAProxy server to drop IP connections from
all countries except those that I've whitelisted. I also want the
ability to create a customized IP list to block any bad actors from
those whitelisted countries. This tool accomplishes that.

## Contents

Expand All @@ -39,15 +39,15 @@ whitelisted countries. This tool accomplishes that.
### Operating System

*banip* runs in Unix-like OSes. Either macOS, a Linux PC, Linux Virtual
Machine, or [Windows Subsystem for Linux (WSL)][def7] is required.
Machine, or [Windows Subsystem for Linux (WSL)][wsl] is required.

### MaxMind Database

You'll need a copy of the [MaxMind][def8] GeoLite2 database for
You'll need a copy of the [MaxMind][mmh] GeoLite2 database for
country-level geotagging of IP addresses. If you have a premium or
corporate MaxMind account, you're all set. If not, the free GeoLite2
account will work just fine ([sign up here][def5]). Once you login,
using the menu on the top right select:
account will work just fine ([sign up here][mmgeo]). Once you log in,
using the menu on the top right, select:

```text
My Account > My Account
Expand All @@ -62,9 +62,9 @@ GeoLite2 Country: CSV format

### uv

*banip* requires [uv][def2] for dependency management. It is well
behaved and extremely fast, and if you're a Python developer you should
check it out. Visit the [uv site][def2] and install it using the
*banip* requires [uv][astral] for dependency management. It is well
behaved and extremely fast, and if you're a Python developer, you should
check it out. Visit the [uv site][astral] and install it using the
instructions for your operating system.

### gitignore (optional)
Expand All @@ -73,15 +73,15 @@ If you want to fork and develop this repo, I've included a file called
`global-gitignore.txt` which is a copy of the `.gitignore` I placed in
my home directory and configured globally for all my development
projects. The `global-gitignore.txt` file reflects my development setup
(for example using tools like vscode), but yours may be different. Just
cherrypick any necessary elements from `global-gitignore.txt` for your
(for example, using tools like vscode), but yours may be different. Just
cherry-pick any necessary elements from `global-gitignore.txt` for your
own use.

*Details on gitignore files are available on [GitHub][def3].*
*Details on gitignore files are available on [GitHub][git-ignore].*

### Global List of Blacklisted IPs

*banip* uses the [ipsum][def9] threat intelligence blacklist. You can
*banip* uses the [ipsum][ipsum] threat intelligence blacklist. You can
direct download it using:

```shell
Expand All @@ -90,8 +90,8 @@ curl -sL https://raw.githubusercontent.com/stamparm/ipsum/master/ipsum.txt > ips

### make

You'll need the [make][def6] utility installed (*it probably
already is*). If not, install it with:
You'll need the [make][make] utility installed (it probably already is).
If not, install it with:

```shell
sudo apt install make
Expand All @@ -113,7 +113,7 @@ Unpack the GeoLite2-Country zip archive and save the files to a location
you can easily get to.

*Note: if you're looking for a quick way to download the MaxMind data
using `curl` and a direct download permalink, [SEE HERE][def4].*
using `curl` and a direct download permalink, [SEE HERE][mmd].*

### Clone the Repository

Expand Down Expand Up @@ -164,10 +164,10 @@ cp ./samples/custom_whitelist.txt ./data/custom_whitelist.txt
```

There may be IP addresses that *banip* will flag as malicious, but you
still want to whitelist them (for example to use for testing). This file
should contain specific IP addresses, one per line, that you want to
allow. This file is optional and if you choose not to use it, *banip*
will create a blank one for you.
still want to whitelist them (for example, to use for testing). This
file should contain specific IP addresses, one per line, that you want
to allow. This file is optional, and if you choose not to use it,
*banip* will create a blank one for you.

#### Custom Blacklist (Optional)

Expand All @@ -176,16 +176,16 @@ cp ./samples/custom_blacklist.txt ./data/custom_blacklist.txt
```

The ipsum database isn't perfect. You may determine that there's an IP
address you want to ban this is not found in `ipsum.txt`. Also, the
`ipsum.txt` file only contains IP addresses and you may want to ban an
address you want to ban that is not found in `ipsum.txt`. Also, the
`ipsum.txt` file only contains IP addresses, and you may want to ban an
entire subnet. The custom blacklist allows you to capture specific IP
addresses or subnets (in [CIDR][def] format), one per line, that you
addresses or subnets (in [CIDR][cidr] format), one per line, that you
want to block. Some of your custom blacklist IPs may be found when you
run the *banip*, so this file (`custom_blacklist.txt`) will be
overwritten to remove the duplicates. The contents of the de-duplicated
file will then be appended to the list generated when you run the
program. Like the whitelist, this file is optional. If you choose not to
use it *banip* will create a blank one when you run it.
use it, *banip* will create a blank one when you run it.

*Note: If you're concerned about keeping your original list of custom
blacklisted IPs, save a copy of it somewhere outside the repository.*
Expand Down Expand Up @@ -218,7 +218,7 @@ data
## <a id="running"></a> Running

After copying/tweaking all the required files, start by activating the
python virtual environment:
Python virtual environment:

```shell
source .venv/bin/activate
Expand All @@ -234,7 +234,7 @@ banip -h

## <a id="updating"></a> Updating

MaxMind updates the GeoLite2 Country database on Tuesdays and Fridays
MaxMind updates the GeoLite2 Country database on Tuesdays and Fridays,
and the list of blacklisted IPs (`ipsum.txt`) is updated daily. Pull
updated copies of both and put them in `banip/data/geolite` (for the
GeoLite2 data) and `banip/data` (for the `ipsum.txt` file). Run *banip*
Expand All @@ -247,16 +247,16 @@ again to generate an updated blacklist.
## <a id="plugins"></a> Plugins

*banip* generates some useful build products that you may want to use
for other purposes. For example, everytime you build a new blacklist
*banip* also creates and saves a textfile of all worldwide subnets, each
tagged with a two-letter country code. The file is saved in:
for other purposes. For example, every time you build a new blacklist,
*banip* also creates and saves a text file of all worldwide subnets,
each tagged with a two-letter country code. The file is saved in:

```'text
./banip/data/haproxy_geo_ip.txt
```

Next time you run *banip*, open that file and take a look at it. Since
you may have a very specific usecase for that data, you can write a
you may have a very specific use case for that data, you can write a
plugin for *banip* which will make use of the build products for your
purposes.

Expand Down Expand Up @@ -288,13 +288,12 @@ rm -rf ~/banip

[top](#top)

[def]: https://aws.amazon.com/what-is/cidr/#:~:text=CIDR%20notation%20represents%20an%20IP,as%20192.168.1.0%2F22.
[def2]: https://docs.astral.sh/uv/
[def3]: https://docs.github.com/en/get-started/getting-started-with-git/ignoring-files
[def4]: https://dev.maxmind.com/geoip/updating-databases#directly-downloading-databases
[def5]: https://dev.maxmind.com/geoip/geolite2-free-geolocation-data
[def6]: https://man7.org/linux/man-pages/man1/make.1p.html
[def7]: https://docs.microsoft.com/en-us/windows/wsl/install
[def8]: https://www.maxmind.com/en/home
[def9]: https://github.com/stamparm/ipsum
[def10]: https://github.com/stamparm/ipsum
[cidr]: https://aws.amazon.com/what-is/cidr/#:~:text=CIDR%20notation%20represents%20an%20IP,as%20192.168.1.0%2F22.
[astral]: https://docs.astral.sh/uv/
[git-ignore]: https://docs.github.com/en/get-started/getting-started-with-git/ignoring-files
[mmd]: https://dev.maxmind.com/geoip/updating-databases#directly-downloading-databases
[mmgeo]: https://dev.maxmind.com/geoip/geolite2-free-geolocation-data
[make]: https://man7.org/linux/man-pages/man1/make.1p.html
[wsl]: https://docs.microsoft.com/en-us/windows/wsl/install
[mmh]: https://www.maxmind.com/en/home
[ipsum]: https://github.com/stamparm/ipsum
18 changes: 11 additions & 7 deletions pyproject.toml
View file
Original file line number Diff line number Diff line change
@@ -1,18 +1,25 @@
[project]
name = "banip"
version = "1.1.3"
version = "1.2.0"
description = "Create a list of banned IPs for specific countries"
license = {file = "LICENSE"}
readme = {file = "README.md", content-type = "text/markdown"}
requires-python = ">=3.12,<3.13"
dependencies = [
"banip",
"rich>=13.9.4",
"tomli>=1.1.0 ; python_full_version < '3.11'",
]
authors = [
{name = "Peter Nardi", email = "[email protected]"},
]

[dependency-groups]
dev = [
"mypy>=1.13.0",
"ruff>=0.7.1",
]

[tool.ruff.lint.isort]
force-single-line = true

Expand All @@ -26,12 +33,6 @@ banip = "banip.app:main"
requires = ["hatchling"]
build-backend = "hatchling.build"

[dependency-groups]
dev = [
"mypy>=1.13.0",
"ruff>=0.7.1",
]

[tool.hatch.build]
exclude = [
".gitignore",
Expand All @@ -40,3 +41,6 @@ exclude = [
"samples/**",
"uv.lock",
]

[tool.hatch.build.targets.wheel.force-include]
"./pyproject.toml" = "banip/pyproject.toml"
4 changes: 2 additions & 2 deletions samples/targets.txt
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,8 @@
# are ignored. NOTE: These are the ISO-3166 ALPHA2 codes, not the
# two-letter Top Level Domain names (which may be different). For
# example, the two-letter TLD for the United Kingdom is "uk", but the
# ISO-3166 code for United Kingdom is "gb". You can find a list of all
# the codes here: https://www.geonames.org/countries/
# ISO-3166 code for the United Kingdom is "gb". You can find a list of
# all the codes here: https://www.geonames.org/countries/

# Norway
no
Expand Down
Loading

0 comments on commit f223785

Please sign in to comment.