Skip to content

fix caching issue #13

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Sep 12, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 10 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,10 +12,18 @@

Make `mmdb` format ip library file which can be read by [`maxmind` official language reader](https://dev.maxmind.com/geoip/geoip2/downloadable/)

[The official perl writer](https://github.com/maxmind/MaxMind-DB-Writer-perl) was written in perl, which was difficult to customize. So I implemented the `MaxmindDB format` ip library in python language
~~[The official perl writer](https://github.com/maxmind/MaxMind-DB-Writer-perl) was written in perl,
which was difficult to customize.
So I implemented the `MaxmindDB format` ip library in python language.~~

MaxMind has now released an official Go version of the MMDB writer.
If you prefer using Go, you can check out the official Go implementation [mmdbwriter](https://github.com/maxmind/mmdbwriter).
This project still provides a Python alternative for those who need it.

## Install

```shell script
pip install -U git+https://github.com/VimT/MaxMind-DB-Writer-python
pip install -U mmdb_writer
```

## Usage
Expand Down
14 changes: 11 additions & 3 deletions mmdb_writer.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
__version__ = "0.2.2"
__version__ = "0.2.3"

import logging
import math
Expand Down Expand Up @@ -355,6 +355,13 @@ def python_type_id(self, value):
return MMDBTypeID.DOUBLE
raise TypeError(f"unknown type {value_type}")

def _freeze(self, value):
if isinstance(value, dict):
return tuple((k, self._freeze(v)) for k, v in value.items())
elif isinstance(value, list):
return tuple(self._freeze(v) for v in value)
return value

def encode_meta(self, meta):
res = self._make_header(MMDBTypeID.MAP, len(meta))
meta_type = {
Expand All @@ -373,8 +380,9 @@ def encode_meta(self, meta):

def encode(self, value, type_id=None):
if self.cache:
cache_key = self._freeze(value)
try:
return self.data_cache[id(value)]
return self.data_cache[cache_key]
except KeyError:
pass

Expand All @@ -401,7 +409,7 @@ def encode(self, value, type_id=None):
pointer_position = self.data_pointer
self.data_pointer += len(res)
pointer = self.encode(pointer_position, 1)
self.data_cache[id(value)] = pointer
self.data_cache[cache_key] = pointer
return pointer
return res

Expand Down