Skip to content

vimt/MaxMind-DB-Writer-python

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

16 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

MaxMind-DB-Writer-python

Make mmdb format ip library file which can be read by maxmind official language reader

The official perl writer was written in perl, which was difficult to customize. So I implemented the MaxmindDB format ip library in python language.

MaxMind has now released an official Go version of the MMDB writer. If you prefer using Go, you can check out the official Go implementation mmdbwriter. This project still provides a Python alternative for those who need it.

Install

pip install -U mmdb_writer

Usage

from netaddr import IPSet

from mmdb_writer import MMDBWriter

writer = MMDBWriter()

writer.insert_network(IPSet(['1.1.0.0/24', '1.1.1.0/24']), {'country': 'COUNTRY', 'isp': 'ISP'})
writer.to_db_file('test.mmdb')

import maxminddb

m = maxminddb.open_database('test.mmdb')
r = m.get('1.1.1.1')
assert r == {'country': 'COUNTRY', 'isp': 'ISP'}

Examples

see csv_to_mmdb.py Here is a professional and clear translation of the README.md section from Chinese into English:

Using the Java Client

If you are using the Java client, you need to be careful to set the int_type parameter so that Java correctly recognizes the integer type in the MMDB file.

Example:

from mmdb_writer import MMDBWriter

writer = MMDBWriter(int_type='i32')

Alternatively, you can explicitly specify data types using the Type Enforcement section.

Underlying Principles

In Java, when deserializing to a structure, the numeric types will use the original MMDB numeric types. The specific conversion relationships are as follows:

mmdb type java type
float Float
double Double
int32 Integer
uint16 Integer
uint32 Long
uint64 BigInteger
uint128 BigInteger

When using the Python writer to generate an MMDB file, by default, it converts integers to the corresponding MMDB type based on the size of the int. For instance, int(1) would convert to uint16, and int(2**16+1) would convert to uint32. This may cause deserialization failures in Java clients. Therefore, it is necessary to specify the int_type parameter when generating MMDB files to define the numeric type accurately.

Type Enforcement

MMDB supports a variety of numeric types such as int32, uint16, uint32, uint64, uint128 for integers, and f32, f64 for floating points, while Python only has one integer type and one float type (actually f64).

Therefore, when generating an MMDB file, you need to specify the int_type parameter to define the numeric type of the MMDB file. The behaviors for different int_type settings are:

int_type Behavior
auto (default) Automatically selects the MMDB numeric type based on the value size.
Rules:
int32 for value < 0
uint16 for 0 <= value < 2^16
uint32 for 2^16 <= value < 2^32
uint64 for 2^32 <= value < 2^64
uint128 for value >= 2^64.
i32 Stores all integer types as int32.
u16 Stores all integer types as uint16.
u32 Stores all integer types as uint32.
u64 Stores all integer types as uint64.
u128 Stores all integer types as uint128.

If you want to use different int types for different scenarios, you can use type wrapping:

from mmdb_writer import MMDBWriter, MmdbI32, MmdbF32

writer = MMDBWriter()
# the value of field "i32" will be stored as int32 type
writer.insert_network(IPSet(["1.0.0.0/24"]), {"i32": MmdbI32(128), "f32": MmdbF32(1.22)})

Reference:

About

No description, website, or topics provided.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published