forked from openmaptiles/openmaptiles-tools
-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathmbtiles-tools
executable file
·188 lines (169 loc) · 9.48 KB
/
mbtiles-tools
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
#!/usr/bin/env python
"""
A toolbox to manage the mbtiles file.
Usage:
mbtiles-tools find-dups <mbtiles-file> [--output=<file>] [--verbose]
[--zoom=<zoom>] [--min-dups=<count>]
mbtiles-tools impute <mbtiles-file> --zoom=<zoom>
[--key=<hash>... | --keyfile=<file>] [--output=<file>]
[--min-dups=<count>] [--verbose]
mbtiles-tools copy <mbtiles-file> <target-mbtiles-file>
([--zoom=<zoom>]... | [--minzoom=<min>] [--maxzoom=<max>])
[--reset] [--auto-minmax] [--show-json] [--show-ranges]
[--exist=<ignore|replace|fail>] [--bbox=<bbox>] [--verbose]
mbtiles-tools meta-all <mbtiles-file> [--show-json] [--show-ranges]
mbtiles-tools meta-get <mbtiles-file> <metakey>
mbtiles-tools meta-set <mbtiles-file> <metakey> [<newvalue>]
mbtiles-tools meta-generate <mbtiles-file> <tileset>
[--reset] [--auto-minmax] [--show-json] [--show-ranges]
[--pghost=<host>] [--pgport=<port>] [--dbname=<db>]
[--user=<user>] [--password=<password>]
mbtiles-tools meta-copy <mbtiles-file> <target-mbtiles-file>
[--reset] [--auto-minmax] [--show-json] [--show-ranges]
mbtiles-tools tile <mbtiles-file> <tile_zxy> [--show-names] [--summary]
mbtiles-tools --help
mbtiles-tools --version
<mbtiles-file> sqlite3 database file with .mbtiles extension
<tileset> Tileset definition yaml file
Actions:
find-dups Get a list of duplicate tile keys at a given zoom levels (optional).
By default requires at least 20 (50 for z13+) duplicates to be shown.
impute Find all duplicate tiles at zoom "<zoom> - 1" (same as find-dups), or
use given keys (md5 hashes as strings or a file). For each found tile,
add tiles at the <zoom> level with the same content as the found ones.
Example: given a key (hash) for an empty water tile, one could copy all
such water tiles from zoom 13 to zoom 14 by using --zoom=14 --key=<hash>.
Alternatively, without the --key/--keyfile param, all tiles used more
than 50 times will be copied from z13 to z14 with just --zoom=14 param.
Use --output to record all tiles that were NOT imputed (i.e. still need
to be generated) to a tile text file ("z/x/y" - one per line).
By default requires at least 20 (50 for z13+) duplicates.
copy Copy tiles from one mbtiles file to another. Copying can be limited by
one or more zooms, and a bounding box. This action also runs meta-copy command.
meta-all validates and prints all values in the metadata table
meta-get Gets a single value from metadata table
meta-set Sets a single value in the metadata table, or deletes it if no value.
meta-generate Initialize metadata table with the values from Postgres and tileset,
optionally overriding the bounding box and other params with env vars:
BBOX, CENTER_ZOOM -- center zoom will only be used if BBOX is set
MIN_ZOOM, MAX_ZOOM -- will override tileset except if --auto-minmax is set
METADATA_ATTRIBUTION, METADATA_DESCRIPTION, METADATA_NAME, METADATA_VERSION
Requires a database connection.
meta-copy Copy all metadata from one mbtile file to another.
While copying, values could also be updated by the same env vars as
meta-generate, and can also be modified with --auto-minmax.
tile Print tile content, where <tile_zxy> is the tile ID,
e.g. "10/4/8" for zoom=10, x=4, y=8. By default hides all "name:*" tags.
Options:
-z --zoom=<zoom> Process a single zoom. For impute, if set, ignores min/max.
-i --min-dups=<count> Number of times a tile is repeated before considering it a dup.
-m --minzoom=<min> Process all zooms beginning with this one (inclusive).
-n --maxzoom=<max> Process all zooms ending at this one (inclusive).
-e --exist=<mode> When copying tiles, specify what to do if tile with the same
coordinates already exists in the target file.
-k --key=<hash> Key of the tile (i.e. an MD5 hash). Could be multiple.
-f --keyfile=<file> A file with tile Keys, one per line.
-o --output=<file> Write a list of tiles to this file. Use '-' for stdout.
-j --show-json Print out the entire JSON field (compact-pretty-printed) when showing metadata.
-r --show-ranges Show tile counts and ranges statistics for each zoom in mbtiles.
-v --verbose Print additional debugging information.
--reset For meta-generate, copy, and meta-copy, clear all existing metadata first.
-a --auto-minmax For meta-generate, copy, and meta-copy, query data tables for available min/max zoom.
--show-names if set, includes all localized names when showing tile content
--summary if set, show just the content summary when showing tile content
--help Show this screen.
--version Show version.
PostgreSQL Options:
-h --pghost=<host> Postgres hostname. By default uses PGHOST env or "localhost" if not set.
-P --pgport=<port> Postgres port. By default uses PGPORT env or "5432" if not set.
-d --dbname=<db> Postgres db name. By default uses PGDATABASE env or "openmaptiles" if not set.
-U --user=<user> Postgres user. By default uses PGUSER env or "openmaptiles" if not set.
--password=<password> Postgres password. By default uses PGPASSWORD env or "openmaptiles" if not set.
These legacy environment variables should not be used, but they are still supported:
POSTGRES_HOST, POSTGRES_PORT, POSTGRES_DB, POSTGRES_USER, POSTGRES_PASSWORD
"""
import asyncio
from pathlib import Path
from docopt import docopt, DocoptExit
import openmaptiles
from openmaptiles.mbtile_tools import Imputer, KeyFinder, Metadata, TileCopier
from openmaptiles.pgutils import parse_pg_args
from openmaptiles.utils import parse_zxy_param, parse_zoom, parse_zoom_list, Bbox
def main():
args = docopt(__doc__, version=openmaptiles.__version__)
if args['find-dups']:
find_dups(args)
elif args['impute']:
impute(args)
elif args['copy']:
if args['--exist'] is None or args['--exist'] == 'ignore':
on_conflict = 'IGNORE'
elif args['--exist'] == 'replace':
on_conflict = 'REPLACE'
elif args['--exist'] == 'fail':
on_conflict = 'FAIL'
else:
raise DocoptExit('Invalid value for --exist parameter.')
TileCopier(source=Metadata(args['<mbtiles-file>'], args['--show-json'],
args['--show-ranges']),
target=args['<target-mbtiles-file>'],
zooms=parse_zoom_list(args['--zoom']),
minzoom=parse_zoom(args['--minzoom']),
maxzoom=parse_zoom(args['--maxzoom']),
reset=args['--reset'],
on_conflict=on_conflict,
auto_minmax=args['--auto-minmax'],
bbox=Bbox(bbox=args['--bbox']) if args['--bbox'] else None,
verbose=bool(args['--verbose'])).run()
elif args['meta-all']:
Metadata(args['<mbtiles-file>'], args['--show-json'],
args['--show-ranges']).print_all()
elif args['meta-get']:
Metadata(args['<mbtiles-file>']).get_value(args['<metakey>'])
elif args['meta-set']:
Metadata(args['<mbtiles-file>']).set_value(args['<metakey>'],
args['<newvalue>'])
elif args['meta-generate']:
pghost, pgport, dbname, user, password = parse_pg_args(args)
asyncio.run(Metadata(args['<mbtiles-file>'], args['--show-json'],
args['--show-ranges']).generate(
args['<tileset>'], reset=args['--reset'], auto_minmax=args['--auto-minmax'],
pghost=pghost, pgport=pgport, dbname=dbname, user=user, password=password))
elif args['meta-copy']:
Metadata(args['<mbtiles-file>'], args['--show-json'],
args['--show-ranges']).copy(
args['<target-mbtiles-file>'], reset=args['--reset'],
auto_minmax=args['--auto-minmax'])
elif args['tile']:
Metadata(args['<mbtiles-file>']).show_tile(
*parse_zxy_param(args['<tile_zxy>']),
show_names=args['--show-names'],
summary=args['--summary'])
else:
raise DocoptExit('Invalid command')
def find_dups(args):
KeyFinder(
mbtiles=args['<mbtiles-file>'],
outfile=args['--output'] or '-',
zoom=parse_zoom(args['--zoom']),
min_dup_count=args['--min-dups'],
verbose=args['--verbose'] or (args['--output'] and args['--output'] != '-'),
).run()
def impute(args):
file = args['<mbtiles-file>']
verbose = args['--verbose']
zoom = parse_zoom(args['--zoom'])
if zoom <= 0:
raise DocoptExit(f'Zooms must be more than 0, {zoom} was given.')
outfile = args['--output']
if args['--keyfile'] is not None:
content = Path(args['--keyfile']).read_text()
keys = [v.strip() for v in content.split('\n') if v.strip()]
elif args['--key']:
keys = args['--key']
else:
keys = KeyFinder(file, show_size=False, zoom=zoom - 1,
min_dup_count=args['--min-dups']).run()
Imputer(file, keys, zoom, outfile, verbose).run()
if __name__ == '__main__':
main()