OSMBright incredibly slow for osm2pgsql output

[bkj]

I used osm2pgsql to load the OpenStreetMap planet.osm.pbf file into Postgres (27 GB in .osm.pbf format, ~350GB in the database). Now when I open the OSMBright project in Tilemill, all of machine's cores light up but no tiles show up. After waiting 5+ minutes layer 0 tiles show up and then a smattering of layer 1 and 2 tiles, but then it seems to stall out. Any ideas on what the bottleneck would be?

Is it possible that some index didn't get built in the database so it's doing some kind of much slower search? Or maybe I need to allocate more memory to Tilemill or Postgres?

I followed the instructions here
https://www.mapbox.com/tilemill/docs/guides/osm-bright-mac-quickstart/
for setup.

Thanks!
Ben

Some specs:
MacBook Pro, OS X 10.9.5, 16GB RAM, 8 CPUs, 100+ GB free disk space

[springmeyer]

TileMill is likely waiting on postgres. So, you could spend effort trying to tune your database. But instead what I would recommend is using Mapbox Studio instead. Because Mapbox Studio works off vector tiles instead of directly from the database rendering is much faster. We've provided an osm-bright starter for Studio here: https://github.com/mapbox/mapbox-studio-osm-bright.tm2

[xicesky]

Maybe related:
Please tell me if i should open a new issue, but i found a relatively slow query on my tile server which uses osm-bright. For the full details please view:
https://help.openstreetmap.org/questions/40718/tile-server-running-slow-queries

Do those queries actually depend on the style sheet? Is there a way to speed it up?

[springmeyer]

@xicesky In your above post you mention ST_IsValid and ST_PointOnSurface are slow. So then you could take queries like that which use complex where clauses and generate new tables using their where clauses. Then re-write the style to point at those pre-processed tables that contain the simplified info (just make sure to index them well).

[xicesky]

@springmeyer Thanks for the tip!
At this moment i am not sure why exactly those functions are running so slowly on such hardware. If i got that info right, the openstreetmap.org server is running very well on less expensive hardware, so i want to check if i encountered any weird bugs before trying to optimize for our use case / a specific stylesheet.

[springmeyer]

@xicesky openstreetmap.org does not use this stylesheet and therefore does not run the same queries. Take a look at https://github.com/gravitystorm/openstreetmap-carto for the stylesheet osm.org uses.

[bkj]

@springmeyer thanks for the answer. I downloaded Mapbox Studio and it seems like I'm running into a similar problem. I can try to add a PostGIS layer pointing at the local server, using "planet_osm_polygons" in the large box in the "SQL" tab. The map will start to render (maybe show one tile), but then it starts using lots of resources and eventually freezes. Can you think of anywhere else in the process where I might be going wrong? Or think of a test I could use to try to isolate the problem?

Thanks

[dionysiusmarquis]

Same here. Don't really know how to use my local PostGIS. I'm able to use one or two layers but then everything is getting real slow and I'm not able to save my .tm2source or use it. OSX 10.10.4. I really wanted to see how vector-tiles perform.

Edit:
You just have to add where z(!scale_denominator!) … to your queries. Followed: https://github.com/mapbox/postgis-vt-util

[andrew-aladev]

It is impossible to use world with osm2pgsql mml. Please add this information to readme. Mapnik's renderd ate 30GB and died with out of memory exception. It is impossible to use imposm 2, because it couldn't launch updates. We should move to imposm 3.