Last active
November 13, 2015 18:17
-
-
Save samuelschumacher/dbe0b836e499e781371c to your computer and use it in GitHub Desktop.
Serving Vector Tiles from a Linux Server- Tilestache + PostgreSQL + Mapnik + Leaflet
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| /*Resources: | |
| PostGIS with Tilestache + Leaflet: https://github.com/NelsonMinar/vector-river-map (Could be great for PLSSFinder reset, includes basemap and server configuration), jump to #extra-ubuntu-1404-details | |
| http://mattmakesmaps.com/blog/2013/10/09/tilestache-rendering-topojson/ | |
| Follow the guide in the first link to install TileStache and Gunicorn. The data folder includes a pre-build .cfg file. When we start our server, we will need a location for this .cfg file and the path will be specified using Tilestache's -c option. | |
| Our server is being set up on 144.92.235.143. | |
| Ownership of wisttest (field data beta database) is lc_public with password = trees. | |
| Ownership of lcfr (old field data, possibly finalized field data repo db?) is sdschumacher. | |
| I have manually imported the data using:*/ | |
| shp2pgsql -s 3071 <filename> <table> lcfr | psql lcfr | |
| /*Tables: | |
| -------------- | |
| agriculture | |
| barren | |
| forest | |
| grassland | |
| shrubland | |
| urban | |
| water*/ | |
| --Turn this into one table: data_2014 | |
| CREATE TABLE data_2014 (gid serial, pointid varchar, lc_class varchar, lev1 varchar, lev2 varchar, lev3 varchar, lev4 varchar); | |
| SELECT AddGeometryColumn('public', 'data_2014', 'geom', 3071, 'POINT', 2); | |
| INSERT INTO data_2014 (pointid, lc_class, lev1, lev2, lev3, lev4, geom) SELECT pointid, lc_class, lev1, lev2, lev3, lev4, geom FROM agriculture; | |
| --etc for other tables | |
| --Then delete | |
| DROP TABLE agcriculture; --etc | |
| VACUUM FULL ANALYZE; | |
| CREATE INDEX data_2014_gist ON data_2014 USING gist (geom); | |
| CLUSTER data_2014 USING data_2014_gist;--physically rearrange data by location, possibly helpful to speed up bounding box queries | |
| VACUUM FULL ANALYZE; |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| From https://github.com/NelsonMinar/vector-river-map (Could be great for PLSSFinder reset, includes basemap and server configuration) | |
| We should set up an NGINX server if we reconfigure the Ubuntu Servers. | |
| Mapbox studio creates vector tiles using protocol buffer format. This is efficient, but also not supported by tilestache currently. We can, however, render JSONs from a postgres database and use the cartoCSS from mapbox for styling. | |
| If one would like to open a project created from a different machine, paths must be changed in the following files (using LCFRstyle4 and LCFRtiles for example, your names may vary): | |
| LCFRstyle4.tm2/project.yml | |
| LCFRstyle4.tm2/project.xml | |
| LCFRtiles.tm2source/project.yml | |
| LCFRtiles.tm2source/project.xml | |
| We can see that the styling can be applied to the shapefiles, but to what end? | |
| We must serve them somehow, although at this time TileStache may be out of the question, as Mapbox yields no simple way to combine the formatting and vector tiles without upload to its site. . | |
| So until more light is shed on the matter, we'll go here : https://github.com/mapbox/mapbox-gl-leaflet/blob/master/examples/basic.html | |
| And configure the mapboxGL object similar to the 'authorize' section of https://www.mapbox.com/mapbox-gl/ | |
| Still need to figure out what's going on with CartoCSS... hopefully style json can include tile stylings, or styles will export with MapBox project!!! | |
| Style reference: https://www.mapbox.com/mapbox-gl-style-spec/ |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| On second thought, let's not use Windows. | |
| "No JSON object could be decoded" | |
| Your text file likely is saving Byte Order Marks before and after your file. Python doesn't like that, so convert to no BOM UTF8. | |
| http://bugs.python.org/issue18958 | |
| "Invalid \escape" | |
| There are a lot of other characters that Python doesn't like, such as tabs, and unescaped slashes. Use double slashes in paths, and remove tabs to simplify the JSON. | |
| "%1 is not a valid Win32 application" | |
| This issue has something to do with the OGR installation. | |
| . | |
| https://groups.google.com/forum/#!msg/tilestache/-mdikI97WDM/bj_5acb1eyEJ |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| #Get to a cozy directory, then download the project | |
| git clone https://github.com/NelsonMinar/vector-river-map.git | |
| #Take a look at the example config file in the vector-river-map file you've downloaded. | |
| cd vector-river-map | |
| nano server/tilestache.cfg | |
| #One query per zoom level, it seems. See my adaptation of it below. | |
| #missing pyproj | |
| sudo pip install pyproj |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| <div><h3>Misc. References Being Used Now:</h3> | |
| <a href="http://blog.thematicmapping.org/2012/07/using-custom-projections-with-tilecache.html">Custom Tilecache projections</a><br> | |
| <a href="http://openstreetmap.us/~migurski/vector-datasource/">Mapnik Vector Tiles</a><br> | |
| <a href="http://tilestache.org/doc/TileStache.Goodies.VecTiles.html">Tilestache VecTiles Provider Manpage</a><br> | |
| <a href="https://github.com/mapnik/Cascadenik/wiki/Examples">Cascadenik: Understanding Mapnik XML vs CartoCSS</a><br> | |
| <a href="http://tilestache.org/doc/#layers">Tilestache API</a><br> | |
| <a href="http://gis.stackexchange.com/questions/42206/how-do-i-project-my-wms-tilelayer-in-leaflet-if-i-want-it-to-display-as-an-overl">Can Leaflet be configured to project on-the-fly?</a><br> | |
| <a href="https://github.com/TileStache/TileStache/blob/master/TileStache/Goodies/Proj4Projection.py">TileStache Proj4Projection.py code</a><br> | |
| <a href="http://mattmakesmaps.com/blog/2013/10/09/tilestache-rendering-topojson/">Generate TopoJSON Vector Tiles from PostGIS DB</a><br> | |
| </div> | |
| <div><h3>Leaflet Configuration:</h3><br> | |
| <a href="http://leafletjs.com/examples/geojson.html">Configuring GeoJSONS (find pointToLayer)</a><br> | |
| <a href="http://leafletjs.com/reference.html#geojson-options">General Leaflet GeoJSON options</a><br> | |
| <a href="https://github.com/glenrobertson/leaflet-tilelayer-geojson">L.TileLayer.GeoJSON Github</a><br> | |
| <a href="https://groups.google.com/forum/#!topic/leaflet-js/J9eRrfFhKbY"></a>Changing marker icons in Leaflet for point layers derived from JSON input<br> | |
| <a href="http://stackoverflow.com/questions/25364072/how-to-use-circle-markers-with-leaflet-tilelayer-geojson/28117172#28117172">Is this how you use Circle Markers with JSON points?</a><br> | |
| <a href="http://dnrmaps.wi.gov/arcgis/rest/services/DW_Map_Cached/EN_Public_Lands_Basemap_WTM_Ext/MapServer">DNR TMS stuff</a><br> | |
| </div> | |
| <div><h3>PROJ4 Configuration:</h3><br> | |
| <a href="http://blog.kartena.se/local-projections-in-a-world-of-spherical-mercator/">Detailed blog post outlining PROJ4 tileset configuration</a><br> | |
| <a href="http://tilestache.org/doc/TileStache.Goodies.Proj4Projection.html">TileStache PROJ4 Man Page</a><br> | |
| <a href="https://github.com/TileStache/TileStache/issues/206">Issue Samuel Opened regarding lack of custom resolution support in TileStache</a><br> | |
| <a href=""></a><br> | |
| </div> | |
| <div><h3>Setting Up:</h3> | |
| <a href="http://www.itsbalamurali.tk/install-setup-tilestache-maptiles-server-ubuntu-server/">Set up a TileStache Server on Ubuntu, front to back (watch for typos)</a><br> | |
| <a href="https://github.com/NelsonMinar/vector-river-map">Vector Map Tilestache Example</a><br> | |
| </div> | |
| <div><h3>Possible ToDo:</h3><br> | |
| <a href="http://www.itopen.it/2012/01/17/serving-your-map-tiles-30-times-faster/">Serve from NGINX</a><br> | |
| <a href="http://serverfault.com/questions/331256/why-do-i-need-nginx-and-something-like-gunicorn">(possibly deprecated) Why to use a two-tier server (NGINX, gunicorn)</a><br> | |
| <a href="http://wiki.nginx.org/FullExample">NGINX Config Example</a><br> | |
| <a href="http://www.reddit.com/r/Python/comments/1rculd/choice_of_wsgi_server_mod_wsgi_uwsgi_something/">If you should have trouble with Apache restarting processes... Use a non-embedded Python Server</a><br> | |
| <a href="https://www.digitalocean.com/community/questions/allow-php-to-delete-nginx-cache-files">If we want to serve dynamic layers through vecTiles, start here when designing scripts to trigger deletion of updated tiles</a><br> | |
| <a href="http://cramer.io/2013/06/27/serving-python-web-applications/">Using NGINX + uWSGI</a><br> | |
| <a href=""></a><br> | |
| </div> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| { | |
| "cache": { | |
| "name": "Disk", | |
| "path": "/tmp/stache", | |
| "dirs": "portable" | |
| }, | |
| "logging": "warning", | |
| "layers": { | |
| "rivers": { | |
| "allowed origin": "*", | |
| "provider": { | |
| "class": "TileStache.Goodies.VecTiles:Provider", | |
| "kwargs": { | |
| "clip": true, | |
| "dbinfo": { | |
| "host": "localhost", | |
| "database": "lcfr" | |
| }, | |
| "queries": [ | |
| ;Zoom levels 0-5 are null, min zoom is 6 | |
| null, null, null, | |
| "select geom as __geometry__, lc_class, lev1, lev2, lev3, lev4 from data_2014 where ", | |
| "select geom as __geometry__, lc_class, lev1, lev2, lev3, lev4 from data_2014 where ", | |
| "select geom as __geometry__, lc_class, lev1, lev2, lev3, lev4 from data_2014 where ", | |
| "select geom as __geometry__, lc_class, lev1, lev2, lev3, lev4 from data_2014 where ", | |
| "select geom as __geometry__, lc_class, lev1, lev2, lev3, lev4 from data_2014 where ", | |
| "select geom as __geometry__, lc_class, lev1, lev2, lev3, lev4 from data_2014 where ", | |
| "select geom as __geometry__, lc_class, lev1, lev2, lev3, lev4 from data_2014 where ", | |
| "select geom as __geometry__, lc_class, lev1, lev2, lev3, lev4 from data_2014 where ", | |
| "select geom as __geometry__, lc_class, lev1, lev2, lev3, lev4 from data_2014 where ", | |
| "select geom as __geometry__, lc_class, lev1, lev2, lev3, lev4 from data_2014 where ", | |
| "select geom as __geometry__, lc_class, lev1, lev2, lev3, lev4 from data_2014 where ", | |
| "select geom as __geometry__, lc_class, lev1, lev2, lev3, lev4 from data_2014 where ", | |
| "select geom as __geometry__, lc_class, lev1, lev2, lev3, lev4 from data_2014 where ", | |
| "select geom as __geometry__, lc_class, lev1, lev2, lev3, lev4 from data_2014 where ", | |
| "select geom as __geometry__, lc_class, lev1, lev2, lev3, lev4 from data_2014 where ", | |
| "select geom as __geometry__, lc_class, lev1, lev2, lev3, lev4 from data_2014 where " | |
| ] | |
| } | |
| }, | |
| "projection": "TileStache.Goodies.Proj4Projection:Proj4Projection('+proj=tmerc +lat_0=0 +lon_0=-90 +k=0.9996 +x_0=520000 +y_0=-4480000 +ellps=GRS80 +units=m +no_defs', [2145.796291592583, 1609.3472186944375, 1072.8981457962916, 536.4490728981458, 268.2245364490729, 134.11226822453645, 67.05613411226823, 33.52806705613411, 25.146050292100586, 16.764033528067056, 12.57302514210503, 8.382016764033528, 6.286512480251461, 4.191008382016764, 2.095504191008382, 1.047752095504191, 0.5238760477520955, 0.26193802387604775, 0.13096901193802388], transformation=Transformation(1, 0, -1, 0))" | |
| } | |
| } | |
| } |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
In tilestache.cfg, need one query per zoom level. Possible and in fact simplest to have all rows in one table due to this. Accepts null queries for unused zoom levels (0-3). Last query is used for all zoom levels following.