Untested but very interesting, see:
- https://hi.stamen.com/stamen-aws-lambda-tiler-blog-post-76fc1138a145
- https://blog.mapbox.com/combining-the-power-of-aws-lambda-and-rasterio-8ffd3648c348
Probably best reserved for a larger project due to setup time and AWS costs.
Untested, I didn't want to manage the server application. In hindsight this may have been the best option due to unexpected requirement for ArcGIS desktop usage (Arc requires that the images be served as WMS/WMTS, it does not support XYZ tiles. This workaround exists - https://gis.stackexchange.com/questions/174569/adding-custom-web-tile-layer-to-arcmap - but does not work currently with a free AGOL account)
Untested, see:
This method chosen because:
- no server setup other than file transfer to existing web server
- zero additional cost within existing infrastructure
- easy consumption with Leaflet
- ArcMap requirement was not identified early on
https://www.gdal.org/gdal2tiles.html
- gdal2tiles.py only dumps to TMS (origin is lower left, see https://gist.github.com/tmcw/4954720)
- QGIS 3 load XYZ tiles easily, and even handles TMS with a check of a box
- convert existing gdal2tiles filenames from TMS to XYZ with https://gist.github.com/kannes/ebfe021458f96e4f30b5
- arcgis desktop can consume xyz tiles if you serve them as WMS using mapproxy
- various gdal2tiles implementations/forks are available on github (and elsewhere) but the latest GDAL (2.3.1) includes a working version that includes multithread support.
Regardless of the method chosen, the data should generally be converted from source TIFFs to jpeg compressed COGs (to save space, speed serving)
What is a TIFF / COG?
- https://medium.com/planet-stories/a-handy-introduction-to-cloud-optimized-geotiffs-1f2c9e716ec3
- https://www.cogeo.org/
- https://trac.osgeo.org/gdal/wiki/CloudOptimizedGeoTIFF
Lazy process the rasters:
Compression:
- http://blog.cleverelephant.ca/2015/02/geotiff-compression-for-dummies.html
- https://kokoalberti.com/articles/geotiff-compression-optimization-guide/
Consider writing a python script with rasterio rather than using gdal translate:
Process a folder of UTMZ10 tiffs:
# build vrt
gdalbuildvrt \
test.vrt \
path/to/tiffs/*.tif
# convert to single 8 bit COG
gdal_translate \
test.vrt \
test.tif \
-ot Byte \
-scale 0 65535 0 255 \
-co TILED=YES \
-co COPY_SRC_OVERVIEWS=NO \
-co COMPRESS=JPEG \
-co PHOTOMETRIC=YCBCR
# create overviews for COG
gdaladdo \
--config COMPRESS_OVERVIEW JPEG \
--config PHOTOMETRIC_OVERVIEW YCBCR \
--config INTERLEAVE_OVERVIEW PIXEL \
-r average \
test.tif \
2 4 8 16
# test that it is compliant
validate_cloud_optimized_geotiff.py test.tif
# create tiles
gdal2tiles.py \
-e \
-s EPSG:26910 \
-z 12-20 \
-w none \
-r lanczos \
-a 0 \
--processes 12 \
test.tif \
test_tiles
When complete, convert file names to XYZ with https://gist.github.com/kannes/ebfe021458f96e4f30b5. Then transfer to web server.
Below config will serve XYZ tiles as WMS. See manual for more . https://mapproxy.org/.
services:
wms:
versions: ['1.1.1']
image_formats: ['image/png']
md:
title: MYTITLE
layers:
- name: cgpr2017
title: TITLE
sources: [cgpr2017cache]
caches:
cgpr2017cache:
grids: [GLOBAL_WEBMERCATOR]
sources: [cgpr2017tilesource]
sources:
cgpr2017tilesource:
type: tile
url: https://URL/%(z)s/%(x)s/%(y)s.png
grid: 'GLOBAL_WEBMERCATOR'
coverage:
bbox: [xmin ymin xmax ymax]
srs: 'EPSG:4326'
globals:
image:
formats:
image/png:
transparent: true
opacity: 0.0
Deployment with gunicorn behind nginx seems to work fine...although I haven't tested the performance beyond successfully opening imagery in an arcmap session. See https://mapproxy.org/docs/1.11.0/deployment.html#nginx
Some server config is necessary to get the python script to run on startup https://mapproxy.org/docs/1.11.0/deployment.html#python-http-server, which I haven't yet bothered with. It is probably an easy config but systemd docs are extensive: https://www.digitalocean.com/community/tutorials/systemd-essentials-working-with-services-units-and-the-journal