Skip to content

Instantly share code, notes, and snippets.

View marklit's full-sized avatar

Mark Litwintschik marklit

View GitHub Profile
@marklit
marklit / japan.gis.parquet.md
Last active August 22, 2024 16:58
Pacific Spatial Solutions: Japanese GIS Data
@marklit
marklit / airport_amenities_osm.md
Last active August 20, 2024 08:12
Airport Amenities

Airport Shops on OSM

Tallinn Airport is said to be the 'cosiest' airport in the world. If you search for that on Google they come up first. I noticed none of the shops or other amenities around the gates are marked in OSM.

I wrote some code to see if this was an issue at airports of a similar size or larger nearby in other EU countries. I got a lot of false-positives but this code might help anyone else looking to enrich the Airport metadata in OSM.

TLL is surrounded by dense urban areas and a major shopping mall so I built a polygon mask around the airport gates and counted the shops within it.

Airports that came back with no shops within their gate area mask where:

@marklit
marklit / partition.md
Last active August 7, 2024 11:40
Partition Denmark by Address
$ aws s3 --no-sign-request sync \
    s3://overturemaps-us-west-2/release/2024-07-22.0/theme=addresses/type=address/ \
    ~/addresses
$ cd ~/addresses
$ ~/duckdb
INSTALL spatial;
@marklit
marklit / old town.md
Created July 28, 2024 16:20
Old Town
SELECT ADOB_ID,
       ADS_OID,
       ADOB_LIIK,
       ORIG_TUNNUS,
       ADS_KEHTIV,
       VIITEPUNKT_X,
       VIITEPUNKT_Y,
@marklit
marklit / issue.md
Created July 21, 2024 17:46
Overture July 2024 Canadian Addresses

DuckDB v1.0.0 1f98600c2c

$ cd '/mnt/j/gis/Global/overture/2024_07/theme=addresses/type=address'
$ COUNTRY=CA
$ echo "COPY (
              SELECT h3_cell_to_boundary_wkt(
                          h3_latlng_to_cell(bbox.ymax, bbox.xmax, 5))::geometry geom,
                      COUNT(*)
 FROM read_parquet('*.parquet')
COPY (
SELECT * EXCLUDE (geometry,
bbox,
names,
sources),
ST_GEOMFROMWKB(geometry) geom,
JSON(bbox) AS bbox,
JSON(names) AS names,
JSON(sources) AS sources
FROM read_parquet('s3://overturemaps-us-west-2/release/2024-06-13-beta.0/theme=buildings/type=building/*00208*',
@marklit
marklit / satellogic-samples.md
Last active July 5, 2024 09:07
Satellogic Samples

Signature version complaint via awscli

$ virtualenv ~/.aws
$ source ~/.aws/bin/activate
$ pip install awscli

$ aws --version
@marklit
marklit / highlights.py
Created May 28, 2024 05:49
More Itertools Highlights
# https://more-itertools.readthedocs.io/en/stable/index.html
list(chunked([1, 2, 3, 4, 5, 6, 7, 8], 3))
# [[1, 2, 3], [4, 5, 6], [7, 8]]
group_1, group_2 = distribute(2, [1, 2, 3, 4, 5, 6])
list(group_1) # [1, 3, 5]
list(group_2) # [2, 4, 6]
children = divide(3, [1, 2, 3, 4, 5, 6, 7])
@marklit
marklit / places.sql
Last active May 19, 2024 22:43
Pull H3s for Overture's Places Dataset for May 2024
COPY (
WITH a AS (
SELECT h3_cell_to_parent(h3_string_to_h3(SUBSTR(id, 0, 17)), 2) h3_2,
COUNT(*) num_recs
FROM read_parquet('s3://overturemaps-us-west-2/release/2024-05-16-beta.0/theme=places/type=place/*.parquet',
filename=true,
hive_partitioning=1)
GROUP BY 1
)
SELECT h3_cell_to_boundary_wkt(h3_2),
@marklit
marklit / hls.py
Last active June 3, 2023 15:40
Video Stream Generator
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
'''
Please see https://tech.marksblogg.com/streaming-video-hls.html for more details on this script.
MIT License
Copyright (c) 2023 Mark Litwintschik
Permission is hereby granted, free of charge, to any person obtaining a copy