Skip to content

Instantly share code, notes, and snippets.

@dirkgr

dirkgr/mmap_hash.py

Created December 14, 2021 19:28
Show Gist options
  • Save dirkgr/d25c7ac198f6f92a7a34fbce7a89351e to your computer and use it in GitHub Desktop.
Save dirkgr/d25c7ac198f6f92a7a34fbce7a89351e to your computer and use it in GitHub Desktop.
Download ZIP
Hashing a memory-mapped file with Python
Raw
mmap_hash.py
import mmap
import xxhash
def _checksum_artifact(path: PathOrStr) -> str:
filepath = Path(path)
if not filepath.is_file():
raise FileNotFoundError(str(filepath))
h = xxhash.xxh128()
with filepath.open("rb") as f:
with mmap.mmap(f.fileno(), 0, prot=mmap.PROT_READ) as m:
h.update(m)
return h.hexdigest()
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment