Skip to content

Instantly share code, notes, and snippets.

@ryan-williams
Last active August 8, 2024 15:17
Show Gist options
  • Save ryan-williams/2e616363c68072bf67cad675fcb18f3b to your computer and use it in GitHub Desktop.
Save ryan-williams/2e616363c68072bf67cad675fcb18f3b to your computer and use it in GitHub Desktop.
Standalone script that mimics GitHub Actions' hashFiles helper

hash-files.py

Standalone script that mimics GitHub Actions' hashFiles helper (used by actions/setup-python, and frequently with actions/cache)

Upstream hashFiles source is here, this script was written while debugging setup-python#919.

Examples

git clone https://gist.github.com/ryan-williams/2e616363c68072bf67cad675fcb18f3b hashfiles-gist
export PATH=$PATH:$(pwd)/hashfiles-gist

hash-files.py test.txt
# 87183f21076ad2e29f35c62196150ed17c6bdd890bf2617188143ef9d63bec96

hash-files.py test*.txt
# 1a4d74031d9ffdb900f56611fd112163d8abdd4d913eb071e899ae48d8d92617

hash-files.py test1.txt test2.txt  # same as above
# 1a4d74031d9ffdb900f56611fd112163d8abdd4d913eb071e899ae48d8d92617

It just sha256sums all provided files, then sha256sums those hashes (as binary data), outputting the result as a hex string.

You can also directly check a specific version of a file:

hash-files.py <(curl -s https://raw.githubusercontent.com/single-cell-data/TileDB-SOMA/274c6f49766e19fa645d93cc99e213d0dcca372f/apis/python/setup.py)
# 0bc0ed4dc8ce9f9b48ebc42fa10052206992eb7a7705c7a75ec8fcbe15e3e65f

which matches the hash used by setup-python in this GHA:

Cache restored from key: setup-python-Linux-22.04-Ubuntu-python-3.11.9-pip-0bc0ed4dc8ce9f9b48ebc42fa10052206992eb7a7705c7a75ec8fcbe15e3e65f
#!/usr/bin/env python
import hashlib
import sys
sha256 = hashlib.sha256()
for path in sys.argv[1:]:
file_sha256 = hashlib.sha256()
with open(path, "rb") as f:
for byte_block in iter(lambda: f.read(4096), b""):
file_sha256.update(byte_block)
sha256.update(file_sha256.digest())
print(sha256.hexdigest())
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment