Skip to content

Instantly share code, notes, and snippets.

@ityonemo

ityonemo/wishlist.md

Created September 8, 2024 18:33
Show Gist options
  • Save ityonemo/266d8fc06bf75ed5953eb3844b26554e to your computer and use it in GitHub Desktop.
Save ityonemo/266d8fc06bf75ed5953eb3844b26554e to your computer and use it in GitHub Desktop.
Download ZIP
Ziglang Wishlist
Raw
wishlist.md

We're currently using ziglang in-prod at https://positron.ai.

Uses the zigler package to interface with elixir for robust and resilient layer on top of unreliable and pre-alpha grade hardware (talk coming soon!).

We heavily use AVX-512 instructions for token picking. http://antirez.com/news/142 and we use a highly optimized sorter in place of what you would see in that code (np.argsort).

The plan is to move away from shipping fp32 values at the interface to shipping bf16 values, so bf16 support at the type-level in zig would be amazing. It seems like the ML industry has settled on bf16 as the datatype of record for most fp operations.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment