Skip to content

Instantly share code, notes, and snippets.

@jasraj

jasraj/strdd.q

Created February 26, 2021 18:02
Show Gist options
  • Save jasraj/ece63030522db03f4fd7634f80b048aa to your computer and use it in GitHub Desktop.
Save jasraj/ece63030522db03f4fd7634f80b048aa to your computer and use it in GitHub Desktop.
Download ZIP
kdb String Deduplication
Raw
strdd.q
/ q).Q.w[]`used
/ 385648
/ 1. Generate some strings with repeats
/ q).Q.w[]`used
/ 10482848
strs:.Q.an @/:(mod[;count .Q.an] 4 cut til 1000000);
/ 2. Deduplicate
/ q).Q.w[]`used
/ 2484864
strs:distinct[strs] raze where each distinct[strs] ~\:/: strs
/ Check reference count of each string
/ q)-16!/:strs
/ 3968 3968 3968 3968 3968 3968 3968 3968 3968 3968 3968 3968 3968 3968 3968 39..
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment