frankyaorenjie/main.js

Last active July 10, 2018 06:20

Star () You must be signed in to star a gist
Fork () You must be signed in to fork a gist

Learn more about clone URLs
Clone this repository at <script src="https://gist.github.com/frankyaorenjie/e15d6ae0ea0745d5743f11898855c846.js"></script>
Save frankyaorenjie/e15d6ae0ea0745d5743f11898855c846 to your computer and use it in GitHub Desktop.

Download ZIP

nodejs

Raw

main.js

	// read big file
	split -l 500000 query_log_distinct_6m.csv split_files/

Raw

set_memory.sh

node --max-old-space-size=8192 do.js

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment