Skip to content

Instantly share code, notes, and snippets.

@srijan-paul
Last active July 18, 2021 10:34
Show Gist options
  • Save srijan-paul/06d640db0b08086757687dbebffb7f1f to your computer and use it in GitHub Desktop.
Save srijan-paul/06d640db0b08086757687dbebffb7f1f to your computer and use it in GitHub Desktop.
Pallene benchmark results.

The below table shows the performance gain on upvalue box merging w.r.t the upvalue count.

Upvalue count Performance increase on upvalue box merging
2 1.06 ± 0.54
4 1.06 ± 0.51
8 1.18 ± 0.58
16 1.10 ± 0.45
32 1.27 ± 0.43
64 1.56 ± 0.28
128 1.58 ± 0.32
200 1.76 ± 0.24

NOTE This might be a bit inaccurate because I ran these on a weak laptop and hyperfine kept saying this after the 64 mark:

Warning: Statistical outliers were detected. 
Consider re-running this benchmark on a quiet PC without any interferences from other programs. 
It might help to use the '--warmup' or '--prepare' options.

Assuming the way I ran the benchmarks was fair and didn't have any big mistakes, and that the above table is mostly accurate, my thoughts would be:

  1. When the number of upvalues gets large, the speed difference starts to matter.
  2. However, is it really realistic to expect 50+ upvalues in a single closure?
  3. How often does this happen?
  4. How often are such closures called? How likely is it for such a closure to be called in performance sensitive regions like a game loop?
  5. This was an artifically crafted example which makes for an extremely ideal scenario to merge boxes. In a real program however, the boxes may not be mergeable because of escaping etc.

I think judging some stats on existing luarocks packages may help determine the tradeoffs of this optimization. Where things stand now, from my personal POV it might make sense to move forward with the optimization, but it's still unclear to what extent it would benefit us without some real data.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment