Skip to content

Instantly share code, notes, and snippets.

@recoilme

recoilme/Gemma-2-Ataraxy-Gemmasutra-9B-slerp-Nous.md

Created September 20, 2024 17:26
Show Gist options
  • Save recoilme/0cae46d3fc090c0b259c91f8d55318db to your computer and use it in GitHub Desktop.
Save recoilme/0cae46d3fc090c0b259c91f8d55318db to your computer and use it in GitHub Desktop.
Download ZIP
Raw
Gemma-2-Ataraxy-Gemmasutra-9B-slerp-Nous.md
Model AGIEval GPT4All TruthfulQA Bigbench
Gemma-2-Ataraxy-Gemmasutra-9B-slerp 40.91 Error: File does not exist 60.1 Error: File does not exist

AGIEval

Task Version Metric Value Stderr
agieval_aqua_rat 0 acc 21.26 ± 2.57
acc_norm 23.23 ± 2.65
agieval_logiqa_en 0 acc 38.56 ± 1.91
acc_norm 37.79 ± 1.90
agieval_lsat_ar 0 acc 24.35 ± 2.84
acc_norm 21.30 ± 2.71
agieval_lsat_lr 0 acc 49.22 ± 2.22
acc_norm 43.53 ± 2.20
agieval_lsat_rc 0 acc 66.91 ± 2.87
acc_norm 62.45 ± 2.96
agieval_sat_en 0 acc 78.64 ± 2.86
acc_norm 79.13 ± 2.84
agieval_sat_en_without_passage 0 acc 28.16 ± 3.14
acc_norm 26.70 ± 3.09
agieval_sat_math 0 acc 36.36 ± 3.25
acc_norm 33.18 ± 3.18

Average: 40.91%

GPT4All

Average: Error: File does not exist%

TruthfulQA

Task Version Metric Value Stderr
truthfulqa_mc 1 mc1 40.15 ± 1.72
mc2 60.10 ± 1.50

Average: 60.1%

Bigbench

Average: Error: File does not exist%

Average score: Not available due to errors

Elapsed time: 03:27:42

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment