main_exp.tex sha256 99c3cfda63aaefab857312e7158f67ff2487d5a8917951fd65a4d6d1adf10598 published.ts SOURCE_SHA256 99c3cfda63aaefab857312e7158f67ff2487d5a8917951fd65a4d6d1adf10598 rows main_exp 21 rows published 21 cells compared 294 mismatches 0 Average vs mean of four dims (printed at 2 decimals; a difference above 0.005+0.005 cannot be rounding) main_exp.tex: 13 Abab7-preview printed 54.92 mean4 54.9250 diff -0.0050 avgMismatch field=[] main_exp.tex: 14 Doubao-pro printed 59.24 mean4 59.2425 diff -0.0025 avgMismatch field=[] main_exp.tex: 15 Step-1-Flash printed 56.57 mean4 56.5700 diff +0.0000 avgMismatch field=[] main_exp.tex: 16 Step-2 printed 58.94 mean4 58.9450 diff -0.0050 avgMismatch field=[] main_exp.tex: 17 GPT-3.5 printed 54.18 mean4 54.1800 diff +0.0000 avgMismatch field=[] main_exp.tex: 18 GPT-4o printed 59.95 mean4 59.9500 diff +0.0000 avgMismatch field=[] main_exp.tex: 19 GPT-4o Mini printed 57.93 mean4 57.9325 diff -0.0025 avgMismatch field=[] main_exp.tex: 20 Gemini Pro printed 59.24 mean4 59.2350 diff +0.0050 avgMismatch field=[] main_exp.tex: 21 Claude-3-Haiku printed 54.71 mean4 54.7150 diff -0.0050 avgMismatch field=[] main_exp.tex: 22 Claude-3.5-Sonnet printed 57.22 mean4 57.2175 diff +0.0025 avgMismatch field=[] main_exp.tex: 26 Mistral-7B printed 51.64 mean4 51.6450 diff -0.0050 avgMismatch field=[] main_exp.tex: 27 Qwen-2-7B printed 45.53 mean4 45.5325 diff -0.0025 avgMismatch field=[] main_exp.tex: 28 LLaMA-3.1-8B printed 52.99 mean4 52.9925 diff -0.0025 avgMismatch field=[] main_exp.tex: 29 CoSER-8B printed 56.45 mean4 56.4450 diff +0.0050 avgMismatch field=[] main_exp.tex: 30 Vicuna-13B-1.5 printed 47.58 mean4 47.5850 diff -0.0050 avgMismatch field=[] main_exp.tex: 31 Mixtral-8x7B printed 48.58 mean4 48.5750 diff +0.0050 avgMismatch field=[] main_exp.tex: 32 Qwen-2-72B printed 57.06 mean4 57.0625 diff -0.0025 avgMismatch field=[] main_exp.tex: 33 LLaMA-3.1-70B printed 55.49 mean4 55.4925 diff -0.0025 avgMismatch field=[] main_exp.tex: 34 Higgs-Llama-3-70B printed 54.74 mean4 54.7375 diff +0.0025 avgMismatch field=[] main_exp.tex: 35 CoSER-70B printed 59.06 mean4 59.0575 diff +0.0025 avgMismatch field=[] main_exp.tex: 36 DeepSeek-V3 printed 56.24 mean4 56.2375 diff +0.0025 avgMismatch field=[] main vs (ID+OOD)/2 per cell (2-decimal print, tolerance 0.015) cells 105 above tolerance: 10 0.710 GPT-4o sq mean(ID,OOD)= 79.62 main= 80.33 exp_idood.tex:22 0.505 GPT-4o sc mean(ID,OOD)= 61.085 main= 61.59 exp_idood.tex:22 0.450 GPT-4o cf mean(ID,OOD)= 48.50 main= 48.95 exp_idood.tex:22 0.450 GPT-4o avg mean(ID,OOD)= 59.50 main= 59.95 exp_idood.tex:22 0.415 GPT-3.5 sc mean(ID,OOD)= 57.635 main= 57.22 exp_idood.tex:21 0.195 GPT-3.5 cf mean(ID,OOD)= 42.485 main= 42.29 exp_idood.tex:21 0.135 GPT-4o an mean(ID,OOD)= 48.795 main= 48.93 exp_idood.tex:22 0.130 GPT-3.5 sq mean(ID,OOD)= 73.78 main= 73.91 exp_idood.tex:21 0.100 GPT-3.5 avg mean(ID,OOD)= 54.28 main= 54.18 exp_idood.tex:21 0.085 GPT-3.5 an mean(ID,OOD)= 43.215 main= 43.30 exp_idood.tex:21 0.005 Vicuna-13B-1.5 an mean(ID,OOD)= 39.115 main= 39.12 exp_idood.tex:34 0.005 Step-2 sc mean(ID,OOD)= 61.425 main= 61.43 exp_idood.tex:20 ID / OOD Average vs mean of their four exp_main_cot_full rows vs main_exp (only w/ I.T. rows) and Average vs mean4 exp_main_cot_full.tex:13 GPT-4o: all equal main = True exp_main_cot_full.tex:14 Qwen-2-72B: all equal main = True exp_main_cot_full.tex:16 CoSER-70B: all equal main = True exp_main_cot_full.tex:19 CoSER-8B: all equal main = True main_exp_cf3 (k=3) Average vs mean4 exp_diff_judges GPT-4o column vs main Average (same judge, same models) exp_diff_judges.tex:9 GPT-3.5: table 52.8 main 54.18 gap -1.38 exp_diff_judges.tex:10 LLaMA-3.1-8B: table 51.8 main 52.99 gap -1.19 exp_diff_judges.tex:11 Abab7-preview: table 53.7 main 54.92 gap -1.22 exp_diff_judges.tex:12 CoSER-8B: table 56.1 main 56.45 gap -0.35 exp_diff_judges.tex:13 GPT-4o: table 58.5 main 59.95 gap -1.45 exp_diff_judges.tex:14 Claude-3.5-Sonnet: table 56.2 main 57.22 gap -1.02 exp_diff_judges.tex:15 CoSER-70B: table 57.4 main 59.06 gap -1.66 Bold/underline: max per column sc: max all GPT-4o 61.59 (bold in tex: True), max open Mistral-7B 59.90 (underlined: True) an: max all CoSER-70B 53.33 (bold in tex: True), max open CoSER-70B 53.33 (underlined: True) cf: max all GPT-4o 48.95 (bold in tex: True), max open CoSER-70B 48.75 (underlined: True) sq: max all GPT-4o 80.33 (bold in tex: True), max open DeepSeek-V3 76.66 (underlined: True) avg: max all GPT-4o 59.95 (bold in tex: True), max open CoSER-70B 59.06 (underlined: True) bleu: max all CoSER-70B 10.10 (bold in tex: True), max open CoSER-70B 10.10 (underlined: True) rouge: max all CoSER-70B 14.78 (bold in tex: True), max open CoSER-70B 14.78 (underlined: True) Model names of main table not listed in Sections/5_Experiments.tex Models paragraph: see report. Spread: Qwen-2-72B BLEU ± 0.00