CharacterEval reproduction audit bundle

Everything behind Sections 2, 3, 5 and 6 of the CharacterEval reproduction report: the check scripts and the platform source as run, their results and logs, and the three independent reviews of the paper, the code and the released files. Nothing here contains test-set text, a character profile or a model's reply: the data stay with their authors (github.com/morecry/CharacterEval); lines of the reviews that quoted them are replaced by a marker (logs/06-redaction.log).

Check without running anything

shasum -a 256 -c SHA256SUMS      # in a copy of this directory

Re-run the checks

git clone https://github.com/morecry/CharacterEval repo && git -C repo checkout c3d44a6
curl -L -o eprint.tar.gz https://arxiv.org/e-print/2401.01275v2 && mkdir eprint && tar xzf eprint.tar.gz -C eprint
python3 -I code/tools/experiments/charactereval_spec_check.py repo          # needs node and the platform source tree
python3 -I code/tools/experiments/charactereval_table_checks.py eprint
python3 -I code/tools/experiments/charactereval_release_checks.py repo eprint
python3 -I code/tools/experiments/charactereval_rm_probes.py repo probes.json
python3 -I code/tools/experiments/charactereval_anchor_diagnostics.py repo results/anchor
python3 -I code/tools/experiments/charactereval_rm_numerics_verify.py repo results/charactereval_rm_numerics_check.json results/anchor

The reward-model deployment check (charactereval_rm_server_check.py) runs on the server that serves the model; the operator's result is results/charactereval_rm_server_check.json (all criteria met on 2026-10-28). The anchor (charactereval_anchor.py) scored the five released outputs through the platform's endpoint; its result and scores are in results/anchor/. Its verdict under the criteria fixed in advance is within one step. The numerical-sensitivity check that examines the cause (charactereval_rm_numerics_check.py, V07a), run by the operator on 2026-10-29, finds that numerics alone move single scores by a root-mean-square of 0.095 (strict bfloat16 accumulation) and 0.262 (float32), above the 0.06 fixed as sufficient to explain the anchor; its bfloat16 scores equal the anchor's for all 1,200 inputs (result, our check).

Files