# CharacterEval: search for later work reporting CharacterEval scores (2026-10-28, UTC)

Record behind Section 9 and Table 8 of the CharacterEval reproduction report. Numbers were compared with the paper's Table 4 as
extracted from the arXiv v2 LaTeX source (worker/src/benchmarks/charactereval/published.ts). Items marked "verified" were re-read by
us in the arXiv HTML of the cited version on 2026-10-28; the others are as found by the search and were not re-checked.

## Queries

- paper index: "CharacterEval"; "BaichuanCharRM 评测 结果"; "CharacterRM"; "在 CharacterEval 上 均分 评测 结果"
- web: `"CharacterEval" benchmark results role-playing model CharacterRM`; `arXiv role-playing "CharacterEval" table BC-NPC-Turbo 3.169 MiniMax`;
  `"CharacterEval" "Index" SLM technical report CharacterYuyan`; `"BaichuanCharRM" arXiv`;
  `"CharacterEval" role-playing reinforcement learning results Qwen 2025 arXiv "Character Consistency" "Conversational Ability" "Role-playing Attractiveness"`;
  `"CharacterEval" evaluation role-play model "CharacterRM" scores table 2024 Neeko OR CharacterGLM OR "Yi" OR "RoleCraft" OR "Crab"`;
  `"evaluated on CharacterEval" OR "results on CharacterEval" role-playing LLM`; `VeriRole role-playing verifiable reward arXiv CharacterEval`;
  `"R4" role-playing reinforcement learning Wang 2026 arXiv CharacterEval`;
  `arXiv 2025 2026 role-playing model "CharacterEval" Chinese benchmark results Doubao Kimi "BC-NPC"`;
  `"CharacterEval" role-play LLM arXiv 2026 GRPO "CharacterRM" OR "BaichuanCharRM" results table Qwen3`

## Sources reporting CharacterEval numbers

1. Ji et al., Enhancing Persona Consistency for LLMs' Role-Playing using Persona-Aware Contrastive Learning (PCL), arXiv:2503.17662v2;
   ACL Findings 2025. CharacterRM; original split and a split by character; 12 metrics and 3 averages. Qwen-7B+ICL and GPT-4+ICL equal
   the paper in all cells. Baichuan2-7B+ICL Flu/Coh/Cons 3.051/3.394/3.327 against the paper's 3.551/3.894/3.827 (each −0.500; verified).
2. Tang et al., ERABAL, arXiv:2409.14710v2. Scorer and test-set size not stated; 12 metrics. Baichuan2-Chat-13B baseline below the paper
   in every metric (e.g. KE 1.561 vs 1.802). Its "+ERABAL (SFT)" row equals the "Baichuan2-AD" row of the same authors' RoleAD paper.
3. Tang et al., RoleAD, arXiv:2402.10618v1 (v2 withdrawn). Scorer not stated; 12 metrics and 3 averages.
4. Feng et al., Reasoning Does Not Necessarily Improve Role-Playing Ability, arXiv:2502.16940v2. BaichuanCharRM through OpenCompass;
   4,564 samples; one score per model (aggregation not defined); 12 metrics for the Qwen2.5 family in an appendix.
5. Zhang et al., OmniCharacter, arXiv:2505.20277v2. Scorer and test set not stated; 12 metrics, 3 averages, an overall. Reprints 12
   rows of the paper. Baichuan2-13B CA average printed 3.759 (paper 3.795; its own Flu/Coh/Cons 3.596/3.924/3.864 give 3.795);
   InternLM-7B overall 2.983 where its printed averages 2.620/3.698/2.784 give 3.034 (verified).
6. Li et al., Index SLM Technical Report, arXiv:2607.09885v4. Not stated; "baseline scores follow Tu et al. (2024)"; 3 averages and an
   overall. Reprints the paper's 15 rows; adds CharacterYuyan-RLHF (3.360) and CharacterYuyan (3.309), which are not in the paper
   (verified). The text says the model "ranks ninth"; in its table it is eighth.
7. Fang et al., AAM (v1: ChARM), arXiv:2505.23923v2. Scorer not stated; 4,564 samples; 3 averages and an average. MiniMax-abab5.5s
   CA/CC/RA 3.72/2.71/2.91 against the paper's 3.784/2.718/2.902.
8. Ye et al., CPO, arXiv:2508.09074v1. CharacterRM; 4,564 test examples; 12 metrics, 3 averages, an overall at 2 decimals. MiniMax row
   equal to the paper at two decimals except Emp. (3.01 vs 3.017). GPT-4o CA/CC/RA/overall 3.63/2.90/3.06/3.20, Claude-3.7-sonnet
   3.90/3.06/3.25/3.41 (verified).
9. Zhang et al., AdvRole, arXiv:2609.28609v1. BaichuanCharRM; 2,000 of the 4,564 test samples used for training, the remaining 2,564 for
   evaluation (verified). Its GPT-4o and Claude-3.7-Sonnet rows equal CPO's full-set numbers above (verified), although the caption
   says baselines were re-implemented under identical splits.
10. Xu et al., RAPO, arXiv:2606.27025v1. "Specifically trained reward models"; test set not stated; 12 metrics and an "Avg" that is about
    the mean of the 12 metrics (e.g. Qwen2.5-7B printed 3.09; mean of 12 metrics 3.108; mean of 3 dimensions 3.177).
11. Peng and Chen, Rethinking Role-Playing Evaluation, arXiv:2603.03915v2. The original reward model on 8,032 items; per metric and per
    dimension; MBTI used as prompt input, not as back-testing.

Not opened (OpenReview browser check): VeriRole (ICLR 2026; abstract claims +4.55% on CharacterEval for Qwen2.5-32B) and R4 (ICLR 2026).

## Sources citing or reusing CharacterEval without per-model scores

CharacterBench (2412.11912; reports only a rank correlation of CharacterEval with human rankings), TRACE Bench (2608.11236), MDRP
(2603.19313), DualMem (2605.25693; a GPT-5.1 judge), RoleRM (2512.10575; evaluates CharacterRM as a reward model), PersonaEval
(2508.10014), HER (2601.21459), OpenCharacter (2501.15427), Neeko (2402.13717v3), Beyond Dialogue (2408.10903v6), RoleMRC (2502.11387),
CoSER (2502.09082v3), RAIDEN-R1 (2505.10218), Character-R1 (2601.04611), and a role-playing agent survey (2601.10122).