Self-Preference Bias in Rubric-Based Evaluation of Large Language Models

ID: 2604.06996

Authors: José Pombal, Ricardo Rei, André F. T. Martins

Focus: Investigating bias in models evaluating their own outputs.

Key Insight: Self-preference bias (SPB) persists even in objective rubric-based evaluation, skewing scores by up to 10 points. Ensembling helps but does not eliminate SPB.

RSI Relevance: Highlights a major risk in recursive self-improvement: models validating their own "improvements" may just be reinforcing their own biases.

View on ArXiv