Glossary
Glossary of terms
The vocabulary of pairwise-comparison ranking, defined in plain language.
- Pairwise comparison
- A single judgement between two items: “which is better, A or B?”. compere aggregates many of these into a ranking rather than asking for absolute scores.
- Multi-armed bandit
- A framework for choosing among options under uncertainty, balancing exploiting known-good choices against exploring uncertain ones. compere treats “which pair to compare next” as a bandit problem.
- UCB1
- Upper Confidence Bound — the bandit rule compere uses for pair selection. It scores each entity by observed win rate plus an exploration bonus that shrinks as the entity is compared more.
- Exploration constant (c)
- The weight on the UCB exploration bonus. Higher c means more exploration of under-compared entities. Default 1.414 (UCB_EXPLORATION_CONSTANT).
- Unexplored weight
- A large initial priority (default 1000.0, UCB_UNEXPLORED_WEIGHT) given to entities with no comparisons yet, so new items are surveyed before they are scored.
- Elo rating
- A rating system from chess. Each entity has a numeric rating; after a comparison, the winner gains and the loser loses points, scaled by how surprising the result was.
- K-factor
- The step size of an Elo update. A larger K makes ratings move faster (and noisier). Default 32 (ELO_K_FACTOR).
- Initial rating
- The Elo rating a new entity starts with before any comparisons. Default 1500 (ELO_INITIAL_RATING).
- Expected score
- The Elo-predicted probability that one entity beats another, computed as 1 / (1 + 10^((opponent − rating) / 400)). The rating update is proportional to actual minus expected.
- Similarity pairing
- An alternative selection strategy (compere/modules/similarity.py, exposed at /comparisons/next) that pairs similar entities instead of UCB-optimal ones.
- Leaderboard
- The list of entities sorted by Elo rating, returned by GET /ratings. It is a deterministic function of the comparisons recorded so far.
- Stopping rule
- A criterion for deciding you have collected enough votes — for example, when the top of the leaderboard stops changing across new comparisons.
Want the maths behind these? See how compere works.