In a pairwise vote, voters are asked to repeatedly pick between pairs of options, selecting the one they favor of the two. The procedure then combines all these pairwise choices in order to establish a global ranking for all options. A pairwise vote is statistical in nature, it must infer preference data that voters have not explicitly stated in order to obtain a result.
This statistical property allows obtaining an approximate preference ordering over a large list of options without overwhelming the voter with too much work. For example, if each voter was asked to establish a preference over 50 items, they would be exhausted and participation would suffer.
The pairwise-beta is a simple bayesian method used to rank items in pairwise votes. It is based on the beta-binomial model [1], which is composed of a beta prior and a binomial likelihood. This model is very tractable: because the beta distribution is conjugate to the binomial, the posterior also has beta form and is easily obtained:
I will not present a formal justification of the pairwise-beta model for pairwise comparisons, rather I will present some intuitions that should convey how and why the model works.
The key bridge to interpret pairwise comparisons in terms of the beta-binomial model is to realise that the better-worse relation between options maps directly into the success/failure outcomes of a bernoulli trial. We can thus establish a correspondence[4] between pairwise comparisons and bernoulli trials:
- each item i corresponds to a sequence of bernoulli trials, Bi
- a comparison in which i wins corresponds to a success in Bi
- a comparison in which i loses corresponds to a failure in Bi
The question we are trying to answer is
Given the proportion of comparisons in which i wins, what is the proportion of items that are globally better than i?
that reformulated in terms of our correspondences becomes
Given a sequence of bernoulli trials Bi, what is the proportion of successes/losses for i?
Which is a case of standard binomial proportion estimation[2]. As we noted before, the posterior of the beta binomial is also a beta distribution, given by
If we want a point estimate we just use the mean for this distribution which is
This gives us, for each item i, an estimation for the number of items that are better/worse than itself. This leads directly to a global ranking: the best ranked items will be those which are estimated to be better than most other items.
In summary, the procedure is
- For each item i, obtain the corresponding sequence of bernoulli trials Bi
- For each item i, calculate the posterior beta distribution mean given the data from 1)
- Create a global ranking based on the proportions for each item, as calculated in 2)
The pairwise-beta model is simple but not perfect. In particular it does not exploit information about the strength of the opposing item in a pairwise comparison. However, despite this drawback it performs well in practice. Please refer to [3] for details.
[1] http://www.cs.cmu.edu/~10701/lecture/technote2_betabinomial.pdf
[2] http://homepage.ntu.edu.tw/~ntucbsc/%A5%CD%AA%AB%C2%E5%BE%C7%B2%CE%ADp%BF%D4%B8%DF%A4@971008/[3]%20Chapter%208.pdf
[3] http://arxiv.org/pdf/1202.0500v2.pdf
[4] The correspondence is two to one, as each comparison yields two bernoulli trials