relation of BLEU score with GMAP

Topics

• BLEU score

In BLEU formula, we have a weighting term $w_n$ whose value is kept $1/n$ to reward higher n-grams more if they match with ground truth. Let’s ignore the brevity penalty and just expand the exp term:

$$\begin{array}{rl}& \exp \left(\sum _{n=1}^N{w}_n\log p_n\right)\\ & =\prod _{n=1}^N{p}_n^{w_n}\\ & =(p_1{)}^{\frac{1}{1}}\cdot (p_2{)}^{\frac{1}{2}}\cdot (p_3{)}^{\frac{1}{3}}\cdot (p_4{)}^{\frac{1}{4}}\end{array}$$

Had we used a fixed $w_n=1/N$, this would be identical to geometric mean average precision (GMAP).

$$\begin{array}{rl}\text{GMAP}& ={\left(\prod _{i=1}^N{\text{AP}}_i\right)}^{\frac{1}{N}}\\ & =\exp \left(\frac{1}{N}\sum _{i=1}^N\log ({\text{AP}}_i)\right)\end{array}$$

where AP is short for average precision.

Related

• geometric mean average precision