BLEU formula explained

Topics

• BLEU score

Here’s the formula for BLEU:

$$\text{BLEU}=\text{BP}\cdot \exp \left(\sum _{n=1}^N{w}_n\log (p_n)\right)$$

where:

• $\text{BP}$ is the brevity penalty
• $p_n$ is the precision of n-grams in BLEU with value in $[0,1]$
• $w_n$ is the weight applied to the n-gram precision score.
  • The weights determine the contribution of each n-gram size to the overall score. A common choice for these weights is $w_n=\frac{1}{n}$ which rewards higher n-grams since they are more challenging to match
  • Expanding the exp, it turns out we are doing ${p_n}^{w_n}$ (math explanation). If we fix $p_n$, this value will be larger for higher $n$
• $N$ is the maximum n-gram size considered (typically 4)

Related