cosine-similarity vs dot-product

Topics

• linear algebra
• distance metrics

The dot product is the simplest way to compare two vectors.

$$\vec{a}\cdot \vec{b}=\sum _{i=1}^n{a}_i{b}_i=a_1{b}_1+a_2{b}_2+...+a_n{b}_n$$

Larger dot product generally indicates more similarity. However, it’s sensitive to the magnitude (length) of the vectors. Cosine similarity, on the other hand, measures the cosine of the angle between two vectors, effectively normalizing for vector length.

$$\text{cosine}(\vec{a},\vec{b})=\frac{\vec{a}\cdot \vec{b}}{|\vec{a}||\vec{b}|}=\frac{{\sum }_{i=1}^n{a}_i{b}_i}{\sqrt{{\sum }_{i=1}^n{a}_i^2}\sqrt{{\sum }_{i=1}^n{b}_i^2}}$$

Dot product values are unbounded, whereas, for cosine similarity, the value ranges from -1 (completely opposite) to 1 (identical direction). It’s not affected by vector magnitude, only direction matters.

In practice, for many NLP tasks, cosine similarity is often preferred because it allows for comparing the semantic similarity of words or documents regardless of their frequency or embedding magnitude. However, in some cases where the magnitude carries important information, dot product might be more appropriate.

Related