old school word-embeddings

Topics

• word embeddings
• linear algebra

Before word2vec (neural-like methods), word embeddings were based on co-occurrence counts. The matrix was constructred, followed by some matrix factorization approach to get lower-dimensional word embeddings.
Advantages

• Very fast
• Captured global statistics that word2vec misses

Note

Despite the advantages (especially in speed), word2vec just works better normally.

Related

• GloVe