Examining the Limits of Word2Vec with Toki Pona

In three linesWord2Vec study on Toki Pona, constructed language with ~130 words. Training on 1.4M sentences (7.95M tokens). Comparison of two models: with and without non-Toki Pona tokens (named entities, loanwords). Finding: sparse tokens bring similar words closer; Word2Vec works even with extremely reduced vocabulary, relying on distributional patterns rather than lexicon size.

Summary generated by Claude — human-verified