Cognitive word embedding evaluation

Word embedding evaluation is traditionally done with word analogy tasks. However, this does not necessarily reflect the word representations humans have in their brains. Thus, a cognitively more plausible approach to evaluate word embeddings is to use them to predict brain activity signals [1,2,3,4].

The goal of this thesis or semester project is to build a comparison system of how well state of the art word-level and sentence-level embeddings can predict brain activity data such as eye-tracking, fMRI and EEG signals [5].

Supervised by Nora Hollenstein


  1. [1] Søgaard, Anders. Evaluating word embeddings with fMRI and eye-tracking. Proceedings of the 1st Workshop on Evaluating Vector-Space Representations for NLP. 2016.
  2. [2] Auguste, J., Rey, A., & Favre, B. Evaluation of word embeddings against cognitive processes: primed reaction times in lexical decision and naming tasks. In Proceedings of the 2nd Workshop on Evaluating Vector Space Representations for NLP. 2017.
  3. [3] Bakarov, A. Can Eye Movement Data Be Used As Ground Truth For Word Embeddings Evaluation?. arXiv preprint arXiv:1804.08749. 2018.
  4. [4] Rodrigues, Joao António, et al. "Predicting Brain Activation with WordNet Embeddings." Proceedings of the Eight Workshop on Cognitive Aspects of Computational Language Learning and Processing. 2018.
  5. [5] Hollenstein N., Rotsztejn J., Troendle M., Pedroni A., Zhang C. and Langer N. ZuCo, a simultaneous EEG and eye-tracking resource for natural sentence reading. Scientific Data. 2018.