Kawin Ethayarajh

I'm a PhD student, NSERC PGS-D Scholar, and Facebook Fellow at Stanford NLP, where I'm working with Dan Jurafsky. I was previously a BMO National Scholar and John H. Moss Scholar at the University of Toronto, where I worked with Graeme Hirst and David Duvenaud.

My research interests are in representation learning (e.g., word embeddings, additive compositionality) and evaluation (i.e., how good is your NLP model really?).

Research Highlights

Human-Centric Evaluation

How Human is Human Evaluation? Improving the Gold Standard for NLG with Utility Theory.
Kawin Ethayarajh and Dan Jurafsky.
preprint.
paper
Dynaboard: An Evaluation-As-A-Service Platform for Holistic Next-Generation Benchmarking.
Zhiyi Ma^*, Kawin Ethayarajh^*, Tristan Thrush^*, Somya Jain, Ledell Wu, Robin Jia, Christopher Potts, Adina Williams, Douwe Kiela.
NeurIPS 2021.
paper demo news bib
Utility is in the Eye of the User: A Critique of NLP Leaderboards.
Kawin Ethayarajh and Dan Jurafsky.
EMNLP 2020.
paper slides bib

Data-Centric Evaluation

Understanding Dataset Difficulty with V-Usable Information.
Kawin Ethayarajh, Yejin Choi, and Swabha Swayamdipta.
ICML 2022 (outstanding paper).
paper code
Understanding Undesirable Word Embedding Associations.
Kawin Ethayarajh, David Duvenaud, and Graeme Hirst.
ACL 2019 (oral).
paper blog slides bib
Towards Understanding Linear Word Analogies.
Kawin Ethayarajh, David Duvenaud, and Graeme Hirst.
ACL 2019.
paper addendum blog bib

Representation-Centric Evaluation

Attention Flows are Shapley Value Explanations.
Kawin Ethayarajh and Dan Jurafsky.
ACL 2021 (oral).
paper slides bib
How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddings.
Kawin Ethayarajh.
EMNLP 2019 (oral).
paper blog code slides bib
Unsupervised Random Walk Sentence Embeddings: A Strong but Simple Baseline.
Kawin Ethayarajh.
ACL 2018 - Repl4NLP (best paper).
paper code poster slides screencast bib