The Paradigm Discovery Problem

Alexander Erdmann, Micha Elsner, Shijie Wu, Ryan Cotterell, Nizar Habash


Abstract
This work treats the paradigm discovery problem (PDP), the task of learning an inflectional morphological system from unannotated sentences. We formalize the PDP and develop evaluation metrics for judging systems. Using currently available resources, we construct datasets for the task. We also devise a heuristic benchmark for the PDP and report empirical results on five diverse languages. Our benchmark system first makes use of word embeddings and string similarity to cluster forms by cell and by paradigm. Then, we bootstrap a neural transducer on top of the clustered data to predict words to realize the empty paradigm slots. An error analysis of our system suggests clustering by cell across different inflection classes is the most pressing challenge for future work.
Anthology ID:
2020.acl-main.695
Volume:
Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics
Month:
July
Year:
2020
Address:
Online
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
7778–7790
Language:
URL:
https://aclanthology.org/2020.acl-main.695
DOI:
10.18653/v1/2020.acl-main.695
Bibkey:
Cite (ACL):
Alexander Erdmann, Micha Elsner, Shijie Wu, Ryan Cotterell, and Nizar Habash. 2020. The Paradigm Discovery Problem. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7778–7790, Online. Association for Computational Linguistics.
Cite (Informal):
The Paradigm Discovery Problem (Erdmann et al., ACL 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.acl-main.695.pdf
Video:
 http://slideslive.com/38928868
Code
 alexerdmann/ParadigmDiscovery
Data
Universal Dependencies
Terminologies: