Multilingual and Cross-lingual Semantic Word Similarity Task

A Multilingual and Cross-lingual Semantic Word Similarity Task is a Semantic Word Similarity Modelling System that can be solved by Multilingual and Cross-lingual Semantic Word Similarity System by implementing a Multilingual and Cross-lingual Semantic Word Similarity Algorithm.

• Context:
  • Task Input: Natural Language Dataset for several Languages.
  • Task Output: Semantic Similarity Metric.
  • Task Requirement(s): a Semantic Word Similarity System.
• Example(s):
  • SemEval-2017 Task 2,
  • …
• Counter-Example(s):
  • a Multilingual Semantic Word Similarity Task,
  • a Cross-lingual Semantic Word Similarity Task.
• See: SemEval-2017 Task 2, Semantic Word Similarity Benchmark Task, Semantic Textual Similarity Benchmark Task, Semantic Similarity Measure, Semantic Relatedness Measure.

---

---

References

2017a

• (Camacho-Collados et al., 2017) ⇒ Jose Camacho-Collados, aMohammad Taher Pilehvar, Nigel Collier, and Roberto Navigli. (2017). “SemEval-2017 Task 2: Multilingual and Cross-lingual Semantic Word Similarity.” In: Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval@ACL 2017).
  • QUOTE: This paper introduces a new task on Multilingual and Cross-lingual Semantic Word Similarity which measures the semantic similarity of word pairs within and across five languages: English, Farsi, German, Italian and Spanish. High quality datasets were manually curated for the five languages with high inter-annotator agreements (consistently in the 0.9 ballpark). These were used for semi-automatic construction of ten cross-lingual datasets. 17 teams participated in the task, submitting 24 systems in subtask 1 and 14 systems in subtask 2. Results show that systems that combine statistical knowledge from text corpora, in the form of word embeddings, and external knowledge from lexical resources are best performers in both subtasks.

2017b

• (Speer & Lowry-Duda, 2017) ⇒ Robyn Speer, and Joanna Lowry-Duda. (2017). “ConceptNet at SemEval-2017 Task 2: Extending Word Embeddings with Multilingual Relational Knowledge.” In: Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval@ACL 2017).
  • QUOTE: The system we submitted to SemEval-2017 Task 2, “Multilingual and Cross-lingual Semantic Word Similarity”, is an update of that system, coinciding with the release of version 5.5.3 of ConceptNet. We added multiple fallback methods for assigning vectors to out-of-vocabulary words. We also experimented with, but did not submit, systems that used additional sources of word embeddings in the five languages of this SemEval task

---