Talk:SemEval/Proposed Revision
| Academics | |
|---|---|
| Disciplines: | Natural Language Processing Computational Linguistics Semantics |
| Umbrella Organization: | ACL-SIGLEX |
| Workshop Overview | |
| Founded: (Origin) | 1998 (Senseval) |
| Latest: | Semseval 2 Summer 2010 (Ended) ACL @ Uppsala, Sweden |
| Upcoming: | Semseval 3 Summer 2012(tentative) ACL @ Jeju Island, Korea |
| History | |
| Senseval-1 | 1998 @ Sussex |
| Senseval-2 | 2001 @ Toulouse |
| Senseval-3 | 2004 @ Barcelona |
| SemEval-1 / Senseval-4 | 2007 @ Prague |
| SemEval-2 | 2010 @ Uppsala |
SemEval (Semantic Evaluation) is an ongoing series of evaluations, tasks and events used to evaluate computational semantic analysis systems. The evaluations are intended to explore the nature of meaning in language. While meaning is intuitive to humans, transferring those intuitions to computational analysis has proved elusive.
SemEval evolved from the Senseval Word sense evaluation series. The first three evaluations, Senseval-1 through Senseval-3, were focused on word sense disambiguation, each time growing in the number of languages offered in the tasks and in the number of participating teams. Beginning with the fourth workshop, SemEval-2007 (SemEval-1), the nature of the tasks evolved to include semantic analysis tasks outside of word sense disambiguation.
List of Senseval and Semeval Workshops[edit]
- Senseval-1 took place in the summer of 1998 for English, French, and Italian, culminating in a workshop held at Herstmonceux Castle, Sussex, England on September 2–4.
- Senseval-2 took place in the summer of 2001, and was followed by a workshop held in July 2001 in Toulouse, in conjunction with ACL 2001. Senseval-2 included tasks for Basque, Chinese, Czech, Danish, Dutch, English, Estonian, Italian, Japanese, Korean, Spanish, Swedish.
- Senseval-3 took place in March–April 2004, followed by a workshop held in July 2004 in Barcelona, in conjunction with ACL 2004. Senseval-3 included 14 different tasks for core word sense disambiguation, as well as identification of semantic roles, multilingual annotations, logic forms, subcategorization acquisition.
- Semeval-1/Senseval-4 took place in 2007, followed by a workshop held in conjunction with ACL in Prague. Semeval-1 included 18 different tasks targeting the evaluation of systems for the semantic analysis of text. A special issue of Language Resources and Evaluation is devoted to the result[1].
- Semeval-2 took place in 2010, followed by a workshop held in conjunction with ACL in Uppsala. Semeval-2 included 18 different tasks targeting the evaluation of semantic analysis systems.
Areas of evaluation[edit]
This list is expected to grow as the field progresses[2]. The following tables shows the areas of studies that are involved from Senseval-1 to SemEval-2:
| Areas of Study | Senseval-1 | Senseval-2 | Senseval-3 | SemEval-1 | SemEval-2 |
|---|---|---|---|---|---|
| Coreference | ✓ | ||||
| Cross-Lingual | ✓ | ✓ | |||
| Ellipsis | ✓ | ||||
| Keyphrase Extraction | ✓ | ||||
| Metonymy | ✓ | ✓ | |||
| Noun Compounds | ✓ | ||||
| Semantic Relations . | ✓ | ✓ | |||
| Semantic Role Labeling | ✓ | ✓ | ✓ | ||
| Sentimental Analysis | ✓ | ✓ | |||
| Time Expression | ✓ | ✓ | |||
| Textual Entailment | ✓ | ||||
| Word Sense Disambiguation | ✓ | ✓ | ✓ | ✓ | ✓ |
See also[edit]
- Computational semantics
- Natural language processing
- Word sense disambiguation
- Semantic analysis (computational)
References[edit]
- ^ Language Resources and Evaluation Volume 43, Number 2
- ^ SemEval Portal (n.d.). In ACLwiki. Retrieved August 12, 2010 from http://aclweb.org/aclwiki/index.php?title=SemEval_Portal
External links[edit]
- Special Interest Group on the Lexicon (SIGLEX) of the Association for Computational Linguistics (ACL)
- Semeval - Semantic Evaluation Workshop (endorsed by SIGLEX)
- Senseval - international organization devoted to the evaluation of Word Sense Disambiguation Systems (endorsed by SIGLEX)
- SemEval Portal on the Wiki of the Association for Computational Linguistics