A first evaluation of logic form identification systems


This paper presents a first experience with evaluating sytems that address the issue of Logic Form Identification (LFi). A Gold Standard approach was used in which experts provide solutions to test data. The expert solutions, the gold standard, are then compared against outputs from participanting systems and different metrics observed. We proposed a few novel metrics, including precision and recall, that are further used to provide comparative results. The test data included 4155 arguments and 2398 predicates grouped in 300 sentences.

Publication Title

Proceedings of the SENSEVAL@ACL 2004: 3rd International Workshop on the Evaluation of Systems for the Semantic Analysis of Text - Held in cooperation with ACL 2004

This document is currently not available here.