Preliminary announcement of the 1st Anaphora Resolution Exercise (ARE)

[Constantin Orasan]

                   [Apologies for multiple postings]

        -------------------------------------------------------
                    Preliminary announcement of the 
                 1st Anaphora Resolution Exercise (ARE)

                    http://clg.wlv.ac.uk/events/ARE/
        -------------------------------------------------------


We are pleased to announce the first Anaphora Resolution Exercise (ARE)
organised in conjunction with the 6th Discourse Anaphora and Anaphor
Resolution Colloquium (DAARC2007). The general objective of ACE is to
develop discourse anaphora resolution methods and to evaluate
them in a common and consistent manner. 

The exercise proposes four tasks:

1. Pronominal resolution on pre-annotated texts: in this task,
participants will be provided with documents in which the NPs are
annotated. Among these NPs, some of them will have an attribute which
indicates that they are referential and have to be resolved. In this
task only referential pronouns will have this attribute (i.e. pleonastic
pronouns will not be annotated). The participants will have to identify
for each pronominal referential expression any correct non-pronominal
antecedent from the list of annotated NPs.

2. Coreferential chains resolution on pre-annotated texts: in this task
participants will be provided with documents in which the NPs are
annotated. The participants will have to identify full coreferential
chains which contain the annotated NPs.

3. Pronominal resolution on raw texts: in this task, participants will
have to resolve referential pronouns in unannotated texts (i.e. the
referential expressions and antecedent candidates are not marked as in
task 1)

4. Coreferential chains resolution on raw texts: in this task,
participants will have to identify full coreferential chains in
unannotated texts (i.e. the NPs are not pre-marked)

In this first edition, the evaluation is organised only for English and
tackles pronominal anaphora and NP coreference. If successful, in future
editions it can be extended to include other types of referential
expressions and anaphoric relations (e.g. bridging references) and other
languages.

The participants will be given a set of guidelines which will explain
which referential expressions are considered, when two elements are
coreferential and the evaluation procedures. In addition, we foresee the
development of a web interface where the participants will be able to
evaluate their results on the training data using the evaluation metrics
to be used in the formal evaluation. Given that in normal circumstances
the tasks of anaphora and coreference resolution are not very
computational intensive the participants will have only 24h between the
moment they download the test data and the time they have to submit the
results. 

Tentative timeline:

2nd Nov 2006: the first announcement of ARE
2nd Nov 2006: the registration opens
15th Nov 2006: the training data available
15th Nov 2006: release of the guidelines
12th March 2007: the test data available
16th March 2007: the competition finishes
28th - 29th March 2007: DAARC2007

The results of this evaluation will be disseminated at DAARC where we
plan to organise a round table and/or a poster session on this theme. If
enough interest is shown by the participants, we also consider
publishing a collection of paper afterwords. 

Participation is open to all. To indicate your intention to participate
or for any other issues please contact Constantin Orasan
(C.Orasan at wlv.ac.uk)

Web page: http://clg.wlv.ac.uk/events/ARE/

Organising committee:
Antonio Branco, University of Lisbon, Portugal 
Dan Cristea, "Alexandru Ioan Cuza" University, Iasi, Romania
Ruslan Miktov, University of Wolverhampton, UK
Constantin Orasan, University of Wolverhampton, UK

-- 
Constantin Orasan <C.Orasan at wlv.ac.uk>
Lecturer in Computational Linguistics
Research Group in Computational Linguistics
http://www.wlv.ac.uk/~in6093/
University of Wolverhampton