Task description

In Facta we represent factuality by means of a combination of three attributes associated with event mentions:

• • certainty, which relates to how sure the main source is about the mentioned event. It admits the following four values: certain, non_certain, and underspecified.

  • time, which specifies the time when an event is reported to have taken place or to be going to take place. Its values are past/present, future, and underspecified.

  • polarity, which captures if an event is affirmed or negated and, consequently, it can be either positive or negative; when there is not enough information available to detect the polarity of an event mention, it is underspecified.

Test data contain annotated gold event mentions; for each given gold event mention, participant systems are required to assign the correct value (selected from a closed list) for the three factuality attributes.

The evaluation of systems' output is based on the combination of the values assigned to the factuality attributes. Systems' performance will be evaluted in terms of precision, recall and F1 score. The official ranking will be based on the F1 score.