PHAEDRA Corpus Entity Mention Annotation Automatic Entity Normalisation Event Annotation Event Attribute Annotation Relation Annotation Coreference Annotation Annotation Format

Relation Annotation

Annotation scheme

We define two types of binary relations that encode static properties between NEs, as a means further enrich the dynamic information that is encoded through event structures. These two relation types are defined in Table 1.

Table 1. Types of Relations annotated

Relation type	Description
Subject_Disorder	Connects Subject phrases and Disorders, when the mentioned disorders correspond to complaints suffered by the subject(s) at the time when pharmacological substances are administered
is_equivalent	Allows links to be established between NEs that constitute alternative names for the same concept within the same sentence. Equivalences may correspond to full drug names/disorders and their abbreviations, to generic drug names and their corresponding brand names or synonyms, etc

Relation annotation statistics

All instances of the relations defined in Table 1 were annotated in all abstracts in the corpus. The total number of annotated relations of each type are shown in Table 2

Agreement

The relation annotations were undertaken by annotators with domain expertise. The quality and consistency of the annotations were verified through the calculation of inter-annotator agreement (IAA) on one quarter of the complete corpus (i.e., 150 abstracts). We calculated IAA in terms of F-Score, as shown in Table 3.