BioNLP'09 Shared Task on
Event Extraction
in conjunction with BioNLP, a NAACL-HLT 2009 workshop, June 4-5 2009, Boulder, Colorado
[Shared Task Home] [Evaluation Methods] [Master Result Table] [Team Profiles] [System Profiles]

Result Master Table

    Localization     [Strict] [Approx. Span] [Approx. Span & Recur.] [Ev. Decom. - Approx. Recur.] [Ev. Decom. - Approx. Span & Recur.]
      Binding        [Strict] [Approx. Span] [Approx. Span & Recur.] [Ev. Decom. - Approx. Recur.] [Ev. Decom. - Approx. Span & Recur.]
  Gene_expression    [Strict] [Approx. Span] [Approx. Span & Recur.] [Ev. Decom. - Approx. Recur.] [Ev. Decom. - Approx. Span & Recur.]
   Transcription     [Strict] [Approx. Span] [Approx. Span & Recur.] [Ev. Decom. - Approx. Recur.] [Ev. Decom. - Approx. Span & Recur.]
 Protein_catabolism  [Strict] [Approx. Span] [Approx. Span & Recur.] [Ev. Decom. - Approx. Recur.] [Ev. Decom. - Approx. Span & Recur.]
  Phosphorylation    [Strict] [Approx. Span] [Approx. Span & Recur.] [Ev. Decom. - Approx. Recur.] [Ev. Decom. - Approx. Span & Recur.]
  ==[EVT-TOTAL]==    [Strict] [Approx. Span] [Approx. Span & Recur.] [Ev. Decom. - Approx. Recur.] [Ev. Decom. - Approx. Span & Recur.]
     Regulation      [Strict] [Approx. Span] [Approx. Span & Recur.] [Ev. Decom. - Approx. Recur.] [Ev. Decom. - Approx. Span & Recur.]
Positive_regulation  [Strict] [Approx. Span] [Approx. Span & Recur.] [Ev. Decom. - Approx. Recur.] [Ev. Decom. - Approx. Span & Recur.]
Negative_regulation  [Strict] [Approx. Span] [Approx. Span & Recur.] [Ev. Decom. - Approx. Recur.] [Ev. Decom. - Approx. Span & Recur.]
  ==[REG-TOTAL]==    [Strict] [Approx. Span] [Approx. Span & Recur.] [Ev. Decom. - Approx. Recur.] [Ev. Decom. - Approx. Span & Recur.]
      Negation       [Strict] [Approx. Span] [Approx. Span & Recur.] [Ev. Decom. - Approx. Recur.] [Ev. Decom. - Approx. Span & Recur.]
    Speculation      [Strict] [Approx. Span] [Approx. Span & Recur.] [Ev. Decom. - Approx. Recur.] [Ev. Decom. - Approx. Span & Recur.]
  ==[MOD-TOTAL]==    [Strict] [Approx. Span] [Approx. Span & Recur.] [Ev. Decom. - Approx. Recur.] [Ev. Decom. - Approx. Span & Recur.]
====[ALL-TOTAL]====  [Strict] [Approx. Span] [Approx. Span & Recur.] [Ev. Decom. - Approx. Recur.] [Ev. Decom. - Approx. Span & Recur.]

In order to avoid unnecessary confusion, we suppose the ALL-TOTAL results by the approximate span & recursive evaluation method the primary scores.



Final Evaluation Results
on ALL-TOTAL events by Approx. Span & Recur. matching

Task 1

------------------------------------------------------------------------------------------------
   TEAM                                 gold (match)   answer (match)   recall    prec.   fscore
------------------------------------------------------------------------------------------------
[ UTurku  ]       ==[ALL-TOTAL]==       3182 ( 1487)     2541 ( 1486)    46.73    58.48    51.95
[JULIELab ]       ==[ALL-TOTAL]==       3182 ( 1458)     3068 ( 1458)    45.82    47.52    46.66
[ConcordU ]       ==[ALL-TOTAL]==       3182 ( 1113)     1807 ( 1113)    34.98    61.59    44.62
[UT+DBCLS ]       ==[ALL-TOTAL]==       3182 ( 1174)     2110 ( 1173)    36.90    55.59    44.35
[VIBGhent ]       ==[ALL-TOTAL]==       3182 ( 1063)     2062 ( 1063)    33.41    51.55    40.54
[ UTokyo  ]       ==[ALL-TOTAL]==       3182 (  895)     1671 (  895)    28.13    53.56    36.88
[  UNSW   ]       ==[ALL-TOTAL]==       3182 (  898)     1957 (  896)    28.22    45.78    34.92
[ UZurich ]       ==[ALL-TOTAL]==       3182 (  883)     1895 (  883)    27.75    46.60    34.78
[ASU+HU+BU]       ==[ALL-TOTAL]==       3182 (  688)     1106 (  688)    21.62    62.21    32.09
[   Cam   ]       ==[ALL-TOTAL]==       3182 (  672)     1181 (  672)    21.12    56.90    30.80
[UAntwerp ]       ==[ALL-TOTAL]==       3182 (  716)     1501 (  716)    22.50    47.70    30.58
[ UNIMAN  ]       ==[ALL-TOTAL]==       3182 (  702)     1444 (  702)    22.06    48.61    30.35
[  SCAI   ]       ==[ALL-TOTAL]==       3182 (  826)     2278 (  826)    25.96    36.26    30.26
[ UAveiro ]       ==[ALL-TOTAL]==       3182 (  666)     1351 (  666)    20.93    49.30    29.38
[ Team 24 ]       ==[ALL-TOTAL]==       3182 (  722)     1778 (  721)    22.69    40.55    29.10
[ USzeged ]       ==[ALL-TOTAL]==       3182 (  685)     1852 (  685)    21.53    36.99    27.21
[  NICTA  ]       ==[ALL-TOTAL]==       3182 (  555)     1388 (  555)    17.44    39.99    24.29
[CNBMadrid]       ==[ALL-TOTAL]==       3182 (  911)     4362 (  911)    28.63    20.88    24.15
[CCP-BTMG ]       ==[ALL-TOTAL]==       3182 (  428)      596 (  428)    13.45    71.81    22.66
[CIPS-ASU ]       ==[ALL-TOTAL]==       3182 (  725)     3809 (  725)    22.78    19.03    20.74
[  UMich  ]       ==[ALL-TOTAL]==       3182 (  968)     6859 (  968)    30.42    14.11    19.28
[  PIKB   ]       ==[ALL-TOTAL]==       3182 (  358)      538 (  358)    11.25    66.54    19.25
[ Team 09 ]       ==[ALL-TOTAL]==       3182 (  372)     1184 (  372)    11.69    31.42    17.04
[ KoreaU  ]       ==[ALL-TOTAL]==       3182 (  299)      485 (  299)     9.40    61.65    16.31

Task 2

------------------------------------------------------------------------------------------------
   TEAM                                 gold (match)   answer (match)   recall    prec.   fscore
------------------------------------------------------------------------------------------------
[UT+DBCLS ]       ==[ALL-TOTAL]==       3193 ( 1145)     2110 ( 1141)    35.86    54.08    43.12
[UAntwerp ]       ==[ALL-TOTAL]==       3193 (  687)     1501 (  687)    21.52    45.77    29.27
[ASU+HU+BU]       ==[ALL-TOTAL]==       3193 (  629)     1106 (  629)    19.70    56.87    29.26
[ Team 24 ]       ==[ALL-TOTAL]==       3193 (  705)     1839 (  704)    22.08    38.28    28.01
[CCP-BTMG ]       ==[ALL-TOTAL]==       3193 (  423)      596 (  423)    13.25    70.97    22.33
[CNBMadrid]       ==[ALL-TOTAL]==       3193 (  799)     4362 (  799)    25.02    18.32    21.15

Task 3

------------------------------------------------------------------------------------------------
   TEAM                                 gold (match)   answer (match)   recall    prec.   fscore
------------------------------------------------------------------------------------------------
[ConcordU ]       ==[ALL-TOTAL]==       3617 ( 1182)     1943 ( 1182)    32.68    60.83    42.52
[VIBGhent ]       ==[ALL-TOTAL]==       3617 ( 1105)     2227 ( 1104)    30.55    49.57    37.80
[ASU+HU+BU]       ==[ALL-TOTAL]==       3617 (  710)     1185 (  710)    19.63    59.92    29.57
[  NICTA  ]       ==[ALL-TOTAL]==       3617 (  577)     1450 (  575)    15.95    39.66    22.75
[ USzeged ]       ==[ALL-TOTAL]==       3617 (  722)     3113 (  722)    19.96    23.19    21.46
[CCP-BTMG ]       ==[ALL-TOTAL]==       3617 (  446)      777 (  446)    12.33    57.40    20.30