BioNLP'09 Shared Task on
Event Extraction
in conjunction with
BioNLP, a NAACL-HLT 2009
workshop, June 4-5 2009, Boulder, Colorado
Final Evaluation Results
of Biocomputing unit, CNB, Madrid
Task 1
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 60) 114 ( 60) 34.48 52.63 41.67
Binding 347 ( 101) 560 ( 101) 29.11 18.04 22.27
Gene_expression 722 ( 350) 679 ( 350) 48.48 51.55 49.96
Transcription 137 ( 36) 341 ( 36) 26.28 10.56 15.06
Protein_catabolism 14 ( 11) 20 ( 11) 78.57 55.00 64.71
Phosphorylation 135 ( 79) 123 ( 79) 58.52 64.23 61.24
==[EVT-TOTAL]== 1529 ( 637) 1837 ( 637) 41.66 34.68 37.85
------------------------------------------------------------------------------------
Regulation 291 ( 27) 555 ( 27) 9.28 4.86 6.38
Positive_regulation 983 ( 112) 1506 ( 112) 11.39 7.44 9.00
Negative_regulation 379 ( 31) 466 ( 31) 8.18 6.65 7.34
==[REG-TOTAL]== 1653 ( 170) 2527 ( 170) 10.28 6.73 8.13
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3182 ( 807) 4364 ( 807) 25.36 18.49 21.39
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 61) 114 ( 61) 35.06 53.51 42.36
Binding 347 ( 115) 560 ( 115) 33.14 20.54 25.36
Gene_expression 722 ( 374) 679 ( 374) 51.80 55.08 53.39
Transcription 137 ( 67) 339 ( 67) 48.91 19.76 28.15
Protein_catabolism 14 ( 11) 20 ( 11) 78.57 55.00 64.71
Phosphorylation 135 ( 81) 123 ( 81) 60.00 65.85 62.79
==[EVT-TOTAL]== 1529 ( 709) 1835 ( 709) 46.37 38.64 42.15
------------------------------------------------------------------------------------
Regulation 291 ( 30) 555 ( 30) 10.31 5.41 7.09
Positive_regulation 983 ( 128) 1506 ( 128) 13.02 8.50 10.29
Negative_regulation 379 ( 36) 466 ( 36) 9.50 7.73 8.52
==[REG-TOTAL]== 1653 ( 194) 2527 ( 194) 11.74 7.68 9.28
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3182 ( 903) 4362 ( 903) 28.38 20.70 23.94
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 61) 114 ( 61) 35.06 53.51 42.36
Binding 347 ( 115) 560 ( 115) 33.14 20.54 25.36
Gene_expression 722 ( 374) 679 ( 374) 51.80 55.08 53.39
Transcription 137 ( 67) 339 ( 67) 48.91 19.76 28.15
Protein_catabolism 14 ( 11) 20 ( 11) 78.57 55.00 64.71
Phosphorylation 135 ( 81) 123 ( 81) 60.00 65.85 62.79
==[EVT-TOTAL]== 1529 ( 709) 1835 ( 709) 46.37 38.64 42.15
------------------------------------------------------------------------------------
Regulation 291 ( 30) 555 ( 30) 10.31 5.41 7.09
Positive_regulation 983 ( 132) 1506 ( 132) 13.43 8.76 10.61
Negative_regulation 379 ( 40) 466 ( 40) 10.55 8.58 9.47
==[REG-TOTAL]== 1653 ( 202) 2527 ( 202) 12.22 7.99 9.67
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3182 ( 911) 4362 ( 911) 28.63 20.88 24.15
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 60) 114 ( 60) 34.48 52.63 41.67
Binding 398 ( 210) 761 ( 210) 52.76 27.60 36.24
Gene_expression 722 ( 350) 679 ( 350) 48.48 51.55 49.96
Transcription 137 ( 36) 341 ( 36) 26.28 10.56 15.06
Protein_catabolism 14 ( 11) 20 ( 11) 78.57 55.00 64.71
Phosphorylation 135 ( 79) 123 ( 79) 58.52 64.23 61.24
==[EVT-TOTAL]== 1580 ( 746) 2038 ( 746) 47.22 36.60 41.24
------------------------------------------------------------------------------------
Regulation 338 ( 51) 771 ( 51) 15.09 6.61 9.20
Positive_regulation 1186 ( 237) 2222 ( 237) 19.98 10.67 13.91
Negative_regulation 416 ( 64) 669 ( 64) 15.38 9.57 11.80
==[REG-TOTAL]== 1940 ( 352) 3662 ( 352) 18.14 9.61 12.57
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3520 ( 1098) 5700 ( 1098) 31.19 19.26 23.82
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 61) 114 ( 61) 35.06 53.51 42.36
Binding 398 ( 238) 753 ( 238) 59.80 31.61 41.36
Gene_expression 722 ( 374) 679 ( 374) 51.80 55.08 53.39
Transcription 137 ( 67) 339 ( 67) 48.91 19.76 28.15
Protein_catabolism 14 ( 11) 20 ( 11) 78.57 55.00 64.71
Phosphorylation 135 ( 81) 123 ( 81) 60.00 65.85 62.79
==[EVT-TOTAL]== 1580 ( 832) 2028 ( 832) 52.66 41.03 46.12
------------------------------------------------------------------------------------
Regulation 338 ( 60) 770 ( 60) 17.75 7.79 10.83
Positive_regulation 1186 ( 277) 2220 ( 276) 23.36 12.43 16.23
Negative_regulation 416 ( 76) 669 ( 76) 18.27 11.36 14.01
==[REG-TOTAL]== 1940 ( 413) 3659 ( 412) 21.29 11.26 14.73
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3520 ( 1245) 5687 ( 1244) 35.37 21.87 27.03
------------------------------------------------------------------------------------
Task 2
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 48) 114 ( 48) 27.59 42.11 33.33
Binding 349 ( 59) 560 ( 59) 16.91 10.54 12.98
Gene_expression 722 ( 350) 679 ( 350) 48.48 51.55 49.96
Transcription 137 ( 36) 341 ( 36) 26.28 10.56 15.06
Protein_catabolism 14 ( 11) 20 ( 11) 78.57 55.00 64.71
Phosphorylation 139 ( 65) 123 ( 65) 46.76 52.85 49.62
==[EVT-TOTAL]== 1535 ( 569) 1837 ( 569) 37.07 30.97 33.75
------------------------------------------------------------------------------------
Regulation 292 ( 22) 555 ( 22) 7.53 3.96 5.19
Positive_regulation 987 ( 79) 1506 ( 79) 8.00 5.25 6.34
Negative_regulation 379 ( 27) 466 ( 27) 7.12 5.79 6.39
==[REG-TOTAL]== 1658 ( 128) 2527 ( 128) 7.72 5.07 6.12
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3193 ( 697) 4364 ( 697) 21.83 15.97 18.45
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 49) 114 ( 49) 28.16 42.98 34.03
Binding 349 ( 68) 560 ( 68) 19.48 12.14 14.96
Gene_expression 722 ( 374) 679 ( 374) 51.80 55.08 53.39
Transcription 137 ( 67) 339 ( 67) 48.91 19.76 28.15
Protein_catabolism 14 ( 11) 20 ( 11) 78.57 55.00 64.71
Phosphorylation 139 ( 69) 123 ( 69) 49.64 56.10 52.67
==[EVT-TOTAL]== 1535 ( 638) 1835 ( 638) 41.56 34.77 37.86
------------------------------------------------------------------------------------
Regulation 292 ( 25) 555 ( 25) 8.56 4.50 5.90
Positive_regulation 987 ( 92) 1506 ( 92) 9.32 6.11 7.38
Negative_regulation 379 ( 31) 466 ( 31) 8.18 6.65 7.34
==[REG-TOTAL]== 1658 ( 148) 2527 ( 148) 8.93 5.86 7.07
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3193 ( 786) 4362 ( 786) 24.62 18.02 20.81
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 49) 114 ( 49) 28.16 42.98 34.03
Binding 349 ( 68) 560 ( 68) 19.48 12.14 14.96
Gene_expression 722 ( 374) 679 ( 374) 51.80 55.08 53.39
Transcription 137 ( 67) 339 ( 67) 48.91 19.76 28.15
Protein_catabolism 14 ( 11) 20 ( 11) 78.57 55.00 64.71
Phosphorylation 139 ( 69) 123 ( 69) 49.64 56.10 52.67
==[EVT-TOTAL]== 1535 ( 638) 1835 ( 638) 41.56 34.77 37.86
------------------------------------------------------------------------------------
Regulation 292 ( 25) 555 ( 25) 8.56 4.50 5.90
Positive_regulation 987 ( 99) 1506 ( 99) 10.03 6.57 7.94
Negative_regulation 379 ( 37) 466 ( 37) 9.76 7.94 8.76
==[REG-TOTAL]== 1658 ( 161) 2527 ( 161) 9.71 6.37 7.69
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3193 ( 799) 4362 ( 799) 25.02 18.32 21.15
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 239 ( 80) 158 ( 80) 33.47 50.63 40.30
Binding 419 ( 211) 1021 ( 211) 50.36 20.67 29.31
Gene_expression 722 ( 350) 679 ( 350) 48.48 51.55 49.96
Transcription 137 ( 36) 341 ( 36) 26.28 10.56 15.06
Protein_catabolism 14 ( 11) 20 ( 11) 78.57 55.00 64.71
Phosphorylation 191 ( 125) 207 ( 125) 65.45 60.39 62.81
==[EVT-TOTAL]== 1722 ( 813) 2426 ( 813) 47.21 33.51 39.20
------------------------------------------------------------------------------------
Regulation 349 ( 55) 914 ( 55) 15.76 6.02 8.71
Positive_regulation 1220 ( 248) 2606 ( 248) 20.33 9.52 12.96
Negative_regulation 425 ( 67) 765 ( 67) 15.76 8.76 11.26
==[REG-TOTAL]== 1994 ( 370) 4285 ( 370) 18.56 8.63 11.79
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3716 ( 1183) 6711 ( 1183) 31.84 17.63 22.69
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 239 ( 82) 158 ( 82) 34.31 51.90 41.31
Binding 419 ( 244) 1013 ( 244) 58.23 24.09 34.08
Gene_expression 722 ( 374) 679 ( 374) 51.80 55.08 53.39
Transcription 137 ( 67) 339 ( 67) 48.91 19.76 28.15
Protein_catabolism 14 ( 11) 20 ( 11) 78.57 55.00 64.71
Phosphorylation 191 ( 129) 207 ( 129) 67.54 62.32 64.82
==[EVT-TOTAL]== 1722 ( 907) 2416 ( 907) 52.67 37.54 43.84
------------------------------------------------------------------------------------
Regulation 349 ( 65) 913 ( 65) 18.62 7.12 10.30
Positive_regulation 1220 ( 291) 2603 ( 290) 23.85 11.14 15.19
Negative_regulation 425 ( 80) 764 ( 80) 18.82 10.47 13.46
==[REG-TOTAL]== 1994 ( 436) 4280 ( 435) 21.87 10.16 13.88
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3716 ( 1343) 6696 ( 1342) 36.14 20.04 25.78
------------------------------------------------------------------------------------
Task 3
N/A
N/A
N/A
N/A
N/A