BioNLP'09 Shared Task on
Event Extraction
in conjunction with
BioNLP, a NAACL-HLT 2009
workshop, June 4-5 2009, Boulder, Colorado
Final Evaluation Results
of Team 09 (anonymized)
Task 1
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 1) 1 ( 1) 0.57 100.00 1.14
Binding 347 ( 9) 112 ( 9) 2.59 8.04 3.92
Gene_expression 722 ( 288) 699 ( 288) 39.89 41.20 40.53
Transcription 137 ( 16) 53 ( 16) 11.68 30.19 16.84
Protein_catabolism 14 ( 0) 0 ( 0) 0.00 0.00 0.00
Phosphorylation 135 ( 15) 20 ( 15) 11.11 75.00 19.35
==[EVT-TOTAL]== 1529 ( 329) 885 ( 329) 21.52 37.18 27.26
------------------------------------------------------------------------------------
Regulation 291 ( 0) 0 ( 0) 0.00 0.00 0.00
Positive_regulation 983 ( 37) 339 ( 37) 3.76 10.91 5.60
Negative_regulation 379 ( 1) 4 ( 1) 0.26 25.00 0.52
==[REG-TOTAL]== 1653 ( 38) 343 ( 38) 2.30 11.08 3.81
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3182 ( 367) 1228 ( 367) 11.53 29.89 16.64
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 1) 1 ( 1) 0.57 100.00 1.14
Binding 347 ( 11) 112 ( 11) 3.17 9.82 4.79
Gene_expression 722 ( 289) 668 ( 289) 40.03 43.26 41.58
Transcription 137 ( 16) 47 ( 16) 11.68 34.04 17.39
Protein_catabolism 14 ( 0) 0 ( 0) 0.00 0.00 0.00
Phosphorylation 135 ( 15) 20 ( 15) 11.11 75.00 19.35
==[EVT-TOTAL]== 1529 ( 332) 848 ( 332) 21.71 39.15 27.93
------------------------------------------------------------------------------------
Regulation 291 ( 0) 0 ( 0) 0.00 0.00 0.00
Positive_regulation 983 ( 39) 332 ( 39) 3.97 11.75 5.93
Negative_regulation 379 ( 1) 4 ( 1) 0.26 25.00 0.52
==[REG-TOTAL]== 1653 ( 40) 336 ( 40) 2.42 11.90 4.02
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3182 ( 372) 1184 ( 372) 11.69 31.42 17.04
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 1) 1 ( 1) 0.57 100.00 1.14
Binding 347 ( 11) 112 ( 11) 3.17 9.82 4.79
Gene_expression 722 ( 289) 668 ( 289) 40.03 43.26 41.58
Transcription 137 ( 16) 47 ( 16) 11.68 34.04 17.39
Protein_catabolism 14 ( 0) 0 ( 0) 0.00 0.00 0.00
Phosphorylation 135 ( 15) 20 ( 15) 11.11 75.00 19.35
==[EVT-TOTAL]== 1529 ( 332) 848 ( 332) 21.71 39.15 27.93
------------------------------------------------------------------------------------
Regulation 291 ( 0) 0 ( 0) 0.00 0.00 0.00
Positive_regulation 983 ( 39) 332 ( 39) 3.97 11.75 5.93
Negative_regulation 379 ( 1) 4 ( 1) 0.26 25.00 0.52
==[REG-TOTAL]== 1653 ( 40) 336 ( 40) 2.42 11.90 4.02
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3182 ( 372) 1184 ( 372) 11.69 31.42 17.04
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 1) 1 ( 1) 0.57 100.00 1.14
Binding 398 ( 51) 162 ( 51) 12.81 31.48 18.21
Gene_expression 722 ( 288) 699 ( 288) 39.89 41.20 40.53
Transcription 137 ( 16) 53 ( 16) 11.68 30.19 16.84
Protein_catabolism 14 ( 0) 0 ( 0) 0.00 0.00 0.00
Phosphorylation 135 ( 15) 20 ( 15) 11.11 75.00 19.35
==[EVT-TOTAL]== 1580 ( 371) 935 ( 371) 23.48 39.68 29.50
------------------------------------------------------------------------------------
Regulation 338 ( 0) 0 ( 0) 0.00 0.00 0.00
Positive_regulation 1186 ( 45) 339 ( 45) 3.79 13.27 5.90
Negative_regulation 416 ( 1) 4 ( 1) 0.24 25.00 0.48
==[REG-TOTAL]== 1940 ( 46) 343 ( 46) 2.37 13.41 4.03
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3520 ( 417) 1278 ( 417) 11.85 32.63 17.38
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 1) 1 ( 1) 0.57 100.00 1.14
Binding 398 ( 63) 162 ( 63) 15.83 38.89 22.50
Gene_expression 722 ( 289) 668 ( 289) 40.03 43.26 41.58
Transcription 137 ( 16) 47 ( 16) 11.68 34.04 17.39
Protein_catabolism 14 ( 0) 0 ( 0) 0.00 0.00 0.00
Phosphorylation 135 ( 15) 20 ( 15) 11.11 75.00 19.35
==[EVT-TOTAL]== 1580 ( 384) 898 ( 384) 24.30 42.76 30.99
------------------------------------------------------------------------------------
Regulation 338 ( 0) 0 ( 0) 0.00 0.00 0.00
Positive_regulation 1186 ( 48) 331 ( 48) 4.05 14.50 6.33
Negative_regulation 416 ( 1) 4 ( 1) 0.24 25.00 0.48
==[REG-TOTAL]== 1940 ( 49) 335 ( 49) 2.53 14.63 4.31
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3520 ( 433) 1233 ( 433) 12.30 35.12 18.22
------------------------------------------------------------------------------------
Task 2
N/A
N/A
N/A
N/A
N/A
Task 3
N/A
N/A
N/A
N/A
N/A