BioNLP'09 Shared Task on
Event Extraction
in conjunction with
BioNLP, a NAACL-HLT 2009
workshop, June 4-5 2009, Boulder, Colorado
Final Evaluation Results
of University of Michigan
Task 1
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 77) 212 ( 77) 44.25 36.32 39.90
Binding 347 ( 90) 879 ( 90) 25.94 10.24 14.68
Gene_expression 722 ( 351) 1322 ( 351) 48.61 26.55 34.34
Transcription 137 ( 37) 701 ( 37) 27.01 5.28 8.83
Protein_catabolism 14 ( 10) 23 ( 10) 71.43 43.48 54.05
Phosphorylation 135 ( 95) 166 ( 95) 70.37 57.23 63.12
==[EVT-TOTAL]== 1529 ( 660) 3303 ( 660) 43.17 19.98 27.32
------------------------------------------------------------------------------------
Regulation 291 ( 6) 236 ( 6) 2.06 2.54 2.28
Positive_regulation 983 ( 133) 2669 ( 133) 13.53 4.98 7.28
Negative_regulation 379 ( 68) 702 ( 68) 17.94 9.69 12.58
==[REG-TOTAL]== 1653 ( 207) 3607 ( 207) 12.52 5.74 7.87
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3182 ( 867) 6910 ( 867) 27.25 12.55 17.18
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 80) 212 ( 80) 45.98 37.74 41.45
Binding 347 ( 110) 872 ( 110) 31.70 12.61 18.05
Gene_expression 722 ( 367) 1309 ( 367) 50.83 28.04 36.14
Transcription 137 ( 66) 696 ( 66) 48.18 9.48 15.85
Protein_catabolism 14 ( 10) 23 ( 10) 71.43 43.48 54.05
Phosphorylation 135 ( 100) 166 ( 100) 74.07 60.24 66.45
==[EVT-TOTAL]== 1529 ( 733) 3278 ( 733) 47.94 22.36 30.50
------------------------------------------------------------------------------------
Regulation 291 ( 9) 235 ( 9) 3.09 3.83 3.42
Positive_regulation 983 ( 152) 2645 ( 152) 15.46 5.75 8.38
Negative_regulation 379 ( 72) 702 ( 72) 19.00 10.26 13.32
==[REG-TOTAL]== 1653 ( 233) 3582 ( 233) 14.10 6.50 8.90
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3182 ( 966) 6860 ( 966) 30.36 14.08 19.24
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 80) 212 ( 80) 45.98 37.74 41.45
Binding 347 ( 110) 872 ( 110) 31.70 12.61 18.05
Gene_expression 722 ( 367) 1309 ( 367) 50.83 28.04 36.14
Transcription 137 ( 66) 696 ( 66) 48.18 9.48 15.85
Protein_catabolism 14 ( 10) 23 ( 10) 71.43 43.48 54.05
Phosphorylation 135 ( 100) 166 ( 100) 74.07 60.24 66.45
==[EVT-TOTAL]== 1529 ( 733) 3278 ( 733) 47.94 22.36 30.50
------------------------------------------------------------------------------------
Regulation 291 ( 9) 235 ( 9) 3.09 3.83 3.42
Positive_regulation 983 ( 153) 2644 ( 153) 15.56 5.79 8.44
Negative_regulation 379 ( 73) 702 ( 73) 19.26 10.40 13.51
==[REG-TOTAL]== 1653 ( 235) 3581 ( 235) 14.22 6.56 8.98
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3182 ( 968) 6859 ( 968) 30.42 14.11 19.28
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 77) 212 ( 77) 44.25 36.32 39.90
Binding 398 ( 234) 1327 ( 234) 58.79 17.63 27.13
Gene_expression 722 ( 351) 1322 ( 351) 48.61 26.55 34.34
Transcription 137 ( 37) 701 ( 37) 27.01 5.28 8.83
Protein_catabolism 14 ( 10) 23 ( 10) 71.43 43.48 54.05
Phosphorylation 135 ( 95) 166 ( 95) 70.37 57.23 63.12
==[EVT-TOTAL]== 1580 ( 804) 3751 ( 804) 50.89 21.43 30.16
------------------------------------------------------------------------------------
Regulation 338 ( 15) 260 ( 15) 4.44 5.77 5.02
Positive_regulation 1186 ( 219) 2872 ( 219) 18.47 7.63 10.79
Negative_regulation 416 ( 80) 754 ( 80) 19.23 10.61 13.68
==[REG-TOTAL]== 1940 ( 314) 3886 ( 314) 16.19 8.08 10.78
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3520 ( 1118) 7637 ( 1118) 31.76 14.64 20.04
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 80) 212 ( 80) 45.98 37.74 41.45
Binding 398 ( 280) 1309 ( 280) 70.35 21.39 32.81
Gene_expression 722 ( 367) 1309 ( 367) 50.83 28.04 36.14
Transcription 137 ( 66) 696 ( 66) 48.18 9.48 15.85
Protein_catabolism 14 ( 10) 23 ( 10) 71.43 43.48 54.05
Phosphorylation 135 ( 100) 166 ( 100) 74.07 60.24 66.45
==[EVT-TOTAL]== 1580 ( 903) 3715 ( 903) 57.15 24.31 34.11
------------------------------------------------------------------------------------
Regulation 338 ( 19) 259 ( 19) 5.62 7.34 6.37
Positive_regulation 1186 ( 248) 2832 ( 248) 20.91 8.76 12.34
Negative_regulation 416 ( 93) 754 ( 93) 22.36 12.33 15.90
==[REG-TOTAL]== 1940 ( 360) 3845 ( 360) 18.56 9.36 12.45
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3520 ( 1263) 7560 ( 1263) 35.88 16.71 22.80
------------------------------------------------------------------------------------
Task 2
N/A
N/A
N/A
N/A
N/A
Task 3
N/A
N/A
N/A
N/A
N/A