BioNLP'09 Shared Task on
Event Extraction
in conjunction with
BioNLP, a NAACL-HLT 2009
workshop, June 4-5 2009, Boulder, Colorado
Final Evaluation Results
of University of Cambridge
Task 1
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 44) 50 ( 44) 25.29 88.00 39.29
Binding 347 ( 41) 138 ( 41) 11.82 29.71 16.91
Gene_expression 722 ( 323) 434 ( 323) 44.74 74.42 55.88
Transcription 137 ( 17) 30 ( 17) 12.41 56.67 20.36
Protein_catabolism 14 ( 5) 6 ( 5) 35.71 83.33 50.00
Phosphorylation 135 ( 64) 86 ( 64) 47.41 74.42 57.92
==[EVT-TOTAL]== 1529 ( 494) 744 ( 494) 32.31 66.40 43.47
------------------------------------------------------------------------------------
Regulation 291 ( 24) 74 ( 24) 8.25 32.43 13.15
Positive_regulation 983 ( 103) 291 ( 103) 10.48 35.40 16.17
Negative_regulation 379 ( 25) 72 ( 25) 6.60 34.72 11.09
==[REG-TOTAL]== 1653 ( 152) 437 ( 152) 9.20 34.78 14.55
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3182 ( 646) 1181 ( 646) 20.30 54.70 29.61
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 45) 50 ( 45) 25.86 90.00 40.18
Binding 347 ( 44) 138 ( 44) 12.68 31.88 18.14
Gene_expression 722 ( 329) 434 ( 329) 45.57 75.81 56.92
Transcription 137 ( 17) 30 ( 17) 12.41 56.67 20.36
Protein_catabolism 14 ( 5) 6 ( 5) 35.71 83.33 50.00
Phosphorylation 135 ( 67) 86 ( 67) 49.63 77.91 60.63
==[EVT-TOTAL]== 1529 ( 507) 744 ( 507) 33.16 68.15 44.61
------------------------------------------------------------------------------------
Regulation 291 ( 27) 74 ( 27) 9.28 36.49 14.79
Positive_regulation 983 ( 112) 291 ( 112) 11.39 38.49 17.58
Negative_regulation 379 ( 26) 72 ( 26) 6.86 36.11 11.53
==[REG-TOTAL]== 1653 ( 165) 437 ( 165) 9.98 37.76 15.79
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3182 ( 672) 1181 ( 672) 21.12 56.90 30.80
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 45) 50 ( 45) 25.86 90.00 40.18
Binding 347 ( 44) 138 ( 44) 12.68 31.88 18.14
Gene_expression 722 ( 329) 434 ( 329) 45.57 75.81 56.92
Transcription 137 ( 17) 30 ( 17) 12.41 56.67 20.36
Protein_catabolism 14 ( 5) 6 ( 5) 35.71 83.33 50.00
Phosphorylation 135 ( 67) 86 ( 67) 49.63 77.91 60.63
==[EVT-TOTAL]== 1529 ( 507) 744 ( 507) 33.16 68.15 44.61
------------------------------------------------------------------------------------
Regulation 291 ( 27) 74 ( 27) 9.28 36.49 14.79
Positive_regulation 983 ( 112) 291 ( 112) 11.39 38.49 17.58
Negative_regulation 379 ( 26) 72 ( 26) 6.86 36.11 11.53
==[REG-TOTAL]== 1653 ( 165) 437 ( 165) 9.98 37.76 15.79
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3182 ( 672) 1181 ( 672) 21.12 56.90 30.80
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 44) 50 ( 44) 25.29 88.00 39.29
Binding 398 ( 72) 138 ( 72) 18.09 52.17 26.87
Gene_expression 722 ( 323) 434 ( 323) 44.74 74.42 55.88
Transcription 137 ( 17) 30 ( 17) 12.41 56.67 20.36
Protein_catabolism 14 ( 5) 6 ( 5) 35.71 83.33 50.00
Phosphorylation 135 ( 64) 86 ( 64) 47.41 74.42 57.92
==[EVT-TOTAL]== 1580 ( 525) 744 ( 525) 33.23 70.56 45.18
------------------------------------------------------------------------------------
Regulation 338 ( 29) 77 ( 29) 8.58 37.66 13.98
Positive_regulation 1186 ( 151) 303 ( 151) 12.73 49.83 20.28
Negative_regulation 416 ( 32) 75 ( 32) 7.69 42.67 13.03
==[REG-TOTAL]== 1940 ( 212) 455 ( 212) 10.93 46.59 17.70
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3520 ( 737) 1199 ( 737) 20.94 61.47 31.24
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 45) 50 ( 45) 25.86 90.00 40.18
Binding 398 ( 75) 138 ( 75) 18.84 54.35 27.99
Gene_expression 722 ( 329) 434 ( 329) 45.57 75.81 56.92
Transcription 137 ( 17) 30 ( 17) 12.41 56.67 20.36
Protein_catabolism 14 ( 5) 6 ( 5) 35.71 83.33 50.00
Phosphorylation 135 ( 67) 86 ( 67) 49.63 77.91 60.63
==[EVT-TOTAL]== 1580 ( 538) 744 ( 538) 34.05 72.31 46.30
------------------------------------------------------------------------------------
Regulation 338 ( 34) 77 ( 34) 10.06 44.16 16.39
Positive_regulation 1186 ( 163) 303 ( 163) 13.74 53.80 21.89
Negative_regulation 416 ( 36) 75 ( 36) 8.65 48.00 14.66
==[REG-TOTAL]== 1940 ( 233) 455 ( 233) 12.01 51.21 19.46
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3520 ( 771) 1199 ( 771) 21.90 64.30 32.68
------------------------------------------------------------------------------------
Task 2
N/A
N/A
N/A
N/A
N/A
Task 3
N/A
N/A
N/A
N/A
N/A