BioNLP'09 Shared Task on
Event Extraction
in conjunction with
BioNLP, a NAACL-HLT 2009
workshop, June 4-5 2009, Boulder, Colorado
Final Evaluation Results
of University of Manchester
Task 1
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 77) 147 ( 77) 44.25 52.38 47.98
Binding 347 ( 39) 109 ( 39) 11.24 35.78 17.11
Gene_expression 722 ( 372) 548 ( 372) 51.52 67.88 58.58
Transcription 137 ( 20) 31 ( 20) 14.60 64.52 23.81
Protein_catabolism 14 ( 6) 12 ( 6) 42.86 50.00 46.15
Phosphorylation 135 ( 103) 197 ( 103) 76.30 52.28 62.05
==[EVT-TOTAL]== 1529 ( 617) 1044 ( 617) 40.35 59.10 47.96
------------------------------------------------------------------------------------
Regulation 291 ( 9) 47 ( 9) 3.09 19.15 5.33
Positive_regulation 983 ( 10) 124 ( 10) 1.02 8.06 1.81
Negative_regulation 379 ( 45) 229 ( 45) 11.87 19.65 14.80
==[REG-TOTAL]== 1653 ( 64) 400 ( 64) 3.87 16.00 6.23
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3182 ( 681) 1444 ( 681) 21.40 47.16 29.44
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 78) 147 ( 78) 44.83 53.06 48.60
Binding 347 ( 44) 109 ( 44) 12.68 40.37 19.30
Gene_expression 722 ( 380) 548 ( 380) 52.63 69.34 59.84
Transcription 137 ( 21) 31 ( 21) 15.33 67.74 25.00
Protein_catabolism 14 ( 6) 12 ( 6) 42.86 50.00 46.15
Phosphorylation 135 ( 106) 197 ( 106) 78.52 53.81 63.86
==[EVT-TOTAL]== 1529 ( 635) 1044 ( 635) 41.53 60.82 49.36
------------------------------------------------------------------------------------
Regulation 291 ( 9) 47 ( 9) 3.09 19.15 5.33
Positive_regulation 983 ( 11) 124 ( 11) 1.12 8.87 1.99
Negative_regulation 379 ( 47) 229 ( 47) 12.40 20.52 15.46
==[REG-TOTAL]== 1653 ( 67) 400 ( 67) 4.05 16.75 6.53
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3182 ( 702) 1444 ( 702) 22.06 48.61 30.35
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 78) 147 ( 78) 44.83 53.06 48.60
Binding 347 ( 44) 109 ( 44) 12.68 40.37 19.30
Gene_expression 722 ( 380) 548 ( 380) 52.63 69.34 59.84
Transcription 137 ( 21) 31 ( 21) 15.33 67.74 25.00
Protein_catabolism 14 ( 6) 12 ( 6) 42.86 50.00 46.15
Phosphorylation 135 ( 106) 197 ( 106) 78.52 53.81 63.86
==[EVT-TOTAL]== 1529 ( 635) 1044 ( 635) 41.53 60.82 49.36
------------------------------------------------------------------------------------
Regulation 291 ( 9) 47 ( 9) 3.09 19.15 5.33
Positive_regulation 983 ( 11) 124 ( 11) 1.12 8.87 1.99
Negative_regulation 379 ( 47) 229 ( 47) 12.40 20.52 15.46
==[REG-TOTAL]== 1653 ( 67) 400 ( 67) 4.05 16.75 6.53
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3182 ( 702) 1444 ( 702) 22.06 48.61 30.35
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 77) 147 ( 77) 44.25 52.38 47.98
Binding 398 ( 70) 109 ( 70) 17.59 64.22 27.61
Gene_expression 722 ( 372) 548 ( 372) 51.52 67.88 58.58
Transcription 137 ( 20) 31 ( 20) 14.60 64.52 23.81
Protein_catabolism 14 ( 6) 12 ( 6) 42.86 50.00 46.15
Phosphorylation 135 ( 103) 197 ( 103) 76.30 52.28 62.05
==[EVT-TOTAL]== 1580 ( 648) 1044 ( 648) 41.01 62.07 49.39
------------------------------------------------------------------------------------
Regulation 338 ( 15) 47 ( 15) 4.44 31.91 7.79
Positive_regulation 1186 ( 14) 124 ( 14) 1.18 11.29 2.14
Negative_regulation 416 ( 51) 229 ( 51) 12.26 22.27 15.81
==[REG-TOTAL]== 1940 ( 80) 400 ( 80) 4.12 20.00 6.84
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3520 ( 728) 1444 ( 728) 20.68 50.42 29.33
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 78) 147 ( 78) 44.83 53.06 48.60
Binding 398 ( 80) 109 ( 80) 20.10 73.39 31.56
Gene_expression 722 ( 380) 548 ( 380) 52.63 69.34 59.84
Transcription 137 ( 21) 31 ( 21) 15.33 67.74 25.00
Protein_catabolism 14 ( 6) 12 ( 6) 42.86 50.00 46.15
Phosphorylation 135 ( 106) 197 ( 106) 78.52 53.81 63.86
==[EVT-TOTAL]== 1580 ( 671) 1044 ( 671) 42.47 64.27 51.14
------------------------------------------------------------------------------------
Regulation 338 ( 16) 47 ( 16) 4.73 34.04 8.31
Positive_regulation 1186 ( 15) 124 ( 15) 1.26 12.10 2.29
Negative_regulation 416 ( 53) 229 ( 53) 12.74 23.14 16.43
==[REG-TOTAL]== 1940 ( 84) 400 ( 84) 4.33 21.00 7.18
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3520 ( 755) 1444 ( 755) 21.45 52.29 30.42
------------------------------------------------------------------------------------
Task 2
N/A
N/A
N/A
N/A
N/A
Task 3
N/A
N/A
N/A
N/A
N/A