BioNLP'09 Shared Task on
Event Extraction
in conjunction with
BioNLP, a NAACL-HLT 2009
workshop, June 4-5 2009, Boulder, Colorado
Final Evaluation Results
of The University of New South Wales
Task 1
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 39) 41 ( 39) 22.41 95.12 36.28
Binding 347 ( 79) 220 ( 79) 22.77 35.91 27.87
Gene_expression 722 ( 347) 501 ( 347) 48.06 69.26 56.75
Transcription 137 ( 21) 103 ( 21) 15.33 20.39 17.50
Protein_catabolism 14 ( 6) 11 ( 6) 42.86 54.55 48.00
Phosphorylation 135 ( 82) 119 ( 82) 60.74 68.91 64.57
==[EVT-TOTAL]== 1529 ( 574) 995 ( 574) 37.54 57.69 45.48
------------------------------------------------------------------------------------
Regulation 291 ( 32) 161 ( 32) 11.00 19.88 14.16
Positive_regulation 983 ( 156) 569 ( 156) 15.87 27.42 20.10
Negative_regulation 379 ( 61) 232 ( 61) 16.09 26.29 19.97
==[REG-TOTAL]== 1653 ( 249) 962 ( 249) 15.06 25.88 19.04
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3182 ( 823) 1957 ( 823) 25.86 42.05 32.03
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 40) 41 ( 40) 22.99 97.56 37.21
Binding 347 ( 82) 220 ( 82) 23.63 37.27 28.92
Gene_expression 722 ( 370) 501 ( 370) 51.25 73.85 60.51
Transcription 137 ( 40) 103 ( 40) 29.20 38.83 33.33
Protein_catabolism 14 ( 7) 11 ( 7) 50.00 63.64 56.00
Phosphorylation 135 ( 85) 119 ( 85) 62.96 71.43 66.93
==[EVT-TOTAL]== 1529 ( 624) 995 ( 624) 40.81 62.71 49.45
------------------------------------------------------------------------------------
Regulation 291 ( 36) 161 ( 36) 12.37 22.36 15.93
Positive_regulation 983 ( 171) 569 ( 169) 17.40 29.70 21.94
Negative_regulation 379 ( 67) 232 ( 67) 17.68 28.88 21.93
==[REG-TOTAL]== 1653 ( 274) 962 ( 272) 16.58 28.27 20.90
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3182 ( 898) 1957 ( 896) 28.22 45.78 34.92
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 40) 41 ( 40) 22.99 97.56 37.21
Binding 347 ( 82) 220 ( 82) 23.63 37.27 28.92
Gene_expression 722 ( 370) 501 ( 370) 51.25 73.85 60.51
Transcription 137 ( 40) 103 ( 40) 29.20 38.83 33.33
Protein_catabolism 14 ( 7) 11 ( 7) 50.00 63.64 56.00
Phosphorylation 135 ( 85) 119 ( 85) 62.96 71.43 66.93
==[EVT-TOTAL]== 1529 ( 624) 995 ( 624) 40.81 62.71 49.45
------------------------------------------------------------------------------------
Regulation 291 ( 36) 161 ( 36) 12.37 22.36 15.93
Positive_regulation 983 ( 171) 569 ( 169) 17.40 29.70 21.94
Negative_regulation 379 ( 67) 232 ( 67) 17.68 28.88 21.93
==[REG-TOTAL]== 1653 ( 274) 962 ( 272) 16.58 28.27 20.90
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3182 ( 898) 1957 ( 896) 28.22 45.78 34.92
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 39) 41 ( 39) 22.41 95.12 36.28
Binding 398 ( 128) 220 ( 128) 32.16 58.18 41.42
Gene_expression 722 ( 347) 501 ( 347) 48.06 69.26 56.75
Transcription 137 ( 21) 103 ( 21) 15.33 20.39 17.50
Protein_catabolism 14 ( 6) 11 ( 6) 42.86 54.55 48.00
Phosphorylation 135 ( 82) 119 ( 82) 60.74 68.91 64.57
==[EVT-TOTAL]== 1580 ( 623) 995 ( 623) 39.43 62.61 48.39
------------------------------------------------------------------------------------
Regulation 338 ( 42) 161 ( 42) 12.43 26.09 16.83
Positive_regulation 1186 ( 239) 569 ( 239) 20.15 42.00 27.24
Negative_regulation 416 ( 69) 232 ( 69) 16.59 29.74 21.30
==[REG-TOTAL]== 1940 ( 350) 962 ( 350) 18.04 36.38 24.12
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3520 ( 973) 1957 ( 973) 27.64 49.72 35.53
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 40) 41 ( 40) 22.99 97.56 37.21
Binding 398 ( 132) 220 ( 132) 33.17 60.00 42.72
Gene_expression 722 ( 370) 501 ( 370) 51.25 73.85 60.51
Transcription 137 ( 40) 103 ( 40) 29.20 38.83 33.33
Protein_catabolism 14 ( 7) 11 ( 7) 50.00 63.64 56.00
Phosphorylation 135 ( 85) 119 ( 85) 62.96 71.43 66.93
==[EVT-TOTAL]== 1580 ( 674) 995 ( 674) 42.66 67.74 52.35
------------------------------------------------------------------------------------
Regulation 338 ( 48) 161 ( 48) 14.20 29.81 19.24
Positive_regulation 1186 ( 260) 569 ( 258) 21.92 45.34 29.56
Negative_regulation 416 ( 78) 232 ( 78) 18.75 33.62 24.07
==[REG-TOTAL]== 1940 ( 386) 962 ( 384) 19.90 39.92 26.56
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3520 ( 1060) 1957 ( 1058) 30.11 54.06 38.68
------------------------------------------------------------------------------------
Task 2
N/A
N/A
N/A
N/A
N/A
Task 3
N/A
N/A
N/A
N/A
N/A