BioNLP'09 Shared Task on
Event Extraction
in conjunction with
BioNLP, a NAACL-HLT 2009
workshop, June 4-5 2009, Boulder, Colorado
Final Evaluation Results
of Fraunhofer Institute SCAI
Task 1
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 46) 58 ( 46) 26.44 79.31 39.66
Binding 347 ( 85) 284 ( 85) 24.50 29.93 26.94
Gene_expression 722 ( 334) 520 ( 334) 46.26 64.23 53.78
Transcription 137 ( 23) 62 ( 23) 16.79 37.10 23.12
Protein_catabolism 14 ( 4) 11 ( 4) 28.57 36.36 32.00
Phosphorylation 135 ( 66) 80 ( 66) 48.89 82.50 61.40
==[EVT-TOTAL]== 1529 ( 558) 1015 ( 558) 36.49 54.98 43.87
------------------------------------------------------------------------------------
Regulation 291 ( 25) 280 ( 25) 8.59 8.93 8.76
Positive_regulation 983 ( 115) 771 ( 115) 11.70 14.92 13.11
Negative_regulation 379 ( 51) 212 ( 51) 13.46 24.06 17.26
==[REG-TOTAL]== 1653 ( 191) 1263 ( 191) 11.55 15.12 13.10
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3182 ( 749) 2278 ( 749) 23.54 32.88 27.44
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 48) 58 ( 48) 27.59 82.76 41.38
Binding 347 ( 100) 284 ( 100) 28.82 35.21 31.70
Gene_expression 722 ( 364) 520 ( 364) 50.42 70.00 58.62
Transcription 137 ( 32) 62 ( 32) 23.36 51.61 32.16
Protein_catabolism 14 ( 4) 11 ( 4) 28.57 36.36 32.00
Phosphorylation 135 ( 69) 80 ( 69) 51.11 86.25 64.19
==[EVT-TOTAL]== 1529 ( 617) 1015 ( 617) 40.35 60.79 48.51
------------------------------------------------------------------------------------
Regulation 291 ( 28) 280 ( 28) 9.62 10.00 9.81
Positive_regulation 983 ( 129) 771 ( 129) 13.12 16.73 14.71
Negative_regulation 379 ( 52) 212 ( 52) 13.72 24.53 17.60
==[REG-TOTAL]== 1653 ( 209) 1263 ( 209) 12.64 16.55 14.33
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3182 ( 826) 2278 ( 826) 25.96 36.26 30.26
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 48) 58 ( 48) 27.59 82.76 41.38
Binding 347 ( 100) 284 ( 100) 28.82 35.21 31.70
Gene_expression 722 ( 364) 520 ( 364) 50.42 70.00 58.62
Transcription 137 ( 32) 62 ( 32) 23.36 51.61 32.16
Protein_catabolism 14 ( 4) 11 ( 4) 28.57 36.36 32.00
Phosphorylation 135 ( 69) 80 ( 69) 51.11 86.25 64.19
==[EVT-TOTAL]== 1529 ( 617) 1015 ( 617) 40.35 60.79 48.51
------------------------------------------------------------------------------------
Regulation 291 ( 28) 280 ( 28) 9.62 10.00 9.81
Positive_regulation 983 ( 129) 771 ( 129) 13.12 16.73 14.71
Negative_regulation 379 ( 52) 212 ( 52) 13.72 24.53 17.60
==[REG-TOTAL]== 1653 ( 209) 1263 ( 209) 12.64 16.55 14.33
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3182 ( 826) 2278 ( 826) 25.96 36.26 30.26
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 46) 58 ( 46) 26.44 79.31 39.66
Binding 398 ( 128) 313 ( 128) 32.16 40.89 36.01
Gene_expression 722 ( 334) 520 ( 334) 46.26 64.23 53.78
Transcription 137 ( 23) 62 ( 23) 16.79 37.10 23.12
Protein_catabolism 14 ( 4) 11 ( 4) 28.57 36.36 32.00
Phosphorylation 135 ( 66) 80 ( 66) 48.89 82.50 61.40
==[EVT-TOTAL]== 1580 ( 601) 1044 ( 601) 38.04 57.57 45.81
------------------------------------------------------------------------------------
Regulation 338 ( 31) 280 ( 31) 9.17 11.07 10.03
Positive_regulation 1186 ( 177) 775 ( 177) 14.92 22.84 18.05
Negative_regulation 416 ( 57) 214 ( 57) 13.70 26.64 18.10
==[REG-TOTAL]== 1940 ( 265) 1269 ( 265) 13.66 20.88 16.52
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3520 ( 866) 2313 ( 866) 24.60 37.44 29.69
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 48) 58 ( 48) 27.59 82.76 41.38
Binding 398 ( 148) 312 ( 148) 37.19 47.44 41.69
Gene_expression 722 ( 364) 520 ( 364) 50.42 70.00 58.62
Transcription 137 ( 32) 62 ( 32) 23.36 51.61 32.16
Protein_catabolism 14 ( 4) 11 ( 4) 28.57 36.36 32.00
Phosphorylation 135 ( 69) 80 ( 69) 51.11 86.25 64.19
==[EVT-TOTAL]== 1580 ( 665) 1043 ( 665) 42.09 63.76 50.71
------------------------------------------------------------------------------------
Regulation 338 ( 34) 280 ( 34) 10.06 12.14 11.00
Positive_regulation 1186 ( 198) 775 ( 198) 16.69 25.55 20.19
Negative_regulation 416 ( 61) 214 ( 61) 14.66 28.50 19.37
==[REG-TOTAL]== 1940 ( 293) 1269 ( 293) 15.10 23.09 18.26
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3520 ( 958) 2312 ( 958) 27.22 41.44 32.85
------------------------------------------------------------------------------------
Task 2
N/A
N/A
N/A
N/A
N/A
Task 3
N/A
N/A
N/A
N/A
N/A