BioNLP'09 Shared Task on
Event Extraction
in conjunction with
BioNLP, a NAACL-HLT 2009
workshop, June 4-5 2009, Boulder, Colorado
Final Evaluation Results
of University of Tokyo, Tsujii lab
Task 1
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 54) 59 ( 54) 31.03 91.53 46.35
Binding 347 ( 112) 253 ( 112) 32.28 44.27 37.33
Gene_expression 722 ( 361) 512 ( 361) 50.00 70.51 58.51
Transcription 137 ( 44) 110 ( 44) 32.12 40.00 35.63
Protein_catabolism 14 ( 4) 5 ( 4) 28.57 80.00 42.11
Phosphorylation 135 ( 58) 73 ( 58) 42.96 79.45 55.77
==[EVT-TOTAL]== 1529 ( 633) 1012 ( 633) 41.40 62.55 49.82
------------------------------------------------------------------------------------
Regulation 291 ( 31) 81 ( 31) 10.65 38.27 16.67
Positive_regulation 983 ( 159) 527 ( 159) 16.17 30.17 21.06
Negative_regulation 379 ( 23) 90 ( 23) 6.07 25.56 9.81
==[REG-TOTAL]== 1653 ( 213) 698 ( 213) 12.89 30.52 18.12
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3182 ( 846) 1710 ( 846) 26.59 49.47 34.59
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 55) 58 ( 55) 31.61 94.83 47.41
Binding 347 ( 120) 237 ( 120) 34.58 50.63 41.10
Gene_expression 722 ( 372) 509 ( 372) 51.52 73.08 60.44
Transcription 137 ( 49) 104 ( 49) 35.77 47.12 40.66
Protein_catabolism 14 ( 4) 5 ( 4) 28.57 80.00 42.11
Phosphorylation 135 ( 60) 72 ( 60) 44.44 83.33 57.97
==[EVT-TOTAL]== 1529 ( 660) 985 ( 660) 43.17 67.01 52.51
------------------------------------------------------------------------------------
Regulation 291 ( 33) 78 ( 33) 11.34 42.31 17.89
Positive_regulation 983 ( 175) 518 ( 175) 17.80 33.78 23.32
Negative_regulation 379 ( 25) 90 ( 25) 6.60 27.78 10.66
==[REG-TOTAL]== 1653 ( 233) 686 ( 233) 14.10 33.97 19.92
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3182 ( 893) 1671 ( 893) 28.06 53.44 36.80
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 55) 58 ( 55) 31.61 94.83 47.41
Binding 347 ( 120) 237 ( 120) 34.58 50.63 41.10
Gene_expression 722 ( 372) 509 ( 372) 51.52 73.08 60.44
Transcription 137 ( 49) 104 ( 49) 35.77 47.12 40.66
Protein_catabolism 14 ( 4) 5 ( 4) 28.57 80.00 42.11
Phosphorylation 135 ( 60) 72 ( 60) 44.44 83.33 57.97
==[EVT-TOTAL]== 1529 ( 660) 985 ( 660) 43.17 67.01 52.51
------------------------------------------------------------------------------------
Regulation 291 ( 33) 78 ( 33) 11.34 42.31 17.89
Positive_regulation 983 ( 177) 518 ( 177) 18.01 34.17 23.58
Negative_regulation 379 ( 25) 90 ( 25) 6.60 27.78 10.66
==[REG-TOTAL]== 1653 ( 235) 686 ( 235) 14.22 34.26 20.09
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3182 ( 895) 1671 ( 895) 28.13 53.56 36.88
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 54) 59 ( 54) 31.03 91.53 46.35
Binding 398 ( 172) 328 ( 172) 43.22 52.44 47.38
Gene_expression 722 ( 361) 512 ( 361) 50.00 70.51 58.51
Transcription 137 ( 44) 110 ( 44) 32.12 40.00 35.63
Protein_catabolism 14 ( 4) 5 ( 4) 28.57 80.00 42.11
Phosphorylation 135 ( 58) 73 ( 58) 42.96 79.45 55.77
==[EVT-TOTAL]== 1580 ( 693) 1087 ( 693) 43.86 63.75 51.97
------------------------------------------------------------------------------------
Regulation 338 ( 38) 84 ( 38) 11.24 45.24 18.01
Positive_regulation 1186 ( 223) 583 ( 223) 18.80 38.25 25.21
Negative_regulation 416 ( 27) 90 ( 27) 6.49 30.00 10.67
==[REG-TOTAL]== 1940 ( 288) 757 ( 288) 14.85 38.04 21.36
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3520 ( 981) 1844 ( 981) 27.87 53.20 36.58
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 55) 58 ( 55) 31.61 94.83 47.41
Binding 398 ( 187) 304 ( 187) 46.98 61.51 53.28
Gene_expression 722 ( 372) 509 ( 372) 51.52 73.08 60.44
Transcription 137 ( 49) 104 ( 49) 35.77 47.12 40.66
Protein_catabolism 14 ( 4) 5 ( 4) 28.57 80.00 42.11
Phosphorylation 135 ( 60) 72 ( 60) 44.44 83.33 57.97
==[EVT-TOTAL]== 1580 ( 727) 1052 ( 727) 46.01 69.11 55.24
------------------------------------------------------------------------------------
Regulation 338 ( 40) 81 ( 40) 11.83 49.38 19.09
Positive_regulation 1186 ( 243) 571 ( 243) 20.49 42.56 27.66
Negative_regulation 416 ( 32) 89 ( 32) 7.69 35.96 12.67
==[REG-TOTAL]== 1940 ( 315) 741 ( 315) 16.24 42.51 23.50
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3520 ( 1042) 1793 ( 1042) 29.60 58.11 39.22
------------------------------------------------------------------------------------
Task 2
N/A
N/A
N/A
N/A
N/A
Task 3
N/A
N/A
N/A
N/A
N/A