BioNLP'09 Shared Task on
Event Extraction
in conjunction with
BioNLP, a NAACL-HLT 2009
workshop, June 4-5 2009, Boulder, Colorado
Final Evaluation Results
of University of Turku
Task 1
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 85) 105 ( 85) 48.85 80.95 60.93
Binding 347 ( 120) 279 ( 120) 34.58 43.01 38.34
Gene_expression 722 ( 495) 642 ( 495) 68.56 77.10 72.58
Transcription 137 ( 50) 78 ( 50) 36.50 64.10 46.51
Protein_catabolism 14 ( 6) 9 ( 6) 42.86 66.67 52.17
Phosphorylation 135 ( 103) 146 ( 103) 76.30 70.55 73.31
==[EVT-TOTAL]== 1529 ( 859) 1259 ( 859) 56.18 68.23 61.62
------------------------------------------------------------------------------------
Regulation 291 ( 67) 194 ( 67) 23.02 34.54 27.63
Positive_regulation 983 ( 314) 783 ( 314) 31.94 40.10 35.56
Negative_regulation 379 ( 117) 306 ( 117) 30.87 38.24 34.16
==[REG-TOTAL]== 1653 ( 498) 1283 ( 498) 30.13 38.82 33.92
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3182 ( 1357) 2542 ( 1357) 42.65 53.38 47.41
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 86) 105 ( 86) 49.43 81.90 61.65
Binding 347 ( 139) 279 ( 139) 40.06 49.82 44.41
Gene_expression 722 ( 504) 642 ( 504) 69.81 78.50 73.90
Transcription 137 ( 54) 78 ( 54) 39.42 69.23 50.23
Protein_catabolism 14 ( 6) 9 ( 6) 42.86 66.67 52.17
Phosphorylation 135 ( 109) 146 ( 109) 80.74 74.66 77.58
==[EVT-TOTAL]== 1529 ( 898) 1259 ( 898) 58.73 71.33 64.42
------------------------------------------------------------------------------------
Regulation 291 ( 74) 194 ( 74) 25.43 38.14 30.52
Positive_regulation 983 ( 378) 782 ( 378) 38.45 48.34 42.83
Negative_regulation 379 ( 130) 306 ( 130) 34.30 42.48 37.96
==[REG-TOTAL]== 1653 ( 582) 1282 ( 582) 35.21 45.40 39.66
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3182 ( 1480) 2541 ( 1480) 46.51 58.24 51.72
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 86) 105 ( 86) 49.43 81.90 61.65
Binding 347 ( 139) 279 ( 139) 40.06 49.82 44.41
Gene_expression 722 ( 504) 642 ( 504) 69.81 78.50 73.90
Transcription 137 ( 54) 78 ( 54) 39.42 69.23 50.23
Protein_catabolism 14 ( 6) 9 ( 6) 42.86 66.67 52.17
Phosphorylation 135 ( 109) 146 ( 109) 80.74 74.66 77.58
==[EVT-TOTAL]== 1529 ( 898) 1259 ( 898) 58.73 71.33 64.42
------------------------------------------------------------------------------------
Regulation 291 ( 74) 194 ( 74) 25.43 38.14 30.52
Positive_regulation 983 ( 381) 782 ( 381) 38.76 48.72 43.17
Negative_regulation 379 ( 134) 306 ( 133) 35.36 43.46 38.99
==[REG-TOTAL]== 1653 ( 589) 1282 ( 588) 35.63 45.87 40.11
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3182 ( 1487) 2541 ( 1486) 46.73 58.48 51.95
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 85) 105 ( 85) 48.85 80.95 60.93
Binding 398 ( 186) 328 ( 186) 46.73 56.71 51.24
Gene_expression 722 ( 495) 642 ( 495) 68.56 77.10 72.58
Transcription 137 ( 50) 78 ( 50) 36.50 64.10 46.51
Protein_catabolism 14 ( 6) 9 ( 6) 42.86 66.67 52.17
Phosphorylation 135 ( 103) 146 ( 103) 76.30 70.55 73.31
==[EVT-TOTAL]== 1580 ( 925) 1308 ( 925) 58.54 70.72 64.06
------------------------------------------------------------------------------------
Regulation 338 ( 87) 215 ( 87) 25.74 40.47 31.46
Positive_regulation 1186 ( 447) 884 ( 447) 37.69 50.57 43.19
Negative_regulation 416 ( 138) 310 ( 138) 33.17 44.52 38.02
==[REG-TOTAL]== 1940 ( 672) 1409 ( 672) 34.64 47.69 40.13
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3520 ( 1597) 2717 ( 1597) 45.37 58.78 51.21
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 86) 105 ( 86) 49.43 81.90 61.65
Binding 398 ( 212) 328 ( 212) 53.27 64.63 58.40
Gene_expression 722 ( 504) 642 ( 504) 69.81 78.50 73.90
Transcription 137 ( 54) 78 ( 54) 39.42 69.23 50.23
Protein_catabolism 14 ( 6) 9 ( 6) 42.86 66.67 52.17
Phosphorylation 135 ( 109) 146 ( 109) 80.74 74.66 77.58
==[EVT-TOTAL]== 1580 ( 971) 1308 ( 971) 61.46 74.24 67.24
------------------------------------------------------------------------------------
Regulation 338 ( 95) 215 ( 95) 28.11 44.19 34.36
Positive_regulation 1186 ( 529) 883 ( 529) 44.60 59.91 51.14
Negative_regulation 416 ( 160) 310 ( 160) 38.46 51.61 44.08
==[REG-TOTAL]== 1940 ( 784) 1408 ( 784) 40.41 55.68 46.83
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3520 ( 1755) 2716 ( 1755) 49.86 64.62 56.29
------------------------------------------------------------------------------------
Task 2
N/A
N/A
N/A
N/A
N/A
Task 3
N/A
N/A
N/A
N/A
N/A