BioNLP'09 Shared Task on
Event Extraction
in conjunction with
BioNLP, a NAACL-HLT 2009
workshop, June 4-5 2009, Boulder, Colorado
Final Evaluation Results
of University of Tokyo + Database Center for Life Sciences
Task 1
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 65) 75 ( 65) 37.36 86.67 52.21
Binding 347 ( 74) 166 ( 74) 21.33 44.58 28.85
Gene_expression 722 ( 442) 606 ( 442) 61.22 72.94 66.57
Transcription 137 ( 20) 77 ( 20) 14.60 25.97 18.69
Protein_catabolism 14 ( 9) 11 ( 9) 64.29 81.82 72.00
Phosphorylation 135 ( 103) 137 ( 103) 76.30 75.18 75.74
==[EVT-TOTAL]== 1529 ( 713) 1072 ( 713) 46.63 66.51 54.83
------------------------------------------------------------------------------------
Regulation 291 ( 65) 169 ( 65) 22.34 38.46 28.26
Positive_regulation 983 ( 220) 615 ( 220) 22.38 35.77 27.53
Negative_regulation 379 ( 92) 254 ( 92) 24.27 36.22 29.07
==[REG-TOTAL]== 1653 ( 377) 1038 ( 377) 22.81 36.32 28.02
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3182 ( 1090) 2110 ( 1090) 34.26 51.66 41.19
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 66) 75 ( 66) 37.93 88.00 53.01
Binding 347 ( 80) 166 ( 80) 23.05 48.19 31.19
Gene_expression 722 ( 455) 606 ( 455) 63.02 75.08 68.52
Transcription 137 ( 23) 77 ( 23) 16.79 29.87 21.50
Protein_catabolism 14 ( 9) 11 ( 9) 64.29 81.82 72.00
Phosphorylation 135 ( 106) 137 ( 106) 78.52 77.37 77.94
==[EVT-TOTAL]== 1529 ( 739) 1072 ( 739) 48.33 68.94 56.82
------------------------------------------------------------------------------------
Regulation 291 ( 69) 169 ( 69) 23.71 40.83 30.00
Positive_regulation 983 ( 260) 615 ( 260) 26.45 42.28 32.54
Negative_regulation 379 ( 100) 254 ( 100) 26.39 39.37 31.60
==[REG-TOTAL]== 1653 ( 429) 1038 ( 429) 25.95 41.33 31.88
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3182 ( 1168) 2110 ( 1168) 36.71 55.36 44.14
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 66) 75 ( 66) 37.93 88.00 53.01
Binding 347 ( 80) 166 ( 80) 23.05 48.19 31.19
Gene_expression 722 ( 455) 606 ( 455) 63.02 75.08 68.52
Transcription 137 ( 23) 77 ( 23) 16.79 29.87 21.50
Protein_catabolism 14 ( 9) 11 ( 9) 64.29 81.82 72.00
Phosphorylation 135 ( 106) 137 ( 106) 78.52 77.37 77.94
==[EVT-TOTAL]== 1529 ( 739) 1072 ( 739) 48.33 68.94 56.82
------------------------------------------------------------------------------------
Regulation 291 ( 69) 169 ( 69) 23.71 40.83 30.00
Positive_regulation 983 ( 263) 615 ( 263) 26.75 42.76 32.92
Negative_regulation 379 ( 103) 254 ( 102) 27.18 40.16 32.42
==[REG-TOTAL]== 1653 ( 435) 1038 ( 434) 26.32 41.81 32.30
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3182 ( 1174) 2110 ( 1173) 36.90 55.59 44.35
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 65) 75 ( 65) 37.36 86.67 52.21
Binding 398 ( 147) 237 ( 147) 36.93 62.03 46.30
Gene_expression 722 ( 442) 606 ( 442) 61.22 72.94 66.57
Transcription 137 ( 20) 77 ( 20) 14.60 25.97 18.69
Protein_catabolism 14 ( 9) 11 ( 9) 64.29 81.82 72.00
Phosphorylation 135 ( 103) 137 ( 103) 76.30 75.18 75.74
==[EVT-TOTAL]== 1580 ( 786) 1143 ( 786) 49.75 68.77 57.73
------------------------------------------------------------------------------------
Regulation 338 ( 86) 198 ( 86) 25.44 43.43 32.09
Positive_regulation 1186 ( 324) 697 ( 324) 27.32 46.48 34.41
Negative_regulation 416 ( 111) 273 ( 111) 26.68 40.66 32.22
==[REG-TOTAL]== 1940 ( 521) 1168 ( 521) 26.86 44.61 33.53
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3520 ( 1307) 2311 ( 1307) 37.13 56.56 44.83
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 66) 75 ( 66) 37.93 88.00 53.01
Binding 398 ( 158) 237 ( 158) 39.70 66.67 49.76
Gene_expression 722 ( 455) 606 ( 455) 63.02 75.08 68.52
Transcription 137 ( 23) 77 ( 23) 16.79 29.87 21.50
Protein_catabolism 14 ( 9) 11 ( 9) 64.29 81.82 72.00
Phosphorylation 135 ( 106) 137 ( 106) 78.52 77.37 77.94
==[EVT-TOTAL]== 1580 ( 817) 1143 ( 817) 51.71 71.48 60.01
------------------------------------------------------------------------------------
Regulation 338 ( 91) 198 ( 91) 26.92 45.96 33.96
Positive_regulation 1186 ( 388) 697 ( 388) 32.72 55.67 41.21
Negative_regulation 416 ( 128) 273 ( 128) 30.77 46.89 37.16
==[REG-TOTAL]== 1940 ( 607) 1168 ( 607) 31.29 51.97 39.06
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3520 ( 1424) 2311 ( 1424) 40.45 61.62 48.84
------------------------------------------------------------------------------------
Task 2
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 56) 75 ( 56) 32.18 74.67 44.98
Binding 349 ( 72) 166 ( 72) 20.63 43.37 27.96
Gene_expression 722 ( 442) 606 ( 442) 61.22 72.94 66.57
Transcription 137 ( 20) 77 ( 20) 14.60 25.97 18.69
Protein_catabolism 14 ( 9) 11 ( 9) 64.29 81.82 72.00
Phosphorylation 139 ( 93) 137 ( 93) 66.91 67.88 67.39
==[EVT-TOTAL]== 1535 ( 692) 1072 ( 692) 45.08 64.55 53.09
------------------------------------------------------------------------------------
Regulation 292 ( 62) 169 ( 62) 21.23 36.69 26.90
Positive_regulation 987 ( 217) 615 ( 217) 21.99 35.28 27.09
Negative_regulation 379 ( 88) 254 ( 88) 23.22 34.65 27.80
==[REG-TOTAL]== 1658 ( 367) 1038 ( 367) 22.14 35.36 27.23
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3193 ( 1059) 2110 ( 1059) 33.17 50.19 39.94
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 57) 75 ( 57) 32.76 76.00 45.78
Binding 349 ( 78) 166 ( 78) 22.35 46.99 30.29
Gene_expression 722 ( 455) 606 ( 455) 63.02 75.08 68.52
Transcription 137 ( 23) 77 ( 23) 16.79 29.87 21.50
Protein_catabolism 14 ( 9) 11 ( 9) 64.29 81.82 72.00
Phosphorylation 139 ( 96) 137 ( 96) 69.06 70.07 69.57
==[EVT-TOTAL]== 1535 ( 718) 1072 ( 718) 46.78 66.98 55.08
------------------------------------------------------------------------------------
Regulation 292 ( 65) 169 ( 65) 22.26 38.46 28.20
Positive_regulation 987 ( 257) 615 ( 257) 26.04 41.79 32.08
Negative_regulation 379 ( 96) 254 ( 96) 25.33 37.80 30.33
==[REG-TOTAL]== 1658 ( 418) 1038 ( 418) 25.21 40.27 31.01
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3193 ( 1136) 2110 ( 1136) 35.58 53.84 42.84
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 57) 75 ( 57) 32.76 76.00 45.78
Binding 349 ( 78) 166 ( 78) 22.35 46.99 30.29
Gene_expression 722 ( 455) 606 ( 455) 63.02 75.08 68.52
Transcription 137 ( 23) 77 ( 23) 16.79 29.87 21.50
Protein_catabolism 14 ( 9) 11 ( 9) 64.29 81.82 72.00
Phosphorylation 139 ( 96) 137 ( 96) 69.06 70.07 69.57
==[EVT-TOTAL]== 1535 ( 718) 1072 ( 718) 46.78 66.98 55.08
------------------------------------------------------------------------------------
Regulation 292 ( 65) 169 ( 65) 22.26 38.46 28.20
Positive_regulation 987 ( 263) 615 ( 260) 26.65 42.28 32.69
Negative_regulation 379 ( 99) 254 ( 98) 26.12 38.58 31.15
==[REG-TOTAL]== 1658 ( 427) 1038 ( 423) 25.75 40.75 31.56
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3193 ( 1145) 2110 ( 1141) 35.86 54.08 43.12
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 239 ( 80) 92 ( 80) 33.47 86.96 48.34
Binding 419 ( 148) 239 ( 148) 35.32 61.92 44.98
Gene_expression 722 ( 442) 606 ( 442) 61.22 72.94 66.57
Transcription 137 ( 20) 77 ( 20) 14.60 25.97 18.69
Protein_catabolism 14 ( 9) 11 ( 9) 64.29 81.82 72.00
Phosphorylation 191 ( 143) 193 ( 143) 74.87 74.09 74.48
==[EVT-TOTAL]== 1722 ( 842) 1218 ( 842) 48.90 69.13 57.28
------------------------------------------------------------------------------------
Regulation 349 ( 87) 201 ( 87) 24.93 43.28 31.64
Positive_regulation 1220 ( 329) 705 ( 329) 26.97 46.67 34.18
Negative_regulation 425 ( 111) 274 ( 111) 26.12 40.51 31.76
==[REG-TOTAL]== 1994 ( 527) 1180 ( 527) 26.43 44.66 33.21
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3716 ( 1369) 2398 ( 1369) 36.84 57.09 44.78
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 239 ( 81) 92 ( 81) 33.89 88.04 48.94
Binding 419 ( 159) 239 ( 159) 37.95 66.53 48.33
Gene_expression 722 ( 455) 606 ( 455) 63.02 75.08 68.52
Transcription 137 ( 23) 77 ( 23) 16.79 29.87 21.50
Protein_catabolism 14 ( 9) 11 ( 9) 64.29 81.82 72.00
Phosphorylation 191 ( 146) 193 ( 146) 76.44 75.65 76.04
==[EVT-TOTAL]== 1722 ( 873) 1218 ( 873) 50.70 71.67 59.39
------------------------------------------------------------------------------------
Regulation 349 ( 92) 201 ( 92) 26.36 45.77 33.45
Positive_regulation 1220 ( 394) 705 ( 394) 32.30 55.89 40.94
Negative_regulation 425 ( 128) 274 ( 128) 30.12 46.72 36.62
==[REG-TOTAL]== 1994 ( 614) 1180 ( 614) 30.79 52.03 38.69
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3716 ( 1487) 2398 ( 1487) 40.02 62.01 48.64
------------------------------------------------------------------------------------
Task 3
N/A
N/A
N/A
N/A
N/A