BioNLP'09 Shared Task on
Event Extraction
in conjunction with
BioNLP, a NAACL-HLT 2009
workshop, June 4-5 2009, Boulder, Colorado
Final Evaluation Results
of Center for Computational Pharacology Biomedical Text Mining Group
Task 1
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 18) 18 ( 18) 10.34 100.00 18.75
Binding 347 ( 44) 110 ( 44) 12.68 40.00 19.26
Gene_expression 722 ( 259) 306 ( 259) 35.87 84.64 50.39
Transcription 137 ( 9) 20 ( 9) 6.57 45.00 11.46
Protein_catabolism 14 ( 4) 6 ( 4) 28.57 66.67 40.00
Phosphorylation 135 ( 30) 30 ( 30) 22.22 100.00 36.36
==[EVT-TOTAL]== 1529 ( 364) 490 ( 364) 23.81 74.29 36.06
------------------------------------------------------------------------------------
Regulation 291 ( 9) 19 ( 9) 3.09 47.37 5.81
Positive_regulation 983 ( 31) 65 ( 31) 3.15 47.69 5.92
Negative_regulation 379 ( 9) 22 ( 9) 2.37 40.91 4.49
==[REG-TOTAL]== 1653 ( 49) 106 ( 49) 2.96 46.23 5.57
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3182 ( 413) 596 ( 413) 12.98 69.30 21.86
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 18) 18 ( 18) 10.34 100.00 18.75
Binding 347 ( 44) 110 ( 44) 12.68 40.00 19.26
Gene_expression 722 ( 263) 306 ( 263) 36.43 85.95 51.17
Transcription 137 ( 18) 20 ( 18) 13.14 90.00 22.93
Protein_catabolism 14 ( 4) 6 ( 4) 28.57 66.67 40.00
Phosphorylation 135 ( 30) 30 ( 30) 22.22 100.00 36.36
==[EVT-TOTAL]== 1529 ( 377) 490 ( 377) 24.66 76.94 37.35
------------------------------------------------------------------------------------
Regulation 291 ( 9) 19 ( 9) 3.09 47.37 5.81
Positive_regulation 983 ( 32) 65 ( 32) 3.26 49.23 6.11
Negative_regulation 379 ( 10) 22 ( 10) 2.64 45.45 4.99
==[REG-TOTAL]== 1653 ( 51) 106 ( 51) 3.09 48.11 5.80
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3182 ( 428) 596 ( 428) 13.45 71.81 22.66
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 18) 18 ( 18) 10.34 100.00 18.75
Binding 347 ( 44) 110 ( 44) 12.68 40.00 19.26
Gene_expression 722 ( 263) 306 ( 263) 36.43 85.95 51.17
Transcription 137 ( 18) 20 ( 18) 13.14 90.00 22.93
Protein_catabolism 14 ( 4) 6 ( 4) 28.57 66.67 40.00
Phosphorylation 135 ( 30) 30 ( 30) 22.22 100.00 36.36
==[EVT-TOTAL]== 1529 ( 377) 490 ( 377) 24.66 76.94 37.35
------------------------------------------------------------------------------------
Regulation 291 ( 9) 19 ( 9) 3.09 47.37 5.81
Positive_regulation 983 ( 32) 65 ( 32) 3.26 49.23 6.11
Negative_regulation 379 ( 10) 22 ( 10) 2.64 45.45 4.99
==[REG-TOTAL]== 1653 ( 51) 106 ( 51) 3.09 48.11 5.80
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3182 ( 428) 596 ( 428) 13.45 71.81 22.66
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 18) 18 ( 18) 10.34 100.00 18.75
Binding 398 ( 70) 111 ( 70) 17.59 63.06 27.50
Gene_expression 722 ( 259) 306 ( 259) 35.87 84.64 50.39
Transcription 137 ( 9) 20 ( 9) 6.57 45.00 11.46
Protein_catabolism 14 ( 4) 6 ( 4) 28.57 66.67 40.00
Phosphorylation 135 ( 30) 30 ( 30) 22.22 100.00 36.36
==[EVT-TOTAL]== 1580 ( 390) 491 ( 390) 24.68 79.43 37.66
------------------------------------------------------------------------------------
Regulation 338 ( 9) 19 ( 9) 2.66 47.37 5.04
Positive_regulation 1186 ( 38) 74 ( 38) 3.20 51.35 6.03
Negative_regulation 416 ( 9) 22 ( 9) 2.16 40.91 4.11
==[REG-TOTAL]== 1940 ( 56) 115 ( 56) 2.89 48.70 5.45
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3520 ( 446) 606 ( 446) 12.67 73.60 21.62
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 18) 18 ( 18) 10.34 100.00 18.75
Binding 398 ( 70) 111 ( 70) 17.59 63.06 27.50
Gene_expression 722 ( 263) 306 ( 263) 36.43 85.95 51.17
Transcription 137 ( 18) 20 ( 18) 13.14 90.00 22.93
Protein_catabolism 14 ( 4) 6 ( 4) 28.57 66.67 40.00
Phosphorylation 135 ( 30) 30 ( 30) 22.22 100.00 36.36
==[EVT-TOTAL]== 1580 ( 403) 491 ( 403) 25.51 82.08 38.92
------------------------------------------------------------------------------------
Regulation 338 ( 9) 19 ( 9) 2.66 47.37 5.04
Positive_regulation 1186 ( 39) 74 ( 39) 3.29 52.70 6.19
Negative_regulation 416 ( 10) 22 ( 10) 2.40 45.45 4.57
==[REG-TOTAL]== 1940 ( 58) 115 ( 58) 2.99 50.43 5.64
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3520 ( 461) 606 ( 461) 13.10 76.07 22.35
------------------------------------------------------------------------------------
Task 2
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 17) 18 ( 17) 9.77 94.44 17.71
Binding 349 ( 43) 110 ( 43) 12.32 39.09 18.74
Gene_expression 722 ( 259) 306 ( 259) 35.87 84.64 50.39
Transcription 137 ( 9) 20 ( 9) 6.57 45.00 11.46
Protein_catabolism 14 ( 4) 6 ( 4) 28.57 66.67 40.00
Phosphorylation 139 ( 28) 30 ( 28) 20.14 93.33 33.14
==[EVT-TOTAL]== 1535 ( 360) 490 ( 360) 23.45 73.47 35.56
------------------------------------------------------------------------------------
Regulation 292 ( 9) 19 ( 9) 3.08 47.37 5.79
Positive_regulation 987 ( 31) 65 ( 31) 3.14 47.69 5.89
Negative_regulation 379 ( 8) 22 ( 8) 2.11 36.36 3.99
==[REG-TOTAL]== 1658 ( 48) 106 ( 48) 2.90 45.28 5.44
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3193 ( 408) 596 ( 408) 12.78 68.46 21.54
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 17) 18 ( 17) 9.77 94.44 17.71
Binding 349 ( 43) 110 ( 43) 12.32 39.09 18.74
Gene_expression 722 ( 263) 306 ( 263) 36.43 85.95 51.17
Transcription 137 ( 18) 20 ( 18) 13.14 90.00 22.93
Protein_catabolism 14 ( 4) 6 ( 4) 28.57 66.67 40.00
Phosphorylation 139 ( 28) 30 ( 28) 20.14 93.33 33.14
==[EVT-TOTAL]== 1535 ( 373) 490 ( 373) 24.30 76.12 36.84
------------------------------------------------------------------------------------
Regulation 292 ( 9) 19 ( 9) 3.08 47.37 5.79
Positive_regulation 987 ( 32) 65 ( 32) 3.24 49.23 6.08
Negative_regulation 379 ( 9) 22 ( 9) 2.37 40.91 4.49
==[REG-TOTAL]== 1658 ( 50) 106 ( 50) 3.02 47.17 5.67
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3193 ( 423) 596 ( 423) 13.25 70.97 22.33
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 17) 18 ( 17) 9.77 94.44 17.71
Binding 349 ( 43) 110 ( 43) 12.32 39.09 18.74
Gene_expression 722 ( 263) 306 ( 263) 36.43 85.95 51.17
Transcription 137 ( 18) 20 ( 18) 13.14 90.00 22.93
Protein_catabolism 14 ( 4) 6 ( 4) 28.57 66.67 40.00
Phosphorylation 139 ( 28) 30 ( 28) 20.14 93.33 33.14
==[EVT-TOTAL]== 1535 ( 373) 490 ( 373) 24.30 76.12 36.84
------------------------------------------------------------------------------------
Regulation 292 ( 9) 19 ( 9) 3.08 47.37 5.79
Positive_regulation 987 ( 32) 65 ( 32) 3.24 49.23 6.08
Negative_regulation 379 ( 9) 22 ( 9) 2.37 40.91 4.49
==[REG-TOTAL]== 1658 ( 50) 106 ( 50) 3.02 47.17 5.67
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3193 ( 423) 596 ( 423) 13.25 70.97 22.33
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 239 ( 18) 18 ( 18) 7.53 100.00 14.01
Binding 419 ( 70) 111 ( 70) 16.71 63.06 26.42
Gene_expression 722 ( 259) 306 ( 259) 35.87 84.64 50.39
Transcription 137 ( 9) 20 ( 9) 6.57 45.00 11.46
Protein_catabolism 14 ( 4) 6 ( 4) 28.57 66.67 40.00
Phosphorylation 191 ( 47) 47 ( 47) 24.61 100.00 39.50
==[EVT-TOTAL]== 1722 ( 407) 508 ( 407) 23.64 80.12 36.50
------------------------------------------------------------------------------------
Regulation 349 ( 9) 19 ( 9) 2.58 47.37 4.89
Positive_regulation 1220 ( 38) 74 ( 38) 3.11 51.35 5.87
Negative_regulation 425 ( 9) 22 ( 9) 2.12 40.91 4.03
==[REG-TOTAL]== 1994 ( 56) 115 ( 56) 2.81 48.70 5.31
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3716 ( 463) 623 ( 463) 12.46 74.32 21.34
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 239 ( 18) 18 ( 18) 7.53 100.00 14.01
Binding 419 ( 70) 111 ( 70) 16.71 63.06 26.42
Gene_expression 722 ( 263) 306 ( 263) 36.43 85.95 51.17
Transcription 137 ( 18) 20 ( 18) 13.14 90.00 22.93
Protein_catabolism 14 ( 4) 6 ( 4) 28.57 66.67 40.00
Phosphorylation 191 ( 47) 47 ( 47) 24.61 100.00 39.50
==[EVT-TOTAL]== 1722 ( 420) 508 ( 420) 24.39 82.68 37.67
------------------------------------------------------------------------------------
Regulation 349 ( 9) 19 ( 9) 2.58 47.37 4.89
Positive_regulation 1220 ( 39) 74 ( 39) 3.20 52.70 6.03
Negative_regulation 425 ( 10) 22 ( 10) 2.35 45.45 4.47
==[REG-TOTAL]== 1994 ( 58) 115 ( 58) 2.91 50.43 5.50
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3716 ( 478) 623 ( 478) 12.86 76.73 22.03
------------------------------------------------------------------------------------
Task 3
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 18) 18 ( 18) 10.34 100.00 18.75
Binding 347 ( 44) 110 ( 44) 12.68 40.00 19.26
Gene_expression 722 ( 259) 306 ( 259) 35.87 84.64 50.39
Transcription 137 ( 9) 20 ( 9) 6.57 45.00 11.46
Protein_catabolism 14 ( 4) 6 ( 4) 28.57 66.67 40.00
Phosphorylation 135 ( 30) 30 ( 30) 22.22 100.00 36.36
==[EVT-TOTAL]== 1529 ( 364) 490 ( 364) 23.81 74.29 36.06
------------------------------------------------------------------------------------
Regulation 291 ( 9) 19 ( 9) 3.09 47.37 5.81
Positive_regulation 983 ( 31) 65 ( 31) 3.15 47.69 5.92
Negative_regulation 379 ( 9) 22 ( 9) 2.37 40.91 4.49
==[REG-TOTAL]== 1653 ( 49) 106 ( 49) 2.96 46.23 5.57
------------------------------------------------------------------------------------
Negation 227 ( 4) 76 ( 4) 1.76 5.26 2.64
Speculation 208 ( 13) 105 ( 13) 6.25 12.38 8.31
==[MOD-TOTAL]== 435 ( 17) 181 ( 17) 3.91 9.39 5.52
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3617 ( 430) 777 ( 430) 11.89 55.34 19.57
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 18) 18 ( 18) 10.34 100.00 18.75
Binding 347 ( 44) 110 ( 44) 12.68 40.00 19.26
Gene_expression 722 ( 263) 306 ( 263) 36.43 85.95 51.17
Transcription 137 ( 18) 20 ( 18) 13.14 90.00 22.93
Protein_catabolism 14 ( 4) 6 ( 4) 28.57 66.67 40.00
Phosphorylation 135 ( 30) 30 ( 30) 22.22 100.00 36.36
==[EVT-TOTAL]== 1529 ( 377) 490 ( 377) 24.66 76.94 37.35
------------------------------------------------------------------------------------
Regulation 291 ( 9) 19 ( 9) 3.09 47.37 5.81
Positive_regulation 983 ( 32) 65 ( 32) 3.26 49.23 6.11
Negative_regulation 379 ( 10) 22 ( 10) 2.64 45.45 4.99
==[REG-TOTAL]== 1653 ( 51) 106 ( 51) 3.09 48.11 5.80
------------------------------------------------------------------------------------
Negation 227 ( 4) 76 ( 4) 1.76 5.26 2.64
Speculation 208 ( 13) 105 ( 13) 6.25 12.38 8.31
==[MOD-TOTAL]== 435 ( 17) 181 ( 17) 3.91 9.39 5.52
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3617 ( 445) 777 ( 445) 12.30 57.27 20.25
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 18) 18 ( 18) 10.34 100.00 18.75
Binding 347 ( 44) 110 ( 44) 12.68 40.00 19.26
Gene_expression 722 ( 263) 306 ( 263) 36.43 85.95 51.17
Transcription 137 ( 18) 20 ( 18) 13.14 90.00 22.93
Protein_catabolism 14 ( 4) 6 ( 4) 28.57 66.67 40.00
Phosphorylation 135 ( 30) 30 ( 30) 22.22 100.00 36.36
==[EVT-TOTAL]== 1529 ( 377) 490 ( 377) 24.66 76.94 37.35
------------------------------------------------------------------------------------
Regulation 291 ( 9) 19 ( 9) 3.09 47.37 5.81
Positive_regulation 983 ( 32) 65 ( 32) 3.26 49.23 6.11
Negative_regulation 379 ( 10) 22 ( 10) 2.64 45.45 4.99
==[REG-TOTAL]== 1653 ( 51) 106 ( 51) 3.09 48.11 5.80
------------------------------------------------------------------------------------
Negation 227 ( 4) 76 ( 4) 1.76 5.26 2.64
Speculation 208 ( 14) 105 ( 14) 6.73 13.33 8.95
==[MOD-TOTAL]== 435 ( 18) 181 ( 18) 4.14 9.94 5.84
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3617 ( 446) 777 ( 446) 12.33 57.40 20.30
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 18) 18 ( 18) 10.34 100.00 18.75
Binding 398 ( 70) 111 ( 70) 17.59 63.06 27.50
Gene_expression 722 ( 259) 306 ( 259) 35.87 84.64 50.39
Transcription 137 ( 9) 20 ( 9) 6.57 45.00 11.46
Protein_catabolism 14 ( 4) 6 ( 4) 28.57 66.67 40.00
Phosphorylation 135 ( 30) 30 ( 30) 22.22 100.00 36.36
==[EVT-TOTAL]== 1580 ( 390) 491 ( 390) 24.68 79.43 37.66
------------------------------------------------------------------------------------
Regulation 338 ( 9) 19 ( 9) 2.66 47.37 5.04
Positive_regulation 1186 ( 38) 74 ( 38) 3.20 51.35 6.03
Negative_regulation 416 ( 9) 22 ( 9) 2.16 40.91 4.11
==[REG-TOTAL]== 1940 ( 56) 115 ( 56) 2.89 48.70 5.45
------------------------------------------------------------------------------------
Negation 220 ( 5) 76 ( 5) 2.27 6.58 3.38
Speculation 201 ( 14) 105 ( 14) 6.97 13.33 9.15
==[MOD-TOTAL]== 421 ( 19) 181 ( 19) 4.51 10.50 6.31
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3941 ( 465) 787 ( 465) 11.80 59.09 19.67
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 18) 18 ( 18) 10.34 100.00 18.75
Binding 398 ( 70) 111 ( 70) 17.59 63.06 27.50
Gene_expression 722 ( 263) 306 ( 263) 36.43 85.95 51.17
Transcription 137 ( 18) 20 ( 18) 13.14 90.00 22.93
Protein_catabolism 14 ( 4) 6 ( 4) 28.57 66.67 40.00
Phosphorylation 135 ( 30) 30 ( 30) 22.22 100.00 36.36
==[EVT-TOTAL]== 1580 ( 403) 491 ( 403) 25.51 82.08 38.92
------------------------------------------------------------------------------------
Regulation 338 ( 9) 19 ( 9) 2.66 47.37 5.04
Positive_regulation 1186 ( 39) 74 ( 39) 3.29 52.70 6.19
Negative_regulation 416 ( 10) 22 ( 10) 2.40 45.45 4.57
==[REG-TOTAL]== 1940 ( 58) 115 ( 58) 2.99 50.43 5.64
------------------------------------------------------------------------------------
Negation 220 ( 5) 76 ( 5) 2.27 6.58 3.38
Speculation 201 ( 14) 105 ( 14) 6.97 13.33 9.15
==[MOD-TOTAL]== 421 ( 19) 181 ( 19) 4.51 10.50 6.31
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3941 ( 480) 787 ( 480) 12.18 60.99 20.30
------------------------------------------------------------------------------------