BioNLP'09 Shared Task on
Event Extraction
in conjunction with
BioNLP, a NAACL-HLT 2009
workshop, June 4-5 2009, Boulder, Colorado
Final Evaluation Results
of CIPS Lab, CSE Dept, Arizona State University
Task 1
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 48) 143 ( 48) 27.59 33.57 30.28
Binding 347 ( 58) 190 ( 58) 16.71 30.53 21.60
Gene_expression 722 ( 318) 804 ( 318) 44.04 39.55 41.68
Transcription 137 ( 15) 133 ( 15) 10.95 11.28 11.11
Protein_catabolism 14 ( 8) 29 ( 8) 57.14 27.59 37.21
Phosphorylation 135 ( 68) 107 ( 68) 50.37 63.55 56.20
==[EVT-TOTAL]== 1529 ( 515) 1406 ( 515) 33.68 36.63 35.09
------------------------------------------------------------------------------------
Regulation 291 ( 27) 521 ( 27) 9.28 5.18 6.65
Positive_regulation 983 ( 103) 1403 ( 103) 10.48 7.34 8.63
Negative_regulation 379 ( 49) 481 ( 49) 12.93 10.19 11.40
==[REG-TOTAL]== 1653 ( 179) 2405 ( 179) 10.83 7.44 8.82
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3182 ( 694) 3811 ( 694) 21.81 18.21 19.85
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 49) 143 ( 49) 28.16 34.27 30.91
Binding 347 ( 60) 190 ( 60) 17.29 31.58 22.35
Gene_expression 722 ( 323) 803 ( 323) 44.74 40.22 42.36
Transcription 137 ( 17) 133 ( 17) 12.41 12.78 12.59
Protein_catabolism 14 ( 8) 29 ( 8) 57.14 27.59 37.21
Phosphorylation 135 ( 72) 107 ( 72) 53.33 67.29 59.50
==[EVT-TOTAL]== 1529 ( 529) 1405 ( 529) 34.60 37.65 36.06
------------------------------------------------------------------------------------
Regulation 291 ( 33) 521 ( 33) 11.34 6.33 8.13
Positive_regulation 983 ( 113) 1402 ( 113) 11.50 8.06 9.48
Negative_regulation 379 ( 50) 481 ( 50) 13.19 10.40 11.63
==[REG-TOTAL]== 1653 ( 196) 2404 ( 196) 11.86 8.15 9.66
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3182 ( 725) 3809 ( 725) 22.78 19.03 20.74
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 49) 143 ( 49) 28.16 34.27 30.91
Binding 347 ( 60) 190 ( 60) 17.29 31.58 22.35
Gene_expression 722 ( 323) 803 ( 323) 44.74 40.22 42.36
Transcription 137 ( 17) 133 ( 17) 12.41 12.78 12.59
Protein_catabolism 14 ( 8) 29 ( 8) 57.14 27.59 37.21
Phosphorylation 135 ( 72) 107 ( 72) 53.33 67.29 59.50
==[EVT-TOTAL]== 1529 ( 529) 1405 ( 529) 34.60 37.65 36.06
------------------------------------------------------------------------------------
Regulation 291 ( 33) 521 ( 33) 11.34 6.33 8.13
Positive_regulation 983 ( 113) 1402 ( 113) 11.50 8.06 9.48
Negative_regulation 379 ( 50) 481 ( 50) 13.19 10.40 11.63
==[REG-TOTAL]== 1653 ( 196) 2404 ( 196) 11.86 8.15 9.66
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3182 ( 725) 3809 ( 725) 22.78 19.03 20.74
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 48) 143 ( 48) 27.59 33.57 30.28
Binding 398 ( 86) 206 ( 86) 21.61 41.75 28.48
Gene_expression 722 ( 318) 804 ( 318) 44.04 39.55 41.68
Transcription 137 ( 15) 133 ( 15) 10.95 11.28 11.11
Protein_catabolism 14 ( 8) 29 ( 8) 57.14 27.59 37.21
Phosphorylation 135 ( 68) 107 ( 68) 50.37 63.55 56.20
==[EVT-TOTAL]== 1580 ( 543) 1422 ( 543) 34.37 38.19 36.18
------------------------------------------------------------------------------------
Regulation 338 ( 29) 521 ( 29) 8.58 5.57 6.75
Positive_regulation 1186 ( 125) 1403 ( 125) 10.54 8.91 9.66
Negative_regulation 416 ( 50) 481 ( 50) 12.02 10.40 11.15
==[REG-TOTAL]== 1940 ( 204) 2405 ( 204) 10.52 8.48 9.39
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3520 ( 747) 3827 ( 747) 21.22 19.52 20.33
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 49) 143 ( 49) 28.16 34.27 30.91
Binding 398 ( 91) 206 ( 91) 22.86 44.17 30.13
Gene_expression 722 ( 323) 803 ( 323) 44.74 40.22 42.36
Transcription 137 ( 17) 133 ( 17) 12.41 12.78 12.59
Protein_catabolism 14 ( 8) 29 ( 8) 57.14 27.59 37.21
Phosphorylation 135 ( 72) 107 ( 72) 53.33 67.29 59.50
==[EVT-TOTAL]== 1580 ( 560) 1421 ( 560) 35.44 39.41 37.32
------------------------------------------------------------------------------------
Regulation 338 ( 35) 521 ( 35) 10.36 6.72 8.15
Positive_regulation 1186 ( 137) 1402 ( 137) 11.55 9.77 10.59
Negative_regulation 416 ( 52) 481 ( 52) 12.50 10.81 11.59
==[REG-TOTAL]== 1940 ( 224) 2404 ( 224) 11.55 9.32 10.31
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3520 ( 784) 3825 ( 784) 22.27 20.50 21.35
------------------------------------------------------------------------------------
Task 2
N/A
N/A
N/A
N/A
N/A
Task 3
N/A
N/A
N/A
N/A
N/A