BioNLP'09 Shared Task on
Event Extraction
in conjunction with
BioNLP, a NAACL-HLT 2009
workshop, June 4-5 2009, Boulder, Colorado
Final Evaluation Results
of JULIE Lab, FSU Jena
Task 1
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 73) 98 ( 73) 41.95 74.49 53.68
Binding 347 ( 135) 489 ( 135) 38.90 27.61 32.30
Gene_expression 722 ( 434) 583 ( 434) 60.11 74.44 66.51
Transcription 137 ( 25) 79 ( 25) 18.25 31.65 23.15
Protein_catabolism 14 ( 11) 13 ( 11) 78.57 84.62 81.48
Phosphorylation 135 ( 97) 113 ( 97) 71.85 85.84 78.23
==[EVT-TOTAL]== 1529 ( 775) 1375 ( 775) 50.69 56.36 53.37
------------------------------------------------------------------------------------
Regulation 291 ( 72) 302 ( 72) 24.74 23.84 24.28
Positive_regulation 983 ( 282) 902 ( 282) 28.69 31.26 29.92
Negative_regulation 379 ( 140) 491 ( 140) 36.94 28.51 32.18
==[REG-TOTAL]== 1653 ( 494) 1695 ( 494) 29.89 29.14 29.51
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3182 ( 1269) 3070 ( 1269) 39.88 41.34 40.60
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 76) 98 ( 76) 43.68 77.55 55.88
Binding 347 ( 172) 488 ( 172) 49.57 35.25 41.20
Gene_expression 722 ( 468) 583 ( 468) 64.82 80.27 71.72
Transcription 137 ( 49) 79 ( 49) 35.77 62.03 45.37
Protein_catabolism 14 ( 11) 13 ( 11) 78.57 84.62 81.48
Phosphorylation 135 ( 103) 113 ( 103) 76.30 91.15 83.06
==[EVT-TOTAL]== 1529 ( 879) 1374 ( 879) 57.49 63.97 60.56
------------------------------------------------------------------------------------
Regulation 291 ( 90) 302 ( 90) 30.93 29.80 30.35
Positive_regulation 983 ( 335) 901 ( 335) 34.08 37.18 35.56
Negative_regulation 379 ( 151) 491 ( 151) 39.84 30.75 34.71
==[REG-TOTAL]== 1653 ( 576) 1694 ( 576) 34.85 34.00 34.42
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3182 ( 1455) 3068 ( 1455) 45.73 47.43 46.56
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 76) 98 ( 76) 43.68 77.55 55.88
Binding 347 ( 172) 488 ( 172) 49.57 35.25 41.20
Gene_expression 722 ( 468) 583 ( 468) 64.82 80.27 71.72
Transcription 137 ( 49) 79 ( 49) 35.77 62.03 45.37
Protein_catabolism 14 ( 11) 13 ( 11) 78.57 84.62 81.48
Phosphorylation 135 ( 103) 113 ( 103) 76.30 91.15 83.06
==[EVT-TOTAL]== 1529 ( 879) 1374 ( 879) 57.49 63.97 60.56
------------------------------------------------------------------------------------
Regulation 291 ( 91) 302 ( 91) 31.27 30.13 30.69
Positive_regulation 983 ( 335) 901 ( 335) 34.08 37.18 35.56
Negative_regulation 379 ( 153) 491 ( 153) 40.37 31.16 35.17
==[REG-TOTAL]== 1653 ( 579) 1694 ( 579) 35.03 34.18 34.60
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3182 ( 1458) 3068 ( 1458) 45.82 47.52 46.66
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 73) 98 ( 73) 41.95 74.49 53.68
Binding 398 ( 207) 467 ( 207) 52.01 44.33 47.86
Gene_expression 722 ( 434) 583 ( 434) 60.11 74.44 66.51
Transcription 137 ( 25) 79 ( 25) 18.25 31.65 23.15
Protein_catabolism 14 ( 11) 13 ( 11) 78.57 84.62 81.48
Phosphorylation 135 ( 97) 113 ( 97) 71.85 85.84 78.23
==[EVT-TOTAL]== 1580 ( 847) 1353 ( 847) 53.61 62.60 57.76
------------------------------------------------------------------------------------
Regulation 338 ( 95) 314 ( 95) 28.11 30.25 29.14
Positive_regulation 1186 ( 411) 965 ( 411) 34.65 42.59 38.21
Negative_regulation 416 ( 160) 445 ( 160) 38.46 35.96 37.17
==[REG-TOTAL]== 1940 ( 666) 1724 ( 666) 34.33 38.63 36.35
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3520 ( 1513) 3077 ( 1513) 42.98 49.17 45.87
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 76) 98 ( 76) 43.68 77.55 55.88
Binding 398 ( 253) 461 ( 253) 63.57 54.88 58.91
Gene_expression 722 ( 468) 583 ( 468) 64.82 80.27 71.72
Transcription 137 ( 49) 79 ( 49) 35.77 62.03 45.37
Protein_catabolism 14 ( 11) 13 ( 11) 78.57 84.62 81.48
Phosphorylation 135 ( 103) 113 ( 103) 76.30 91.15 83.06
==[EVT-TOTAL]== 1580 ( 960) 1347 ( 960) 60.76 71.27 65.60
------------------------------------------------------------------------------------
Regulation 338 ( 119) 312 ( 119) 35.21 38.14 36.62
Positive_regulation 1186 ( 482) 964 ( 482) 40.64 50.00 44.84
Negative_regulation 416 ( 176) 445 ( 176) 42.31 39.55 40.88
==[REG-TOTAL]== 1940 ( 777) 1721 ( 777) 40.05 45.15 42.45
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3520 ( 1737) 3068 ( 1737) 49.35 56.62 52.73
------------------------------------------------------------------------------------
Task 2
N/A
N/A
N/A
N/A
N/A
Task 3
N/A
N/A
N/A
N/A
N/A