BioNLP'09 Shared Task on
Event Extraction
in conjunction with
BioNLP, a NAACL-HLT 2009
workshop, June 4-5 2009, Boulder, Colorado
Final Evaluation Results
of University of Szeged
Task 1
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 13) 33 ( 13) 7.47 39.39 12.56
Binding 347 ( 53) 206 ( 53) 15.27 25.73 19.17
Gene_expression 722 ( 401) 793 ( 401) 55.54 50.57 52.94
Transcription 137 ( 25) 150 ( 25) 18.25 16.67 17.42
Protein_catabolism 14 ( 6) 17 ( 6) 42.86 35.29 38.71
Phosphorylation 135 ( 106) 274 ( 106) 78.52 38.69 51.83
==[EVT-TOTAL]== 1529 ( 604) 1473 ( 604) 39.50 41.00 40.24
------------------------------------------------------------------------------------
Regulation 291 ( 11) 65 ( 11) 3.78 16.92 6.18
Positive_regulation 983 ( 52) 314 ( 52) 5.29 16.56 8.02
Negative_regulation 379 ( 0) 0 ( 0) 0.00 0.00 0.00
==[REG-TOTAL]== 1653 ( 63) 379 ( 63) 3.81 16.62 6.20
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3182 ( 667) 1852 ( 667) 20.96 36.02 26.50
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 14) 33 ( 14) 8.05 42.42 13.53
Binding 347 ( 53) 206 ( 53) 15.27 25.73 19.17
Gene_expression 722 ( 406) 793 ( 406) 56.23 51.20 53.60
Transcription 137 ( 28) 150 ( 28) 20.44 18.67 19.51
Protein_catabolism 14 ( 6) 17 ( 6) 42.86 35.29 38.71
Phosphorylation 135 ( 109) 274 ( 109) 80.74 39.78 53.30
==[EVT-TOTAL]== 1529 ( 616) 1473 ( 616) 40.29 41.82 41.04
------------------------------------------------------------------------------------
Regulation 291 ( 15) 65 ( 15) 5.15 23.08 8.43
Positive_regulation 983 ( 54) 314 ( 54) 5.49 17.20 8.33
Negative_regulation 379 ( 0) 0 ( 0) 0.00 0.00 0.00
==[REG-TOTAL]== 1653 ( 69) 379 ( 69) 4.17 18.21 6.79
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3182 ( 685) 1852 ( 685) 21.53 36.99 27.21
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 14) 33 ( 14) 8.05 42.42 13.53
Binding 347 ( 53) 206 ( 53) 15.27 25.73 19.17
Gene_expression 722 ( 406) 793 ( 406) 56.23 51.20 53.60
Transcription 137 ( 28) 150 ( 28) 20.44 18.67 19.51
Protein_catabolism 14 ( 6) 17 ( 6) 42.86 35.29 38.71
Phosphorylation 135 ( 109) 274 ( 109) 80.74 39.78 53.30
==[EVT-TOTAL]== 1529 ( 616) 1473 ( 616) 40.29 41.82 41.04
------------------------------------------------------------------------------------
Regulation 291 ( 15) 65 ( 15) 5.15 23.08 8.43
Positive_regulation 983 ( 54) 314 ( 54) 5.49 17.20 8.33
Negative_regulation 379 ( 0) 0 ( 0) 0.00 0.00 0.00
==[REG-TOTAL]== 1653 ( 69) 379 ( 69) 4.17 18.21 6.79
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3182 ( 685) 1852 ( 685) 21.53 36.99 27.21
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 13) 33 ( 13) 7.47 39.39 12.56
Binding 398 ( 90) 206 ( 90) 22.61 43.69 29.80
Gene_expression 722 ( 401) 793 ( 401) 55.54 50.57 52.94
Transcription 137 ( 25) 150 ( 25) 18.25 16.67 17.42
Protein_catabolism 14 ( 6) 17 ( 6) 42.86 35.29 38.71
Phosphorylation 135 ( 106) 274 ( 106) 78.52 38.69 51.83
==[EVT-TOTAL]== 1580 ( 641) 1473 ( 641) 40.57 43.52 41.99
------------------------------------------------------------------------------------
Regulation 338 ( 11) 65 ( 11) 3.25 16.92 5.46
Positive_regulation 1186 ( 74) 314 ( 74) 6.24 23.57 9.87
Negative_regulation 416 ( 0) 0 ( 0) 0.00 0.00 0.00
==[REG-TOTAL]== 1940 ( 85) 379 ( 85) 4.38 22.43 7.33
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3520 ( 726) 1852 ( 726) 20.62 39.20 27.03
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 14) 33 ( 14) 8.05 42.42 13.53
Binding 398 ( 90) 206 ( 90) 22.61 43.69 29.80
Gene_expression 722 ( 406) 793 ( 406) 56.23 51.20 53.60
Transcription 137 ( 28) 150 ( 28) 20.44 18.67 19.51
Protein_catabolism 14 ( 6) 17 ( 6) 42.86 35.29 38.71
Phosphorylation 135 ( 109) 274 ( 109) 80.74 39.78 53.30
==[EVT-TOTAL]== 1580 ( 653) 1473 ( 653) 41.33 44.33 42.78
------------------------------------------------------------------------------------
Regulation 338 ( 15) 65 ( 15) 4.44 23.08 7.44
Positive_regulation 1186 ( 79) 314 ( 79) 6.66 25.16 10.53
Negative_regulation 416 ( 0) 0 ( 0) 0.00 0.00 0.00
==[REG-TOTAL]== 1940 ( 94) 379 ( 94) 4.85 24.80 8.11
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3520 ( 747) 1852 ( 747) 21.22 40.33 27.81
------------------------------------------------------------------------------------
Task 2
N/A
N/A
N/A
N/A
N/A
Task 3
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 13) 33 ( 13) 7.47 39.39 12.56
Binding 347 ( 53) 206 ( 53) 15.27 25.73 19.17
Gene_expression 722 ( 401) 793 ( 401) 55.54 50.57 52.94
Transcription 137 ( 25) 150 ( 25) 18.25 16.67 17.42
Protein_catabolism 14 ( 6) 17 ( 6) 42.86 35.29 38.71
Phosphorylation 135 ( 106) 274 ( 106) 78.52 38.69 51.83
==[EVT-TOTAL]== 1529 ( 604) 1473 ( 604) 39.50 41.00 40.24
------------------------------------------------------------------------------------
Regulation 291 ( 11) 65 ( 11) 3.78 16.92 6.18
Positive_regulation 983 ( 52) 314 ( 52) 5.29 16.56 8.02
Negative_regulation 379 ( 0) 0 ( 0) 0.00 0.00 0.00
==[REG-TOTAL]== 1653 ( 63) 379 ( 63) 3.81 16.62 6.20
------------------------------------------------------------------------------------
Negation 227 ( 9) 617 ( 9) 3.96 1.46 2.13
Speculation 208 ( 25) 644 ( 25) 12.02 3.88 5.87
==[MOD-TOTAL]== 435 ( 34) 1261 ( 34) 7.82 2.70 4.01
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3617 ( 701) 3113 ( 701) 19.38 22.52 20.83
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 14) 33 ( 14) 8.05 42.42 13.53
Binding 347 ( 53) 206 ( 53) 15.27 25.73 19.17
Gene_expression 722 ( 406) 793 ( 406) 56.23 51.20 53.60
Transcription 137 ( 28) 150 ( 28) 20.44 18.67 19.51
Protein_catabolism 14 ( 6) 17 ( 6) 42.86 35.29 38.71
Phosphorylation 135 ( 109) 274 ( 109) 80.74 39.78 53.30
==[EVT-TOTAL]== 1529 ( 616) 1473 ( 616) 40.29 41.82 41.04
------------------------------------------------------------------------------------
Regulation 291 ( 15) 65 ( 15) 5.15 23.08 8.43
Positive_regulation 983 ( 54) 314 ( 54) 5.49 17.20 8.33
Negative_regulation 379 ( 0) 0 ( 0) 0.00 0.00 0.00
==[REG-TOTAL]== 1653 ( 69) 379 ( 69) 4.17 18.21 6.79
------------------------------------------------------------------------------------
Negation 227 ( 10) 617 ( 10) 4.41 1.62 2.37
Speculation 208 ( 25) 644 ( 25) 12.02 3.88 5.87
==[MOD-TOTAL]== 435 ( 35) 1261 ( 35) 8.05 2.78 4.13
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3617 ( 720) 3113 ( 720) 19.91 23.13 21.40
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 14) 33 ( 14) 8.05 42.42 13.53
Binding 347 ( 53) 206 ( 53) 15.27 25.73 19.17
Gene_expression 722 ( 406) 793 ( 406) 56.23 51.20 53.60
Transcription 137 ( 28) 150 ( 28) 20.44 18.67 19.51
Protein_catabolism 14 ( 6) 17 ( 6) 42.86 35.29 38.71
Phosphorylation 135 ( 109) 274 ( 109) 80.74 39.78 53.30
==[EVT-TOTAL]== 1529 ( 616) 1473 ( 616) 40.29 41.82 41.04
------------------------------------------------------------------------------------
Regulation 291 ( 15) 65 ( 15) 5.15 23.08 8.43
Positive_regulation 983 ( 54) 314 ( 54) 5.49 17.20 8.33
Negative_regulation 379 ( 0) 0 ( 0) 0.00 0.00 0.00
==[REG-TOTAL]== 1653 ( 69) 379 ( 69) 4.17 18.21 6.79
------------------------------------------------------------------------------------
Negation 227 ( 12) 617 ( 12) 5.29 1.94 2.84
Speculation 208 ( 25) 644 ( 25) 12.02 3.88 5.87
==[MOD-TOTAL]== 435 ( 37) 1261 ( 37) 8.51 2.93 4.36
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3617 ( 722) 3113 ( 722) 19.96 23.19 21.46
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 13) 33 ( 13) 7.47 39.39 12.56
Binding 398 ( 90) 206 ( 90) 22.61 43.69 29.80
Gene_expression 722 ( 401) 793 ( 401) 55.54 50.57 52.94
Transcription 137 ( 25) 150 ( 25) 18.25 16.67 17.42
Protein_catabolism 14 ( 6) 17 ( 6) 42.86 35.29 38.71
Phosphorylation 135 ( 106) 274 ( 106) 78.52 38.69 51.83
==[EVT-TOTAL]== 1580 ( 641) 1473 ( 641) 40.57 43.52 41.99
------------------------------------------------------------------------------------
Regulation 338 ( 11) 65 ( 11) 3.25 16.92 5.46
Positive_regulation 1186 ( 74) 314 ( 74) 6.24 23.57 9.87
Negative_regulation 416 ( 0) 0 ( 0) 0.00 0.00 0.00
==[REG-TOTAL]== 1940 ( 85) 379 ( 85) 4.38 22.43 7.33
------------------------------------------------------------------------------------
Negation 220 ( 10) 617 ( 10) 4.55 1.62 2.39
Speculation 201 ( 25) 644 ( 25) 12.44 3.88 5.92
==[MOD-TOTAL]== 421 ( 35) 1261 ( 35) 8.31 2.78 4.16
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3941 ( 761) 3113 ( 761) 19.31 24.45 21.58
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 14) 33 ( 14) 8.05 42.42 13.53
Binding 398 ( 90) 206 ( 90) 22.61 43.69 29.80
Gene_expression 722 ( 406) 793 ( 406) 56.23 51.20 53.60
Transcription 137 ( 28) 150 ( 28) 20.44 18.67 19.51
Protein_catabolism 14 ( 6) 17 ( 6) 42.86 35.29 38.71
Phosphorylation 135 ( 109) 274 ( 109) 80.74 39.78 53.30
==[EVT-TOTAL]== 1580 ( 653) 1473 ( 653) 41.33 44.33 42.78
------------------------------------------------------------------------------------
Regulation 338 ( 15) 65 ( 15) 4.44 23.08 7.44
Positive_regulation 1186 ( 79) 314 ( 79) 6.66 25.16 10.53
Negative_regulation 416 ( 0) 0 ( 0) 0.00 0.00 0.00
==[REG-TOTAL]== 1940 ( 94) 379 ( 94) 4.85 24.80 8.11
------------------------------------------------------------------------------------
Negation 220 ( 12) 617 ( 12) 5.45 1.94 2.87
Speculation 201 ( 25) 644 ( 25) 12.44 3.88 5.92
==[MOD-TOTAL]== 421 ( 37) 1261 ( 37) 8.79 2.93 4.40
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3941 ( 784) 3113 ( 784) 19.89 25.18 22.23
------------------------------------------------------------------------------------