BioNLP'09 Shared Task on
Event Extraction
in conjunction with
BioNLP, a NAACL-HLT 2009
workshop, June 4-5 2009, Boulder, Colorado
Final Evaluation Results
of Concordia University, CLaC lab
Task 1
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 61) 67 ( 61) 35.06 91.04 50.62
Binding 347 ( 68) 175 ( 68) 19.60 38.86 26.05
Gene_expression 722 ( 394) 506 ( 394) 54.57 77.87 64.17
Transcription 137 ( 19) 35 ( 19) 13.87 54.29 22.09
Protein_catabolism 14 ( 9) 16 ( 9) 64.29 56.25 60.00
Phosphorylation 135 ( 91) 98 ( 91) 67.41 92.86 78.11
==[EVT-TOTAL]== 1529 ( 642) 897 ( 642) 41.99 71.57 52.93
------------------------------------------------------------------------------------
Regulation 291 ( 63) 153 ( 63) 21.65 41.18 28.38
Positive_regulation 983 ( 261) 562 ( 261) 26.55 46.44 33.79
Negative_regulation 379 ( 95) 196 ( 95) 25.07 48.47 33.04
==[REG-TOTAL]== 1653 ( 419) 911 ( 419) 25.35 45.99 32.68
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3182 ( 1061) 1808 ( 1061) 33.34 58.68 42.53
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 62) 67 ( 62) 35.63 92.54 51.45
Binding 347 ( 71) 175 ( 71) 20.46 40.57 27.20
Gene_expression 722 ( 402) 506 ( 402) 55.68 79.45 65.47
Transcription 137 ( 21) 35 ( 21) 15.33 60.00 24.42
Protein_catabolism 14 ( 9) 16 ( 9) 64.29 56.25 60.00
Phosphorylation 135 ( 94) 98 ( 94) 69.63 95.92 80.69
==[EVT-TOTAL]== 1529 ( 659) 897 ( 659) 43.10 73.47 54.33
------------------------------------------------------------------------------------
Regulation 291 ( 70) 153 ( 70) 24.05 45.75 31.53
Positive_regulation 983 ( 279) 561 ( 279) 28.38 49.73 36.14
Negative_regulation 379 ( 98) 196 ( 98) 25.86 50.00 34.09
==[REG-TOTAL]== 1653 ( 447) 910 ( 447) 27.04 49.12 34.88
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3182 ( 1106) 1807 ( 1106) 34.76 61.21 44.34
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 62) 67 ( 62) 35.63 92.54 51.45
Binding 347 ( 71) 175 ( 71) 20.46 40.57 27.20
Gene_expression 722 ( 402) 506 ( 402) 55.68 79.45 65.47
Transcription 137 ( 21) 35 ( 21) 15.33 60.00 24.42
Protein_catabolism 14 ( 9) 16 ( 9) 64.29 56.25 60.00
Phosphorylation 135 ( 94) 98 ( 94) 69.63 95.92 80.69
==[EVT-TOTAL]== 1529 ( 659) 897 ( 659) 43.10 73.47 54.33
------------------------------------------------------------------------------------
Regulation 291 ( 70) 153 ( 70) 24.05 45.75 31.53
Positive_regulation 983 ( 283) 561 ( 283) 28.79 50.45 36.66
Negative_regulation 379 ( 101) 196 ( 101) 26.65 51.53 35.13
==[REG-TOTAL]== 1653 ( 454) 910 ( 454) 27.47 49.89 35.43
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3182 ( 1113) 1807 ( 1113) 34.98 61.59 44.62
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 61) 67 ( 61) 35.06 91.04 50.62
Binding 398 ( 131) 230 ( 131) 32.91 56.96 41.72
Gene_expression 722 ( 394) 506 ( 394) 54.57 77.87 64.17
Transcription 137 ( 19) 35 ( 19) 13.87 54.29 22.09
Protein_catabolism 14 ( 9) 16 ( 9) 64.29 56.25 60.00
Phosphorylation 135 ( 91) 98 ( 91) 67.41 92.86 78.11
==[EVT-TOTAL]== 1580 ( 705) 952 ( 705) 44.62 74.05 55.69
------------------------------------------------------------------------------------
Regulation 338 ( 84) 174 ( 84) 24.85 48.28 32.81
Positive_regulation 1186 ( 381) 650 ( 381) 32.12 58.62 41.50
Negative_regulation 416 ( 118) 217 ( 118) 28.37 54.38 37.28
==[REG-TOTAL]== 1940 ( 583) 1041 ( 583) 30.05 56.00 39.11
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3520 ( 1288) 1993 ( 1288) 36.59 64.63 46.73
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 62) 67 ( 62) 35.63 92.54 51.45
Binding 398 ( 134) 229 ( 134) 33.67 58.52 42.74
Gene_expression 722 ( 402) 506 ( 402) 55.68 79.45 65.47
Transcription 137 ( 21) 35 ( 21) 15.33 60.00 24.42
Protein_catabolism 14 ( 9) 16 ( 9) 64.29 56.25 60.00
Phosphorylation 135 ( 94) 98 ( 94) 69.63 95.92 80.69
==[EVT-TOTAL]== 1580 ( 722) 951 ( 722) 45.70 75.92 57.05
------------------------------------------------------------------------------------
Regulation 338 ( 91) 174 ( 91) 26.92 52.30 35.55
Positive_regulation 1186 ( 404) 649 ( 404) 34.06 62.25 44.03
Negative_regulation 416 ( 123) 217 ( 123) 29.57 56.68 38.86
==[REG-TOTAL]== 1940 ( 618) 1040 ( 618) 31.86 59.42 41.48
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3520 ( 1340) 1991 ( 1340) 38.07 67.30 48.63
------------------------------------------------------------------------------------
Task 2
N/A
N/A
N/A
N/A
N/A
Task 3
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 61) 67 ( 61) 35.06 91.04 50.62
Binding 347 ( 68) 175 ( 68) 19.60 38.86 26.05
Gene_expression 722 ( 394) 506 ( 394) 54.57 77.87 64.17
Transcription 137 ( 19) 35 ( 19) 13.87 54.29 22.09
Protein_catabolism 14 ( 9) 16 ( 9) 64.29 56.25 60.00
Phosphorylation 135 ( 91) 98 ( 91) 67.41 92.86 78.11
==[EVT-TOTAL]== 1529 ( 642) 897 ( 642) 41.99 71.57 52.93
------------------------------------------------------------------------------------
Regulation 291 ( 63) 153 ( 63) 21.65 41.18 28.38
Positive_regulation 983 ( 261) 562 ( 261) 26.55 46.44 33.79
Negative_regulation 379 ( 95) 196 ( 95) 25.07 48.47 33.04
==[REG-TOTAL]== 1653 ( 419) 911 ( 419) 25.35 45.99 32.68
------------------------------------------------------------------------------------
Negation 227 ( 29) 67 ( 29) 12.78 43.28 19.73
Speculation 208 ( 31) 69 ( 31) 14.90 44.93 22.38
==[MOD-TOTAL]== 435 ( 60) 136 ( 60) 13.79 44.12 21.02
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3617 ( 1121) 1944 ( 1121) 30.99 57.66 40.32
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 62) 67 ( 62) 35.63 92.54 51.45
Binding 347 ( 71) 175 ( 71) 20.46 40.57 27.20
Gene_expression 722 ( 402) 506 ( 402) 55.68 79.45 65.47
Transcription 137 ( 21) 35 ( 21) 15.33 60.00 24.42
Protein_catabolism 14 ( 9) 16 ( 9) 64.29 56.25 60.00
Phosphorylation 135 ( 94) 98 ( 94) 69.63 95.92 80.69
==[EVT-TOTAL]== 1529 ( 659) 897 ( 659) 43.10 73.47 54.33
------------------------------------------------------------------------------------
Regulation 291 ( 70) 153 ( 70) 24.05 45.75 31.53
Positive_regulation 983 ( 279) 561 ( 279) 28.38 49.73 36.14
Negative_regulation 379 ( 98) 196 ( 98) 25.86 50.00 34.09
==[REG-TOTAL]== 1653 ( 447) 910 ( 447) 27.04 49.12 34.88
------------------------------------------------------------------------------------
Negation 227 ( 30) 67 ( 30) 13.22 44.78 20.41
Speculation 208 ( 32) 69 ( 32) 15.38 46.38 23.10
==[MOD-TOTAL]== 435 ( 62) 136 ( 62) 14.25 45.59 21.72
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3617 ( 1168) 1943 ( 1168) 32.29 60.11 42.01
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 62) 67 ( 62) 35.63 92.54 51.45
Binding 347 ( 71) 175 ( 71) 20.46 40.57 27.20
Gene_expression 722 ( 402) 506 ( 402) 55.68 79.45 65.47
Transcription 137 ( 21) 35 ( 21) 15.33 60.00 24.42
Protein_catabolism 14 ( 9) 16 ( 9) 64.29 56.25 60.00
Phosphorylation 135 ( 94) 98 ( 94) 69.63 95.92 80.69
==[EVT-TOTAL]== 1529 ( 659) 897 ( 659) 43.10 73.47 54.33
------------------------------------------------------------------------------------
Regulation 291 ( 70) 153 ( 70) 24.05 45.75 31.53
Positive_regulation 983 ( 283) 561 ( 283) 28.79 50.45 36.66
Negative_regulation 379 ( 101) 196 ( 101) 26.65 51.53 35.13
==[REG-TOTAL]== 1653 ( 454) 910 ( 454) 27.47 49.89 35.43
------------------------------------------------------------------------------------
Negation 227 ( 34) 67 ( 34) 14.98 50.75 23.13
Speculation 208 ( 35) 69 ( 35) 16.83 50.72 25.27
==[MOD-TOTAL]== 435 ( 69) 136 ( 69) 15.86 50.74 24.17
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3617 ( 1182) 1943 ( 1182) 32.68 60.83 42.52
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 61) 67 ( 61) 35.06 91.04 50.62
Binding 398 ( 131) 230 ( 131) 32.91 56.96 41.72
Gene_expression 722 ( 394) 506 ( 394) 54.57 77.87 64.17
Transcription 137 ( 19) 35 ( 19) 13.87 54.29 22.09
Protein_catabolism 14 ( 9) 16 ( 9) 64.29 56.25 60.00
Phosphorylation 135 ( 91) 98 ( 91) 67.41 92.86 78.11
==[EVT-TOTAL]== 1580 ( 705) 952 ( 705) 44.62 74.05 55.69
------------------------------------------------------------------------------------
Regulation 338 ( 84) 174 ( 84) 24.85 48.28 32.81
Positive_regulation 1186 ( 381) 650 ( 381) 32.12 58.62 41.50
Negative_regulation 416 ( 118) 217 ( 118) 28.37 54.38 37.28
==[REG-TOTAL]== 1940 ( 583) 1041 ( 583) 30.05 56.00 39.11
------------------------------------------------------------------------------------
Negation 220 ( 32) 67 ( 32) 14.55 47.76 22.30
Speculation 201 ( 34) 69 ( 34) 16.92 49.28 25.19
==[MOD-TOTAL]== 421 ( 66) 136 ( 66) 15.68 48.53 23.70
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3941 ( 1354) 2129 ( 1354) 34.36 63.60 44.61
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 62) 67 ( 62) 35.63 92.54 51.45
Binding 398 ( 134) 229 ( 134) 33.67 58.52 42.74
Gene_expression 722 ( 402) 506 ( 402) 55.68 79.45 65.47
Transcription 137 ( 21) 35 ( 21) 15.33 60.00 24.42
Protein_catabolism 14 ( 9) 16 ( 9) 64.29 56.25 60.00
Phosphorylation 135 ( 94) 98 ( 94) 69.63 95.92 80.69
==[EVT-TOTAL]== 1580 ( 722) 951 ( 722) 45.70 75.92 57.05
------------------------------------------------------------------------------------
Regulation 338 ( 91) 174 ( 91) 26.92 52.30 35.55
Positive_regulation 1186 ( 404) 649 ( 404) 34.06 62.25 44.03
Negative_regulation 416 ( 123) 217 ( 123) 29.57 56.68 38.86
==[REG-TOTAL]== 1940 ( 618) 1040 ( 618) 31.86 59.42 41.48
------------------------------------------------------------------------------------
Negation 220 ( 33) 67 ( 33) 15.00 49.25 23.00
Speculation 201 ( 35) 69 ( 35) 17.41 50.72 25.93
==[MOD-TOTAL]== 421 ( 68) 136 ( 68) 16.15 50.00 24.42
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3941 ( 1408) 2127 ( 1408) 35.73 66.20 46.41
------------------------------------------------------------------------------------