Return-Path: X-Original-To: apmail-ctakes-dev-archive@www.apache.org Delivered-To: apmail-ctakes-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 0599D17827 for ; Mon, 10 Nov 2014 17:17:55 +0000 (UTC) Received: (qmail 10889 invoked by uid 500); 10 Nov 2014 17:17:54 -0000 Delivered-To: apmail-ctakes-dev-archive@ctakes.apache.org Received: (qmail 10837 invoked by uid 500); 10 Nov 2014 17:17:54 -0000 Mailing-List: contact dev-help@ctakes.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@ctakes.apache.org Delivered-To: mailing list dev@ctakes.apache.org Received: (qmail 10822 invoked by uid 99); 10 Nov 2014 17:17:54 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 10 Nov 2014 17:17:54 +0000 X-ASF-Spam-Status: No, hits=-2.3 required=5.0 tests=RCVD_IN_DNSWL_MED,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of Pei.Chen@childrens.harvard.edu designates 134.174.13.91 as permitted sender) Received: from [134.174.13.91] (HELO mailsmtp1.childrenshospital.org) (134.174.13.91) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 10 Nov 2014 17:17:48 +0000 Received: from pps.filterd (mailsmtp1.childrenshospital.org [127.0.0.1]) by mailsmtp1.childrenshospital.org (8.14.7/8.14.7) with SMTP id sAAH9dfp021479 for ; Mon, 10 Nov 2014 12:11:26 -0500 Received: from smtpndc2.chboston.org (smtpndc2.chboston.org [10.20.50.105]) by mailsmtp1.childrenshospital.org with ESMTP id 1qhbcf1dkb-1 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Mon, 10 Nov 2014 12:11:22 -0500 Received: from pps.filterd (smtpndc2.chboston.org [127.0.0.1]) by smtpndc2.chboston.org (8.14.7/8.14.7) with SMTP id sAAH9Hod004571 for ; Mon, 10 Nov 2014 12:11:22 -0500 Received: from chexhubcasbdc1.chboston.org (chexhubcasbdc1.chboston.org [10.20.18.71]) by smtpndc2.chboston.org with ESMTP id 1qbtekdqfm-1 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NOT) for ; Mon, 10 Nov 2014 12:11:22 -0500 Received: from CHEXMBX1A.CHBOSTON.ORG ([fe80::3c05:8ca9:55a6:f320]) by CHEXHUBCASBDC1.CHBOSTON.ORG ([::1]) with mapi id 14.03.0169.001; Mon, 10 Nov 2014 12:11:21 -0500 From: "Chen, Pei" To: "" Subject: Re: svn commit: r1637884 - in /ctakes/trunk/ctakes-temporal/src/main/java/org/apache/ctakes/temporal: ae/ eval/ Thread-Topic: svn commit: r1637884 - in /ctakes/trunk/ctakes-temporal/src/main/java/org/apache/ctakes/temporal: ae/ eval/ Thread-Index: AQHP/Pni4O40s4gpM0CCCNtMgOVuS5xaGL7q Date: Mon, 10 Nov 2014 17:11:20 +0000 Message-ID: <2E6EAFEA-1CC0-4C87-8930-113760A4EFB8@childrens.harvard.edu> References: <20141110151956.9F3BE23888A6@eris.apache.org> In-Reply-To: <20141110151956.9F3BE23888A6@eris.apache.org> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.12.52,1.0.28,0.0.0000 definitions=2014-11-10_04:2014-11-10,2014-11-10,1970-01-01 signatures=0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.12.52,1.0.28,0.0.0000 definitions=2014-11-10_04:2014-11-10,2014-11-10,1970-01-01 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 kscore.is_bulkscore=1.1576634095789e-10 kscore.compositescore=1 circleOfTrustscore=805.392 compositescore=0.411528110256591 urlsuspect_oldscore=0.0115281102565911 suspectscore=0 recipient_domain_to_sender_totalscore=17270 phishscore=0 bulkscore=0 kscore.is_spamscore=0 recipient_to_sender_totalscore=0 recipient_domain_to_sender_domain_totalscore=31594 rbsscore=0.411528110256591 spamscore=0 recipient_to_sender_domain_totalscore=0 urlsuspectscore=0.9 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=7.0.1-1402240000 definitions=main-1411100128 X-Virus-Checked: Checked by ClamAV on apache.org Chen, Does this need to go into this upcoming release or can it wait till the nex= t one? Sent from my iPhone > On Nov 10, 2014, at 10:20 AM, "clin@apache.org" wrote: >=20 > Author: clin > Date: Mon Nov 10 15:19:55 2014 > New Revision: 1637884 >=20 > URL: http://svn.apache.org/r1637884 > Log: > add annotators and update evaluation code for i2b2 data. > add more system-generated events for candidate temporal relations. >=20 > Added: > ctakes/trunk/ctakes-temporal/src/main/java/org/apache/ctakes/temporal/= ae/EventEventI2B2RelationAnnotator.java > ctakes/trunk/ctakes-temporal/src/main/java/org/apache/ctakes/temporal/= ae/EventTimeI2B2RelationAnnotator.java > Modified: > ctakes/trunk/ctakes-temporal/src/main/java/org/apache/ctakes/temporal/= ae/EventEventRelationAnnotator.java > ctakes/trunk/ctakes-temporal/src/main/java/org/apache/ctakes/temporal/= eval/EvaluationOfEventEventThymeRelations.java > ctakes/trunk/ctakes-temporal/src/main/java/org/apache/ctakes/temporal/= eval/EvaluationOfEventTimeRelations.java > ctakes/trunk/ctakes-temporal/src/main/java/org/apache/ctakes/temporal/= eval/EvaluationOfI2B2TemporalRelations.java >=20 > Added: ctakes/trunk/ctakes-temporal/src/main/java/org/apache/ctakes/tempo= ral/ae/EventEventI2B2RelationAnnotator.java > URL: http://svn.apache.org/viewvc/ctakes/trunk/ctakes-temporal/src/main/j= ava/org/apache/ctakes/temporal/ae/EventEventI2B2RelationAnnotator.java?rev= =3D1637884&view=3Dauto > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D > --- ctakes/trunk/ctakes-temporal/src/main/java/org/apache/ctakes/temporal= /ae/EventEventI2B2RelationAnnotator.java (added) > +++ ctakes/trunk/ctakes-temporal/src/main/java/org/apache/ctakes/temporal= /ae/EventEventI2B2RelationAnnotator.java Mon Nov 10 15:19:55 2014 > @@ -0,0 +1,280 @@ > +/** > + * Licensed to the Apache Software Foundation (ASF) under one > + * or more contributor license agreements. See the NOTICE file > + * distributed with this work for additional information > + * regarding copyright ownership. The ASF licenses this file > + * to you under the Apache License, Version 2.0 (the > + * "License"); you may not use this file except in compliance > + * with the License. You may obtain a copy of the License at > + * > + * http://www.apache.org/licenses/LICENSE-2.0 > + * > + * Unless required by applicable law or agreed to in writing, > + * software distributed under the License is distributed on an > + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY > + * KIND, either express or implied. See the License for the > + * specific language governing permissions and limitations > + * under the License. > + */ > +package org.apache.ctakes.temporal.ae; > + > +import java.io.File; > +import java.util.ArrayList; > +import java.util.Arrays; > +import java.util.Collection; > +import java.util.List; > +import java.util.Map; > + > +import org.apache.ctakes.relationextractor.ae.RelationExtractorAnnotator= ; > +import org.apache.ctakes.relationextractor.ae.features.PartOfSpeechFeatu= resExtractor; > +import org.apache.ctakes.relationextractor.ae.features.RelationFeaturesE= xtractor; > +//import org.apache.ctakes.relationextractor.ae.features.TokenFeaturesEx= tractor; > +import org.apache.ctakes.temporal.ae.feature.CheckSpecialWordRelationExt= ractor; > +import org.apache.ctakes.temporal.ae.feature.ConjunctionRelationFeatures= Extractor; > +//import org.apache.ctakes.temporal.ae.feature.DependencyParseUtils; > +import org.apache.ctakes.temporal.ae.feature.DependencyPathFeaturesExtra= ctor; > +import org.apache.ctakes.temporal.ae.feature.CoordinateFeaturesExtractor= ; > +import org.apache.ctakes.temporal.ae.feature.DependingVerbsFeatureExtrac= tor; > +//import org.apache.ctakes.temporal.ae.feature.EventInBetweenPropertyExt= ractor; > +//import org.apache.ctakes.temporal.ae.feature.EventOutsidePropertyExtra= ctor; > +import org.apache.ctakes.temporal.ae.feature.SpecialAnnotationRelationEx= tractor; > +import org.apache.ctakes.temporal.ae.feature.TemporalPETFlatExtractor; > +import org.apache.ctakes.temporal.ae.feature.TokenPropertyFeaturesExtrac= tor; > +import org.apache.ctakes.temporal.ae.feature.DeterminerRelationFeaturesE= xtractor; > +import org.apache.ctakes.temporal.ae.feature.EventArgumentPropertyExtrac= tor; > +import org.apache.ctakes.temporal.ae.feature.EventTimeRelationFeatureExt= ractor; > +import org.apache.ctakes.temporal.ae.feature.EventPositionRelationFeatur= esExtractor; > +import org.apache.ctakes.temporal.ae.feature.NumberOfEventsInTheSameSent= enceExtractor; > +import org.apache.ctakes.temporal.ae.feature.NearbyVerbTenseRelationExtr= actor; > +import org.apache.ctakes.temporal.ae.feature.NumberOfEventTimeBetweenCan= didatesExtractor; > +import org.apache.ctakes.temporal.ae.feature.OverlappedHeadFeaturesExtra= ctor; > +import org.apache.ctakes.temporal.ae.feature.SRLRelationFeaturesExtracto= r; > +import org.apache.ctakes.temporal.ae.feature.TimeXRelationFeaturesExtrac= tor; > +import org.apache.ctakes.temporal.ae.feature.SectionHeaderRelationExtrac= tor; > +//import org.apache.ctakes.temporal.ae.feature.TemporalAttributeFeatureE= xtractor; > +import org.apache.ctakes.temporal.ae.feature.UmlsFeatureExtractor; > +import org.apache.ctakes.temporal.ae.feature.UnexpandedTokenFeaturesExtr= actor; > +//import org.apache.ctakes.temporal.ae.feature.treekernel.TemporalPETExt= ractor; > +import org.apache.ctakes.typesystem.type.relation.BinaryTextRelation; > +import org.apache.ctakes.typesystem.type.relation.RelationArgument; > +import org.apache.ctakes.typesystem.type.relation.TemporalTextRelation; > +//import org.apache.ctakes.typesystem.type.syntax.ConllDependencyNode; > +import org.apache.ctakes.typesystem.type.textsem.EventMention; > +import org.apache.ctakes.typesystem.type.textsem.IdentifiedAnnotation; > +import org.apache.ctakes.typesystem.type.textspan.Sentence; > +import org.apache.uima.analysis_engine.AnalysisEngineDescription; > +import org.apache.uima.jcas.JCas; > +import org.apache.uima.jcas.tcas.Annotation; > +import org.apache.uima.resource.ResourceInitializationException; > +import org.cleartk.ml.CleartkAnnotator; > +import org.cleartk.ml.DataWriter; > +import org.cleartk.ml.jar.DefaultDataWriterFactory; > +import org.cleartk.ml.jar.DirectoryDataWriterFactory; > +import org.cleartk.ml.jar.GenericJarClassifierFactory; > +import org.apache.uima.fit.factory.AnalysisEngineFactory; > +import org.apache.uima.fit.util.JCasUtil; > + > +import com.google.common.collect.Lists; > + > +public class EventEventI2B2RelationAnnotator extends RelationExtractorAn= notator { > + > + public static AnalysisEngineDescription createDataWriterDescription( > + Class> dataWriterClass, > + File outputDirectory, > + double probabilityOfKeepingANegativeExample) throws = ResourceInitializationException { > + return AnalysisEngineFactory.createEngineDescription( > + EventEventI2B2RelationAnnotator.class, > + CleartkAnnotator.PARAM_IS_TRAINING, > + true, > + DefaultDataWriterFactory.PARAM_DATA_WRITER_CLASS_NAME, > + dataWriterClass, > + DirectoryDataWriterFactory.PARAM_OUTPUT_DIRECTORY, > + outputDirectory, > + RelationExtractorAnnotator.PARAM_PROBABILITY_OF_KEEPING_= A_NEGATIVE_EXAMPLE, > + // not sure why this has to be cast; something funny goi= ng on in uimaFIT maybe? > + (float) probabilityOfKeepingANegativeExample); > + } > + > + public static AnalysisEngineDescription createAnnotatorDescription(S= tring modelPath) > + throws ResourceInitializationException { > + return AnalysisEngineFactory.createEngineDescription( > + EventEventI2B2RelationAnnotator.class, > + CleartkAnnotator.PARAM_IS_TRAINING, > + false, > + GenericJarClassifierFactory.PARAM_CLASSIFIER_JAR_PATH, > + modelPath); > + } > + > + /** > + * @deprecated use String path instead of File. > + * ClearTK will automatically Resolve the String to an InputStream. > + * This will allow resources to be read within from a jar as well as= File. =20 > + */ =20 > + @SuppressWarnings("dep-ann") > + public static AnalysisEngineDescription createAnnotatorDescription(F= ile modelDirectory) > + throws ResourceInitializationException { > + return AnalysisEngineFactory.createEngineDescription( > + EventEventI2B2RelationAnnotator.class, > + CleartkAnnotator.PARAM_IS_TRAINING, > + false, > + GenericJarClassifierFactory.PARAM_CLASSIFIER_JAR_PATH, > + new File(modelDirectory, "model.jar")); > + } > + > + @Override > + protected List getFeatureExtractors() { > + return Lists.newArrayList( > + // new TokenFeaturesExtractor() > + new UnexpandedTokenFeaturesExtractor() //use unexpanded = version for i2b2 data > + , new PartOfSpeechFeaturesExtractor() > + // , new TemporalPETExtractor() > + , new EventArgumentPropertyExtractor() > + , new NumberOfEventTimeBetweenCandidatesExtractor() > + , new SectionHeaderRelationExtractor() > + , new NearbyVerbTenseRelationExtractor() > + , new CheckSpecialWordRelationExtractor() > + , new UmlsFeatureExtractor() > + , new DependencyPathFeaturesExtractor() > + , new CoordinateFeaturesExtractor() > + , new OverlappedHeadFeaturesExtractor() > + , new SRLRelationFeaturesExtractor() > + , new NumberOfEventsInTheSameSentenceExtractor() > + , new EventPositionRelationFeaturesExtractor() //not hel= pful > + , new TimeXRelationFeaturesExtractor() //not helpful > + , new ConjunctionRelationFeaturesExtractor() > + , new DeterminerRelationFeaturesExtractor() > + , new EventTimeRelationFeatureExtractor() > + , new TokenPropertyFeaturesExtractor() > + , new DependingVerbsFeatureExtractor() > + , new SpecialAnnotationRelationExtractor() //not helpful > + , new TemporalPETFlatExtractor() > + // , new EventInBetweenPropertyExtractor(= ) > + // , new EventOutsidePropertyExtractor() > + ); > + } > + > + @Override > + protected Class getCoveringClass() { > + return Sentence.class; > + } > + > + @Override > + protected List getCandidateRelationArgumen= tPairs( > + JCas jCas, Annotation sentence) { > + > + Map> coveringMap =3D > + JCasUtil.indexCovering(jCas, EventMention.class, EventMe= ntion.class); > + > + List pairs =3D Lists.newArrayList(); > + List events =3D new ArrayList<>(JCasUtil.selectCov= ered(jCas, EventMention.class, sentence)); > + //filter events: > + List realEvents =3D Lists.newArrayList(); > + for( EventMention event : events){ > + if(event.getClass().equals(EventMention.class)){ > + realEvents.add(event); > + } > + } > + events =3D realEvents; > + > + int eventNum =3D events.size(); > + > + for (int i =3D 0; i < eventNum-1; i++){ > + for(int j =3D i+1; j < eventNum; j++){ > + EventMention eventA =3D events.get(j); > + EventMention eventB =3D events.get(i); > + > + if(this.isTraining()){ > + //pairing covering system events: > + for (EventMention event1 : coveringMap.get(eventA)){ > + for(EventMention event2 : coveringMap.get(eventB= )){ > + pairs.add(new IdentifiedAnnotationPair(event= 1, event2)); =20 > + } > + pairs.add(new IdentifiedAnnotationPair(event1, e= ventB)); > + } > + for(EventMention event2 : coveringMap.get(eventB)){ > + pairs.add(new IdentifiedAnnotationPair(eventA, e= vent2)); =20 > + } > + //pairing covered system events: > + for(EventMention event1 : JCasUtil.selectCovered(jCa= s, EventMention.class, eventA)){ > + for(EventMention event2 : JCasUtil.selectCovered= (jCas, EventMention.class, eventB)){ > + pairs.add(new IdentifiedAnnotationPair(event= 1, event2)); > + } > + pairs.add(new IdentifiedAnnotationPair(event1, e= ventB)); > + } > + for(EventMention event2 : JCasUtil.selectCovered(jCa= s, EventMention.class, eventB)){ > + pairs.add(new IdentifiedAnnotationPair(eventA, e= vent2)); > + } > + } > + pairs.add(new IdentifiedAnnotationPair(eventA, eventB)); > + > + } > + } > + > + > + // if(eventNum >=3D 2){ > + // for ( int i =3D 0; i< eventNum -1 ; i ++){ > + // EventMention evI =3D events.get(i); > + // for(int j =3D i+1; j< eventNum; j++){ > + // EventMention evJ =3D events.get(j); > + // if(j-i=3D=3D1 || j-i=3D=3DeventNum-1){//if= two events are consecutive, or major events > + // pairs.add(new IdentifiedAnnotationPair= (evJ, evI)); > + // }else if(ifDependent(jCas, evI, evJ)){//if= the event pairs are dependent// eventNum < 7 &&=20 > + // pairs.add(new IdentifiedAnnotationPair= (evJ, evI)); > + // }else{// if the=20 > + // continue; > + // } > + // } > + // } > + // } > + > + return pairs; > + } > + > + @Override > + protected void createRelation(JCas jCas, IdentifiedAnnotation arg1, > + IdentifiedAnnotation arg2, String predictedCategory) { > + RelationArgument relArg1 =3D new RelationArgument(jCas); > + relArg1.setArgument(arg1); > + relArg1.setRole("Arg1"); > + relArg1.addToIndexes(); > + RelationArgument relArg2 =3D new RelationArgument(jCas); > + relArg2.setArgument(arg2); > + relArg2.setRole("Arg2"); > + relArg2.addToIndexes(); > + TemporalTextRelation relation =3D new TemporalTextRelation(jCas)= ; > + relation.setArg1(relArg1); > + relation.setArg2(relArg2); > + relation.setCategory(predictedCategory); > + relation.addToIndexes(); > + } > + > + @Override > + protected String getRelationCategory( > + Map, BinaryTextRelation> relationLookup, > + IdentifiedAnnotation arg1, > + IdentifiedAnnotation arg2) { > + BinaryTextRelation relation =3D relationLookup.get(Arrays.asList= (arg1, arg2)); > + String category =3D null; > + if (relation !=3D null && relation instanceof TemporalTextRelati= on) { > + category =3D relation.getCategory(); > + } else { > + relation =3D relationLookup.get(Arrays.asList(arg2, arg1)); > + if (relation !=3D null && relation instanceof TemporalTextRe= lation) { > + if(relation.getCategory().equals("OVERLAP")){ > + category =3D relation.getCategory(); > + // }else if (relation.getCategory().e= quals("BEFORE")){ > + // category =3D "AFTER"; > + // }else if (relation.getCategory().e= quals("AFTER")){ > + // category =3D "BEFORE"; > + // } > + }else{ > + category =3D relation.getCategory() + "-1"; > + } > + } > + } > + if (category =3D=3D null && coin.nextDouble() <=3D this.probabil= ityOfKeepingANegativeExample) { > + category =3D NO_RELATION_CATEGORY; > + } > + return category; > + } > +} >=20 > Modified: ctakes/trunk/ctakes-temporal/src/main/java/org/apache/ctakes/te= mporal/ae/EventEventRelationAnnotator.java > URL: http://svn.apache.org/viewvc/ctakes/trunk/ctakes-temporal/src/main/j= ava/org/apache/ctakes/temporal/ae/EventEventRelationAnnotator.java?rev=3D16= 37884&r1=3D1637883&r2=3D1637884&view=3Ddiff > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D > --- ctakes/trunk/ctakes-temporal/src/main/java/org/apache/ctakes/temporal= /ae/EventEventRelationAnnotator.java (original) > +++ ctakes/trunk/ctakes-temporal/src/main/java/org/apache/ctakes/temporal= /ae/EventEventRelationAnnotator.java Mon Nov 10 15:19:55 2014 > @@ -198,15 +198,25 @@ public class EventEventRelationAnnotator > } > if(eventAValid && eventBValid){ > if(this.isTraining()){ > + //pairing covering system events: > for (EventMention event1 : coveringMap.get(eventA)= ){ > for(EventMention event2 : coveringMap.get(even= tB)){ > pairs.add(new IdentifiedAnnotationPair(eve= nt1, event2)); > } > + pairs.add(new IdentifiedAnnotationPair(event= 1, eventB)); > } > + for(EventMention event2 : coveringMap.get(eventB= )){ > + pairs.add(new IdentifiedAnnotationPair(event= A, event2)); > + } > + //pairing covered system events: > for(EventMention event1 : JCasUtil.selectCovered(j= Cas, EventMention.class, eventA)){ > for(EventMention event2 : JCasUtil.selectCover= ed(jCas, EventMention.class, eventB)){ > pairs.add(new IdentifiedAnnotationPair(eve= nt1, event2)); > } > + pairs.add(new IdentifiedAnnotationPair(event= 1, eventB)); > + } > + for(EventMention event2 : JCasUtil.selectCovered= (jCas, EventMention.class, eventB)){ > + pairs.add(new IdentifiedAnnotationPair(event= A, event2)); > } > } > pairs.add(new IdentifiedAnnotationPair(eventA, eventB)= ); >=20 > Added: ctakes/trunk/ctakes-temporal/src/main/java/org/apache/ctakes/tempo= ral/ae/EventTimeI2B2RelationAnnotator.java > URL: http://svn.apache.org/viewvc/ctakes/trunk/ctakes-temporal/src/main/j= ava/org/apache/ctakes/temporal/ae/EventTimeI2B2RelationAnnotator.java?rev= =3D1637884&view=3Dauto > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D > --- ctakes/trunk/ctakes-temporal/src/main/java/org/apache/ctakes/temporal= /ae/EventTimeI2B2RelationAnnotator.java (added) > +++ ctakes/trunk/ctakes-temporal/src/main/java/org/apache/ctakes/temporal= /ae/EventTimeI2B2RelationAnnotator.java Mon Nov 10 15:19:55 2014 > @@ -0,0 +1,253 @@ > +/** > + * Licensed to the Apache Software Foundation (ASF) under one > + * or more contributor license agreements. See the NOTICE file > + * distributed with this work for additional information > + * regarding copyright ownership. The ASF licenses this file > + * to you under the Apache License, Version 2.0 (the > + * "License"); you may not use this file except in compliance > + * with the License. You may obtain a copy of the License at > + * > + * http://www.apache.org/licenses/LICENSE-2.0 > + * > + * Unless required by applicable law or agreed to in writing, > + * software distributed under the License is distributed on an > + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY > + * KIND, either express or implied. See the License for the > + * specific language governing permissions and limitations > + * under the License. > + */ > +package org.apache.ctakes.temporal.ae; > + > +import java.io.File; > +import java.util.Arrays; > +import java.util.Collection; > +import java.util.List; > +import java.util.Map; > + > +import org.apache.ctakes.relationextractor.ae.RelationExtractorAnnotator= ; > +import org.apache.ctakes.relationextractor.ae.features.PartOfSpeechFeatu= resExtractor; > +import org.apache.ctakes.relationextractor.ae.features.RelationFeaturesE= xtractor; > +import org.apache.ctakes.temporal.ae.feature.CheckSpecialWordRelationExt= ractor; > +import org.apache.ctakes.temporal.ae.feature.ConjunctionRelationFeatures= Extractor; > +import org.apache.ctakes.temporal.ae.feature.DependencyFeatureExtractor; > +import org.apache.ctakes.temporal.ae.feature.DependencyPathFeaturesExtra= ctor; > +import org.apache.ctakes.temporal.ae.feature.EventArgumentPropertyExtrac= tor; > +//import org.apache.ctakes.temporal.ae.feature.EventIndexOfSameSentenceR= elationFeaturesExtractor; > +//import org.apache.ctakes.temporal.ae.feature.EventPositionRelationFeat= uresExtractor; > +//import org.apache.ctakes.temporal.ae.feature.EventTimeRelationFeatureE= xtractor; > +import org.apache.ctakes.temporal.ae.feature.NearbyVerbTenseRelationExtr= actor; > +import org.apache.ctakes.temporal.ae.feature.NearestFlagFeatureExtractor= ; > +import org.apache.ctakes.temporal.ae.feature.NumberOfEventTimeBetweenCan= didatesExtractor; > +import org.apache.ctakes.temporal.ae.feature.OverlappedHeadFeaturesExtra= ctor; > +//import org.apache.ctakes.temporal.ae.feature.SRLRelationFeaturesExtrac= tor; > +//import org.apache.ctakes.temporal.ae.feature.SectionHeaderRelationExtr= actor; > +import org.apache.ctakes.temporal.ae.feature.TemporalAttributeFeatureExt= ractor; > +import org.apache.ctakes.temporal.ae.feature.TemporalPETFlatExtractor; > +//import org.apache.ctakes.temporal.ae.feature.TimeWordTypeRelationExtra= ctor; > +import org.apache.ctakes.temporal.ae.feature.TimeXPropertyRelationFeatur= esExtractor; > +import org.apache.ctakes.temporal.ae.feature.TimeXRelationFeaturesExtrac= tor; > +import org.apache.ctakes.temporal.ae.feature.UnexpandedTokenFeaturesExtr= actor; > +//import org.apache.ctakes.temporal.ae.feature.TemporalAttributeFeatureE= xtractor; > +//import org.apache.ctakes.temporal.ae.feature.treekernel.EventTimeFlatT= reeFeatureExtractor; > +//import org.apache.ctakes.temporal.ae.feature.treekernel.EventVerbRelat= ionTreeExtractor; > +//import org.apache.ctakes.temporal.ae.feature.treekernel.TemporalPETExt= ractor; > +//import org.apache.ctakes.temporal.ae.feature.treekernel.TemporalPathEx= tractor; > +import org.apache.ctakes.typesystem.type.relation.BinaryTextRelation; > +import org.apache.ctakes.typesystem.type.relation.RelationArgument; > +import org.apache.ctakes.typesystem.type.relation.TemporalTextRelation; > +import org.apache.ctakes.typesystem.type.textsem.EventMention; > +import org.apache.ctakes.typesystem.type.textsem.IdentifiedAnnotation; > +import org.apache.ctakes.typesystem.type.textsem.TimeMention; > +import org.apache.ctakes.typesystem.type.textspan.Sentence; > +import org.apache.uima.analysis_engine.AnalysisEngineDescription; > +import org.apache.uima.jcas.JCas; > +import org.apache.uima.jcas.tcas.Annotation; > +import org.apache.uima.resource.ResourceInitializationException; > +import org.cleartk.ml.CleartkAnnotator; > +import org.cleartk.ml.DataWriter; > +import org.cleartk.ml.jar.DefaultDataWriterFactory; > +import org.cleartk.ml.jar.DirectoryDataWriterFactory; > +import org.cleartk.ml.jar.GenericJarClassifierFactory; > +import org.apache.uima.fit.factory.AnalysisEngineFactory; > +import org.apache.uima.fit.util.JCasUtil; > + > +import com.google.common.collect.Lists; > + > +public class EventTimeI2B2RelationAnnotator extends RelationExtractorAnn= otator { > + > + public static AnalysisEngineDescription createDataWriterDescription( > + Class> dataWriterClass, > + File outputDirectory, > + double probabilityOfKeepingANegativeExample) throws = ResourceInitializationException { > + return AnalysisEngineFactory.createEngineDescription( > + EventTimeI2B2RelationAnnotator.class, > + CleartkAnnotator.PARAM_IS_TRAINING, > + true, > + DefaultDataWriterFactory.PARAM_DATA_WRITER_CLASS_NAME, > + dataWriterClass, > + DirectoryDataWriterFactory.PARAM_OUTPUT_DIRECTORY, > + outputDirectory, > + RelationExtractorAnnotator.PARAM_PROBABILITY_OF_KEEPING_= A_NEGATIVE_EXAMPLE, > + // not sure why this has to be cast; something funny goi= ng on in uimaFIT maybe? > + (float) probabilityOfKeepingANegativeExample); > + } > + > + public static AnalysisEngineDescription createEngineDescription(Stri= ng modelPath) > + throws ResourceInitializationException { > + return AnalysisEngineFactory.createEngineDescription( > + EventTimeI2B2RelationAnnotator.class, > + CleartkAnnotator.PARAM_IS_TRAINING, > + false, > + GenericJarClassifierFactory.PARAM_CLASSIFIER_JAR_PATH, > + modelPath); > + } > + /** > + * @deprecated use String path instead of File. > + * ClearTK will automatically Resolve the String to an InputStream. > + * This will allow resources to be read within from a jar as well as= File. =20 > + */ =20 > + @Deprecated > + public static AnalysisEngineDescription createEngineDescription(File= modelDirectory) > + throws ResourceInitializationException { > + return AnalysisEngineFactory.createEngineDescription( > + EventTimeI2B2RelationAnnotator.class, > + CleartkAnnotator.PARAM_IS_TRAINING, > + false, > + GenericJarClassifierFactory.PARAM_CLASSIFIER_JAR_PATH, > + new File(modelDirectory, "model.jar")); > + } > + > + @Override > + protected List getFeatureExtractors() { > + return Lists.newArrayList( > +// new TokenFeaturesExtractor() > + new UnexpandedTokenFeaturesExtractor() //use unexpanded = version for i2b2 data > + , new PartOfSpeechFeaturesExtractor() > + , new TemporalAttributeFeatureExtractor() > + // , new EventTimeFlatTreeFeatureExtracto= r() > + // , new TemporalPETExtractor() > + //, new TemporalPathExtractor() > + // , new EventVerbRelationTreeExtractor() > + , new NumberOfEventTimeBetweenCandidatesExtractor() > + // , new SectionHeaderRelationExtractor() > + , new NearbyVerbTenseRelationExtractor() > + , new CheckSpecialWordRelationExtractor() > + , new NearestFlagFeatureExtractor() > + , new DependencyPathFeaturesExtractor() > + , new DependencyFeatureExtractor() > + // , new SRLRelationFeaturesExtractor()//= tried, but not helpful > + , new EventArgumentPropertyExtractor() > + , new OverlappedHeadFeaturesExtractor() > + // , new EventTimeRelationFeatureExtracto= r() > + , new ConjunctionRelationFeaturesExtractor() > + // , new EventPositionRelationFeaturesExt= ractor() //tried, but not helpful > + , new TimeXRelationFeaturesExtractor() > + , new TemporalPETFlatExtractor() > + , new TimeXPropertyRelationFeaturesExtractor() > + // , new TimeWordTypeRelationExtractor() = //tried, but not helpful > + // , new EventIndexOfSameSentenceRelation= FeaturesExtractor() //tried, but not helpful > + ); > + } > + > + @Override > + protected Class getCoveringClass() { > + return Sentence.class; > + } > + > + @Override > + public List getCandidateRelationArgumentPa= irs( > + JCas jCas, > + Annotation sentence) { > + Map> coveringMap =3D > + JCasUtil.indexCovering(jCas, EventMention.class, EventMe= ntion.class); > + > + List pairs =3D Lists.newArrayList(); > + for (EventMention event : JCasUtil.selectCovered(jCas, EventMent= ion.class, sentence)) { > + boolean eventValid =3D false; > + if (event.getClass().equals(EventMention.class)) {//event is= a gold event > + eventValid =3D true; > + } > + > + if(eventValid){ > + // ignore subclasses like Procedure and Disease/Disorder > + if(this.isTraining()){//if training mode, train on both = gold event and span-overlapping system events > + for (TimeMention time : JCasUtil.selectCovered(jCas,= TimeMention.class, sentence)) { > + Collection eventList =3D coveringM= ap.get(event); > + for(EventMention covEvent : eventList){ > + pairs.add(new IdentifiedAnnotationPair(covEv= ent, time)); > + } > + pairs.add(new IdentifiedAnnotationPair(event, ti= me)); > + for(EventMention covedEvent : JCasUtil.selectCov= ered(jCas, EventMention.class, event)){//select covered events > + pairs.add(new IdentifiedAnnotationPair(coved= Event, time)); > + } > + } > + }else{//if testing mode, only test on system generated e= vents > + for (TimeMention time : JCasUtil.selectCovered(jCas,= TimeMention.class, sentence)) { > + pairs.add(new IdentifiedAnnotationPair(event, ti= me)); > + } > + } > + } > + } > + > + return pairs; > + } > + > + @Override > + protected void createRelation(JCas jCas, IdentifiedAnnotation arg1, > + IdentifiedAnnotation arg2, String predictedCategory) { > + RelationArgument relArg1 =3D new RelationArgument(jCas); > + relArg1.setArgument(arg1); > + relArg1.setRole("Arg1"); > + relArg1.addToIndexes(); > + RelationArgument relArg2 =3D new RelationArgument(jCas); > + relArg2.setArgument(arg2); > + relArg2.setRole("Arg2"); > + relArg2.addToIndexes(); > + TemporalTextRelation relation =3D new TemporalTextRelation(jCas)= ; > + relation.setArg1(relArg1); > + relation.setArg2(relArg2); > + relation.setCategory(predictedCategory); > + relation.addToIndexes(); > + } > + > + > + @Override > + protected String getRelationCategory( > + Map, BinaryTextRelation> relationLookup, > + IdentifiedAnnotation arg1, > + IdentifiedAnnotation arg2) { > + BinaryTextRelation relation =3D relationLookup.get(Arrays.asList= (arg1, arg2)); > + String category =3D null; > + if (relation !=3D null) { > + category =3D relation.getCategory(); > + } else { > + relation =3D relationLookup.get(Arrays.asList(arg2, arg1)); > + if (relation !=3D null) { > + if(relation.getCategory().equals("OVERLAP")){ > + category =3D relation.getCategory(); > + // }else if (relation.getCategory().e= quals("BEFORE")){ > + // category =3D "AFTER"; > + // }else if (relation.getCategory().e= quals("AFTER")){ > + // category =3D "BEFORE"; > + // } > + }else{ > + category =3D relation.getCategory() + "-1"; > + } > + } > + } > + > + if(category!=3Dnull){ > + if(!((EventMention)arg1).getClass().equals(EventMention.clas= s)){ > + System.out.println("find system-event relations: "+ arg1= .getCoveredText() + " -"+category+"- " + arg2.getCoveredText()); > + }else{ > + System.out.println("find gold-event relations: "+ arg1.g= etCoveredText() + " -"+category+"- " + arg2.getCoveredText()); > + } > + } > + > + if (category =3D=3D null && coin.nextDouble() <=3D this.probabil= ityOfKeepingANegativeExample) { > + category =3D NO_RELATION_CATEGORY; > + } > + > + return category; > + } > +} >=20 > Modified: ctakes/trunk/ctakes-temporal/src/main/java/org/apache/ctakes/te= mporal/eval/EvaluationOfEventEventThymeRelations.java > URL: http://svn.apache.org/viewvc/ctakes/trunk/ctakes-temporal/src/main/j= ava/org/apache/ctakes/temporal/eval/EvaluationOfEventEventThymeRelations.ja= va?rev=3D1637884&r1=3D1637883&r2=3D1637884&view=3Ddiff > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D > --- ctakes/trunk/ctakes-temporal/src/main/java/org/apache/ctakes/temporal= /eval/EvaluationOfEventEventThymeRelations.java (original) > +++ ctakes/trunk/ctakes-temporal/src/main/java/org/apache/ctakes/temporal= /eval/EvaluationOfEventEventThymeRelations.java Mon Nov 10 15:19:55 2014 > @@ -329,7 +329,7 @@ EvaluationOfTemporalRelations_ImplBase{ > } >=20 > // HideOutput hider =3D new HideOutput(); > - JarClassifierBuilder.trainAndPackage(new File(directory,"event-e= vent"),"-c", optArray[1]);//"-w1","0.09","-w2","4","-w3","9","-w5","2","-w6= ","16","-w7","10","-w8","6", "-w9","45","-w10","30","-c", optArray[1]);//"-= c", "0.05");//optArray); > + JarClassifierBuilder.trainAndPackage(new File(directory,"event-e= vent"),"-w1","0.08","-w3","3","-w4","17","-w5","20","-w6","16","-w7","10","= -w8","6", "-w9","45","-w10","30","-c", optArray[1]);//"-c", "0.05");//optAr= ray); > // JarClassifierBuilder.trainAndPackage(new File(directory,= "event-event"), "-h","0","-c", "1000"); > // hider.restoreOutput(); > // hider.close(); > @@ -529,6 +529,20 @@ EvaluationOfTemporalRelations_ImplBase{ > for(EventMention covEventB : coveringMap.get(event= 2)){ > createRelation(relationView, covEventA, covEve= ntB, relation.getCategory()); > } > + createRelation(relationView, covEventA, event2, = relation.getCategory()); > + } > + for(EventMention covEventB : coveringMap.get(event2)= ){ > + createRelation(relationView, event1, covEventB, = relation.getCategory()); > + } > + //get covered system events: > + for(EventMention covedEventA : JCasUtil.selectCovere= d(jCas, EventMention.class, event1)){//select covered events > + for(EventMention covedEventB : JCasUtil.selectCo= vered(jCas, EventMention.class, event2)){ > + createRelation(relationView, covedEventA, co= vedEventB, relation.getCategory()); > + } > + createRelation(relationView, covedEventA, event2= , relation.getCategory()); > + } > + for(EventMention covedEventB : JCasUtil.selectCovere= d(jCas, EventMention.class, event2)){ > + createRelation(relationView, event1, covedEventB= , relation.getCategory()); > } > } > } >=20 > Modified: ctakes/trunk/ctakes-temporal/src/main/java/org/apache/ctakes/te= mporal/eval/EvaluationOfEventTimeRelations.java > URL: http://svn.apache.org/viewvc/ctakes/trunk/ctakes-temporal/src/main/j= ava/org/apache/ctakes/temporal/eval/EvaluationOfEventTimeRelations.java?rev= =3D1637884&r1=3D1637883&r2=3D1637884&view=3Ddiff > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D > --- ctakes/trunk/ctakes-temporal/src/main/java/org/apache/ctakes/temporal= /eval/EvaluationOfEventTimeRelations.java (original) > +++ ctakes/trunk/ctakes-temporal/src/main/java/org/apache/ctakes/temporal= /eval/EvaluationOfEventTimeRelations.java Mon Nov 10 15:19:55 2014 > @@ -34,6 +34,7 @@ import java.util.List; > import java.util.Map; > import java.util.Set; >=20 > +import org.apache.ctakes.relationextractor.ae.RelationExtractorAnnotator= .IdentifiedAnnotationPair; > import org.apache.ctakes.relationextractor.eval.RelationExtractorEvaluati= on.HashableArguments; > import org.apache.ctakes.temporal.ae.EventTimeRelationAnnotator; > import org.apache.ctakes.temporal.ae.EventTimeSelfRelationAnnotator; > @@ -693,6 +694,9 @@ EvaluationOfTemporalRelations_ImplBase{ > createRelation(relationView, covEvent, arg2, r= elation.getCategory()); > } > } > + for(EventMention covedEvent : JCasUtil.selectCovered= (jCas, EventMention.class, event)){//select covered events > + createRelation(relationView, covedEvent, arg2, r= elation.getCategory()); > + } > }else if(arg2 instanceof EventMention && arg1 instanceof T= imeMention){ > event =3D (EventMention) arg2; > Collection eventList =3D coveringMap.get= (event); > @@ -701,6 +705,9 @@ EvaluationOfTemporalRelations_ImplBase{ > createRelation(relationView, arg1, covEvent, r= elation.getCategory()); > } > } > + for(EventMention covedEvent : JCasUtil.selectCovered= (jCas, EventMention.class, event)){//select covered events > + createRelation(relationView, arg1, covedEvent, r= elation.getCategory()); > + } > } > } >=20 >=20 > Modified: ctakes/trunk/ctakes-temporal/src/main/java/org/apache/ctakes/te= mporal/eval/EvaluationOfI2B2TemporalRelations.java > URL: http://svn.apache.org/viewvc/ctakes/trunk/ctakes-temporal/src/main/j= ava/org/apache/ctakes/temporal/eval/EvaluationOfI2B2TemporalRelations.java?= rev=3D1637884&r1=3D1637883&r2=3D1637884&view=3Ddiff > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D > --- ctakes/trunk/ctakes-temporal/src/main/java/org/apache/ctakes/temporal= /eval/EvaluationOfI2B2TemporalRelations.java (original) > +++ ctakes/trunk/ctakes-temporal/src/main/java/org/apache/ctakes/temporal= /eval/EvaluationOfI2B2TemporalRelations.java Mon Nov 10 15:19:55 2014 > @@ -38,14 +38,15 @@ import org.apache.ctakes.core.ae.CDASegm > import org.apache.ctakes.relationextractor.eval.RelationExtractorEvaluati= on.HashableArguments; > import org.apache.ctakes.temporal.ae.ConsecutiveSentencesEventEventRelati= onAnnotator; > import org.apache.ctakes.temporal.ae.ConsecutiveSentencesEventTimeRelatio= nAnnotator; > -import org.apache.ctakes.temporal.ae.EventTimeSelfRelationAnnotator; > import org.apache.ctakes.temporal.ae.TemporalRelationRuleAnnotator; > import org.apache.ctakes.temporal.ae.DocTimeRelAnnotator; > import org.apache.ctakes.temporal.ae.EventAdmissionTimeAnnotator; > import org.apache.ctakes.temporal.ae.EventDischargeTimeAnnotator; > -import org.apache.ctakes.temporal.ae.EventEventRelationAnnotator; > -import org.apache.ctakes.temporal.ae.EventTimeRelationAnnotator; > +import org.apache.ctakes.temporal.ae.EventEventI2B2RelationAnnotator; > +import org.apache.ctakes.temporal.ae.EventTimeI2B2RelationAnnotator; > import org.apache.ctakes.temporal.ae.baselines.RecallBaselineEventTimeRel= ationAnnotator; > +import org.apache.ctakes.temporal.eval.EvaluationOfEventEventThymeRelati= ons.AddEEPotentialRelations; > +import org.apache.ctakes.temporal.eval.EvaluationOfEventTimeRelations.Ad= dPotentialRelations; > import org.apache.ctakes.temporal.eval.EvaluationOfEventTimeRelations.Par= ameterSettings; > import org.apache.ctakes.temporal.eval.EvaluationOfTemporalRelations_Impl= Base.RemoveNonContainsRelations.RemoveGoldAttributes; > import org.apache.ctakes.temporal.utils.AnnotationIdCollection; > @@ -66,7 +67,6 @@ import org.apache.uima.fit.component.JCa > import org.apache.uima.fit.descriptor.ConfigurationParameter; > import org.apache.uima.fit.factory.AggregateBuilder; > import org.apache.uima.fit.factory.AnalysisEngineFactory; > -import org.apache.uima.fit.pipeline.JCasIterator; > import org.apache.uima.fit.pipeline.SimplePipeline; > import org.apache.uima.fit.util.JCasUtil; > import org.apache.uima.jcas.JCas; > @@ -75,7 +75,7 @@ import org.apache.uima.resource.Resource > import org.apache.uima.util.FileUtils; > import org.cleartk.eval.AnnotationStatistics; > import org.cleartk.ml.jar.JarClassifierBuilder; > -//import org.cleartk.ml.libsvm.LibSvmStringOutcomeDataWriter; > +import org.cleartk.ml.libsvm.LibSvmStringOutcomeDataWriter; > import org.cleartk.ml.liblinear.LibLinearStringOutcomeDataWriter; > //import org.cleartk.classifier.tksvmlight.TKSVMlightStringOutcomeDataWri= ter; > import org.cleartk.ml.tksvmlight.model.CompositeKernel.ComboOperator; > @@ -292,13 +292,18 @@ EvaluationOfTemporalRelations_ImplBase{ > // CAS.NAME_DEFAULT_SOFA, > // GOLD_VIEW_NAME); > // aggregateBuilder.add(AnalysisEngineFactory.createPrimiti= veDescription(RemoveNonContainsRelations.class)); > - aggregateBuilder.add(AnalysisEngineFactory.createEngineDescripti= on(AddFlippedOverlap.class));//add flipped overlap instances to training da= ta > +// aggregateBuilder.add(AnalysisEngineFactory.createEngineDescrip= tion(AddFlippedOverlap.class));//add flipped overlap instances to training = data >=20 > // aggregateBuilder.add(AnalysisEngineFactory.createPrimiti= veDescription(RemoveEventEventRelations.class)); > //test rules: > // aggregateBuilder.add(AnalysisEngineFactory.createPrimiti= veDescription(TemporalRelationRuleAnnotator.class)); > + =20 > + //add unlabeled nearby system events as potential event-time lin= ks:=20 > + aggregateBuilder.add(AnalysisEngineFactory.createEngineDescripti= on(AddPotentialRelations.class)); > + //add unlabeled nearby system events as potential event-event li= nks:=20 > + aggregateBuilder.add(AnalysisEngineFactory.createEngineDescripti= on(AddEEPotentialRelations.class)); >=20 > - aggregateBuilder.add(EventTimeSelfRelationAnnotator.createDataWr= iterDescription( > + aggregateBuilder.add(EventTimeI2B2RelationAnnotator.createDataWr= iterDescription( > // LibSvmStringOutcomeDataWriter.class, > LibLinearStringOutcomeDataWriter.class, > // TKSVMlightStringOutcomeDataWriter.class, > @@ -306,27 +311,27 @@ EvaluationOfTemporalRelations_ImplBase{ > // SVMlightStringOutcomeDataWriter.class, =20 > new File(directory,EVENT_TIME), > params.probabilityOfKeepingANegativeExample)); > - aggregateBuilder.add(EventEventRelationAnnotator.createDataWrite= rDescription( > + aggregateBuilder.add(EventEventI2B2RelationAnnotator.createDataW= riterDescription( > LibLinearStringOutcomeDataWriter.class,//TKSVMlightStringO= utcomeDataWriter.class,// > // LIBLINEARStringOutcomeDataWriter.class, > new File(directory,EVENT_EVENT),=20 > params.probabilityOfKeepingANegativeExample)); > aggregateBuilder.add(EventDischargeTimeAnnotator.createDataWriterD= escription( > - // LibSvmStringOutcomeDataWriter.class, > - LibLinearStringOutcomeDataWriter.class, > + LibSvmStringOutcomeDataWriter.class, > +// LibLinearStringOutcomeDataWriter.class, > new File(directory,EVENT_DISCHARGE))); > aggregateBuilder.add(EventAdmissionTimeAnnotator.createDataWriterD= escription( > - // LibSvmStringOutcomeDataWriter.class, > - LibLinearStringOutcomeDataWriter.class, > + LibSvmStringOutcomeDataWriter.class, > +// LibLinearStringOutcomeDataWriter.class, > new File(directory,EVENT_ADMISSION))); > aggregateBuilder.add(ConsecutiveSentencesEventEventRelationAnnotat= or.createDataWriterDescription( > - // LibSvmStringOutcomeDataWriter.class, > - LibLinearStringOutcomeDataWriter.class, > + LibSvmStringOutcomeDataWriter.class, > +// LibLinearStringOutcomeDataWriter.class, > new File(directory,TEMP_CROSSSENT),=20 > params.probabilityOfKeepingANegativeExample)); > aggregateBuilder.add(ConsecutiveSentencesEventTimeRelationAnnotato= r.createDataWriterDescription( > - // LibSvmStringOutcomeDataWriter.class, > - LibLinearStringOutcomeDataWriter.class, > + LibSvmStringOutcomeDataWriter.class, > +// LibLinearStringOutcomeDataWriter.class, > new File(directory,TEMPET_CROSSSENT),=20 > params.probabilityOfKeepingANegativeExample)); > SimplePipeline.runPipeline(collectionReader, aggregateBuilder.crea= teAggregate()); > @@ -355,14 +360,14 @@ EvaluationOfTemporalRelations_ImplBase{ > } >=20 > // HideOutput hider =3D new HideOutput(); > - JarClassifierBuilder.trainAndPackage(new File(directory,EVENT_TI= ME), "-c", "0.001");//"-h","0","-c", "1000");//optArray);//"-c", "0.05");// > - JarClassifierBuilder.trainAndPackage(new File(directory,EVENT_EV= ENT), "-c", "0.001"); > - JarClassifierBuilder.trainAndPackage(new File(directory,EVENT_DI= SCHARGE), "-c", "0.001"); > - JarClassifierBuilder.trainAndPackage(new File(directory,EVENT_AD= MISSION), "-c", "0.001"); > + JarClassifierBuilder.trainAndPackage(new File(directory,EVENT_TI= ME), "-c", "0.0002", "-w2","0.2","-w3","7","-w4","7","-w5","13");//"-h","0"= ,"-c", "1000");//optArray);//"-c", "0.05");// > + JarClassifierBuilder.trainAndPackage(new File(directory,EVENT_EV= ENT), "-c", "0.0002","-w1","0.07","-w2","14","-w3","5","-w4","10","-w6","20= "); > + JarClassifierBuilder.trainAndPackage(new File(directory,EVENT_DI= SCHARGE), "-h","0","-c", "1000"); > + JarClassifierBuilder.trainAndPackage(new File(directory,EVENT_AD= MISSION), "-h","0","-c", "1000"); > // JarClassifierBuilder.trainAndPackage(new File(directory,= TIME_ADMISSION), "-h","0","-c", "1000"); > // JarClassifierBuilder.trainAndPackage(new File(directory,= TIME_DISCHARGE), "-h","0","-c", "1000"); > - JarClassifierBuilder.trainAndPackage(new File(directory,TEMP_CRO= SSSENT), "-c", "0.001"); > - JarClassifierBuilder.trainAndPackage(new File(directory,TEMPET_C= ROSSSENT), "-c", "0.001"); > + JarClassifierBuilder.trainAndPackage(new File(directory,TEMP_CRO= SSSENT), "-h","0","-c", "1000"); > + JarClassifierBuilder.trainAndPackage(new File(directory,TEMPET_C= ROSSSENT), "-h","0","-c", "1000"); > // hider.restoreOutput(); > // hider.close(); > } > @@ -419,8 +424,8 @@ EvaluationOfTemporalRelations_ImplBase{ > // aggregateBuilder.add(TimexDischargeTimeAnnotator.createA= nnotatorDescription(new File(directory,TIME_DISCHARGE))); >=20 > aggregateBuilder.add(this.baseline ? RecallBaselineEventTimeRelati= onAnnotator.createAnnotatorDescription(directory) : > - EventTimeSelfRelationAnnotator.createEngineDescription(new F= ile(directory,EVENT_TIME))); > - aggregateBuilder.add(EventEventRelationAnnotator.createAnnotator= Description(new File(directory,EVENT_EVENT))); > + EventTimeI2B2RelationAnnotator.createEngineDescription(new F= ile(directory,EVENT_TIME))); > + aggregateBuilder.add(EventEventI2B2RelationAnnotator.createAnnot= atorDescription(new File(directory,EVENT_EVENT))); > aggregateBuilder.add(ConsecutiveSentencesEventEventRelationAnnotat= or.createAnnotatorDescription(new File(directory,TEMP_CROSSSENT))); > aggregateBuilder.add(ConsecutiveSentencesEventTimeRelationAnnotato= r.createAnnotatorDescription(new File(directory,TEMPET_CROSSSENT))); > aggregateBuilder.add(AnalysisEngineFactory.createEngineDescription= (TemporalRelationRuleAnnotator.class)); >=20 >=20