ctakes-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From vijay garla <vnga...@gmail.com>
Subject Re: FW: svn commit: r1551805 - /ctakes/branches/ytex/ctakes-assertion/src/main/java/org/apache/ctakes/assertion/medfacts/i2b2/api/CharacterOffsetToLineTokenConverterCtakesImpl.java
Date Thu, 19 Dec 2013 16:04:57 GMT
Hi Pei,

I'm not sure if that would solve the problem: change in the ytex branch
causes newlines to be ignored (i.e. not treated as a token).  trunk's
sentence splitter is splits sentences on newlines, so newlines would never
be found in a sentence.  However, if we had a reproducer we could check it
fairly easily in the ytex branch.

Best,

VJ


On Thu, Dec 19, 2013 at 10:15 AM, Chen, Pei
<Pei.Chen@childrens.harvard.edu>wrote:

> Vj,
> Do you think this is what was causing the NPE's [1]?
> If so, shall we make the same fix in trunk?
> --Pei
>
> [1]
> http://mail-archives.apache.org/mod_mbox/ctakes-dev/201309.mbox/%3C924DE05C19409B438EB81DE683A942D9105A93CB%40CHEXMBX1A.CHBOSTON.ORG%3E
>
> -----Original Message-----
> From: vjapache@apache.org [mailto:vjapache@apache.org]
> Sent: Tuesday, December 17, 2013 9:15 PM
> To: commits@ctakes.apache.org
> Subject: svn commit: r1551805 -
> /ctakes/branches/ytex/ctakes-assertion/src/main/java/org/apache/ctakes/assertion/medfacts/i2b2/api/CharacterOffsetToLineTokenConverterCtakesImpl.java
>
> Author: vjapache
> Date: Wed Dec 18 02:14:13 2013
> New Revision: 1551805
>
> URL: http://svn.apache.org/r1551805
> Log:
> add support for sentences that contain newline tokens.
>
> Modified:
>
> ctakes/branches/ytex/ctakes-assertion/src/main/java/org/apache/ctakes/assertion/medfacts/i2b2/api/CharacterOffsetToLineTokenConverterCtakesImpl.java
>
> Modified:
> ctakes/branches/ytex/ctakes-assertion/src/main/java/org/apache/ctakes/assertion/medfacts/i2b2/api/CharacterOffsetToLineTokenConverterCtakesImpl.java
> URL:
> http://svn.apache.org/viewvc/ctakes/branches/ytex/ctakes-assertion/src/main/java/org/apache/ctakes/assertion/medfacts/i2b2/api/CharacterOffsetToLineTokenConverterCtakesImpl.java?rev=1551805&r1=1551804&r2=1551805&view=diff
>
> ==============================================================================
> ---
> ctakes/branches/ytex/ctakes-assertion/src/main/java/org/apache/ctakes/assertion/medfacts/i2b2/api/CharacterOffsetToLineTokenConverterCtakesImpl.java
> (original)
> +++ ctakes/branches/ytex/ctakes-assertion/src/main/java/org/apache/ctake
> +++ s/assertion/medfacts/i2b2/api/CharacterOffsetToLineTokenConverterCta
> +++ kesImpl.java Wed Dec 18 02:14:13 2013
> @@ -32,8 +32,8 @@ import org.apache.uima.jcas.tcas.Annotat  import
> org.mitre.medfacts.i2b2.api.ApiConcept;
>  import org.mitre.medfacts.zoner.CharacterOffsetToLineTokenConverter;
>  import org.mitre.medfacts.zoner.LineAndTokenPosition;
> -
>  import org.apache.ctakes.typesystem.type.syntax.BaseToken;
> +import org.apache.ctakes.typesystem.type.syntax.NewlineToken;
>  import org.apache.ctakes.typesystem.type.textspan.Sentence;
>
>  public class CharacterOffsetToLineTokenConverterCtakesImpl implements
> CharacterOffsetToLineTokenConverter
> @@ -78,11 +78,13 @@ public class CharacterOffsetToLineTokenC
>           for (Annotation current : annotationIndex)
>           {
>                   BaseToken bt = (BaseToken)current;
> -                 int begin = bt.getBegin();
> -                 int end = bt.getEnd();
> -
> -                 tokenBeginEndTreeSet.add(begin);
> -                 tokenBeginEndTreeSet.add(end);
> +                 // filter out NewlineToken
> +                 if (!(bt instanceof NewlineToken)) {
> +                         int begin = bt.getBegin();
> +                         int end = bt.getEnd();
> +                         tokenBeginEndTreeSet.add(begin);
> +                         tokenBeginEndTreeSet.add(end);
> +                 }
>           }
>    }
>
>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message