Mailing-List: contact user-help@uima.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@uima.apache.org
Received-SPF: pass (nike.apache.org: local policy)
From: Thomas Ginter <thomas.ginter@utah.edu>
To: "user@uima.apache.org" <user@uima.apache.org>
Subject: Re: FilteredIterator is very slow
Thread-Topic: FilteredIterator is very slow
Thread-Index: Ac9NCVioa1j39iBuQf+mmqoaQdV18wARDwQA
Date: Mon, 31 Mar 2014 19:56:23 +0000
Message-ID: <68C2DF13-2495-4714-A69A-6B6971193C99@utah.edu>
References: 
 <BA79DB2185486945B9DA903834E5A0964C5AFF00@DDCEP50015.na.corp.mckesson.com>
In-Reply-To: 
 <BA79DB2185486945B9DA903834E5A0964C5AFF00@DDCEP50015.na.corp.mckesson.com>
Accept-Language: en-US
Content-Language: en-US
Content-Type: text/plain; charset="us-ascii"
Content-ID: <47E67E2E83BED34DB331E477843DC89C@umail.utah.edu>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0

Larry,

A faster way to get the list of types that you will skip would be to do the=
 following:

FSIndex<TitlePersonHonorificAnnotation> titlePersonHAIndex =3D aJCas.getAnn=
otationIndex(TitlePersonHonorificAnnotation.type);

Doing this for each type will yield an index that points to just the annota=
tions in the CAS of each type you are interested in.  From there you can ge=
t an iterator reference ( titlePersonHAIndex.iterator() ) and either traver=
se each one separately or else add them to a common Collection such as an A=
rrayList and iterate through that.  You could also take advantage of the fa=
ct that the default index in UIMA sorts on ascending order on the begin ind=
ex and descending order on the ending index to stop once you have traversed=
 the list past the ending index of the dictTerm. =20

An important design decision though would be to consider whether the dictTe=
rm annotations are much more numerous than the TitlePersonHonorificAnnotati=
on, MeasurementAnnotation, and ProgFactorTerm filtering annotation types.  =
Generally if the filter types are much more plentiful and the dictTerm type=
 was more rare then looking for overlapping filter types will yield fewer i=
terations of your algorithm, however if there are a lot of dictTerm occurre=
nces and only a few of the filter types then it may be more efficient to it=
erate through the filter types and eliminate dictTerms that overlap or are =
covered. =20

Thanks,

Thomas Ginter
801-448-7676
thomas.ginter@utah.edu


On Mar 31, 2014, at 11:47 AM, Kline, Larry <Larry.Kline@mckesson.com> wrote=
:

> When I use a filtered FSIterator it's an order of magnitude slower than a=
 non-filtered iterator.  Here's my code:
>=20
> Create the iterator:
>       private FSIterator<Annotation> createConstrainedIterator(JCas aJCas=
) throws CASException {
>              FSIterator<Annotation> it =3D aJCas.getAnnotationIndex().ite=
rator();
>              FSTypeConstraint constraint =3D aJCas.getConstraintFactory()=
.createTypeConstraint();
>              constraint.add((new TitlePersonHonorificAnnotation(aJCas)).g=
etType());
>              constraint.add((new MeasurementAnnotation(aJCas)).getType())=
;
>              constraint.add((new ProgFactorTerm(aJCas)).getType());
>              it =3D aJCas.createFilteredIterator(it, constraint);
>              return it;
>       }
> Use the iterator:
>       public void process(JCas aJCas) throws AnalysisEngineProcessExcepti=
on {
>              ...
> // The following is done in a loop
>                           if (shouldSkip(dictTerm, skipIter))
>                                  continue;
>              ...
>       }
> Here's the method called:
>       private boolean shouldSkip(G2DictTerm dictTerm, FSIterator<Annotati=
on> skipIter) throws CASException {
>              boolean shouldSkip =3D false;
>              skipIter.moveToFirst();
>              while (skipIter.hasNext()) {
>                     Annotation annotation =3D skipIter.next();
>                     if (UIMAUtils.annotationsOverlap(dictTerm, annotation=
)) {
>                           shouldSkip =3D true;
>                           break;
>                     }
>              }
>              return shouldSkip;
>       }
>=20
> If I change the method, createConstrainedIterator(), to this (that is, no=
 constraints):
>       private FSIterator<Annotation> createConstrainedIterator(JCas aJCas=
) throws CASException {
>              FSIterator<Annotation> it =3D aJCas.getAnnotationIndex().ite=
rator();
>              return it;
>       }
>=20
> It runs literally 10 times faster.  Doing some profiling I see that all o=
f the time is spent in the skipIter.moveToFirst() call.  I also tried creat=
ing the filtered iterator each time anew in the shouldSkip() method instead=
 of passing it in, but that has even slightly worse performance.
>=20
> Given this performance I suppose I should probably use a non-filtered ite=
rator and just check for the types I'm interested in inside the loop.
>=20
> Any other suggestions welcome.
>=20
> Thanks,
> Larry Kline
>=20
>=20