Mailing-List: contact user-help@uima.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@uima.apache.org
Received-SPF: pass (athena.apache.org: domain of richard.eckart@gmail.com
 designates 74.125.83.48 as permitted sender)
Content-Type: text/plain; charset=windows-1252
Mime-Version: 1.0 (Mac OS X Mail 6.5 \(1508\))
Subject: Re: Using uima pipeline as an API 
From: Richard Eckart de Castilho <richard.eckart@gmail.com>
In-Reply-To: <loom.20130718T091255-263@post.gmane.org>
Date: Thu, 18 Jul 2013 09:54:54 +0200
Content-Transfer-Encoding: quoted-printable
Message-Id: <E74A2DC7-250E-4D59-B5D4-15F2DBDA5BD2@gmail.com>
References: <loom.20130718T091255-263@post.gmane.org>
To: user@uima.apache.org

> I have this particular requirement for a API that we wrap over a Uima=20=

> pipeline.
>=20
> public List<String> analyse(String inputFolderPath, String modelName);
>=20
> This method is supposed to accept a collection of files (residing in =
the=20
> inputFolderPath), run the files (as CAS) through a pipeline of UIMA =
AEs, and=20
> return the results (one String per CAS).
>=20
> To return the strings, I will need to somehow access the CAS after the =
AEs=20
> have finished their job and transform/extract whatever inside the CAS =
into=20
> the string that I will return to the caller of this method.
>=20
> But if I run the AEs using a SimplePipeline.runPipeline()
> How I can get hold of the CAS that are coming out of the AEs?
> Do I attach a CAS Consumer at the tail of the pipeline and read the =
CAS=20
> contents at that point? Then I append each result to the List<String> =
that I=20
> constructed at the begining.

You should take a look at the JCasIterable (cf. [1] - Example in Groovy, =
but
JCasIterable is a Java class and works nicely in Java too, just I have =
no=20
example in Java).

JCasIterable basically allows you to iterate over the CASes produced by =
your
pipeline. In such a look, you can extract and collect the data you need =
from
the CASes, e.g. putting into a List<String> and returning it. Mind that =
you
should *not* try to keep hold of full CASes, FeatureStructure (including
Annotations and stuff). You need to copy the data from the CAS, =
otherwise
it will be corrupted.

> If so, is this scalable?=20

Well=85 up to a point, but not in general.

> If I have thousands of files in the inputFolderPath, and if the =
strings are=20
> very large, would I run out of memory soon?
> Is there a more scalable way to do this?

You could write your strings to a file and then return an implementation =
of=20
List<String> which directly accesses the file. Depending on how much you =
want
to scale, you'll have to look into different solutions. The easiest =
would be
to buy more memory, the most complex would probably be porting your =
stuff to
some kind of cluster. The latter will most likely require a change of =
API,
possibly even of the whole processing paradigm. List<String> most =
probably
won't do then ;)

Cheers,

-- Richard


[1] =
http://code.google.com/p/dkpro-core-asl/wiki/GroovyRecipies#OpenNLP_Part-o=
f-speech_tagging_pipeline_using_JCasIterable_and_c=