Mailing-List: contact uima-user-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: uima-user@incubator.apache.org
Received-SPF: pass (herse.apache.org: domain of lally.adam@gmail.com
 designates 64.233.162.238 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=beta;
        h=received:message-id:date:from:sender:to:subject:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references:x-google-sender-auth;
        b=iyarvWjxjpVrqqiOqUJh+oWbh/EBJX6UFl39SHX/LwmkIjAYxZAg4uPWfQPOrOcCyCIQpR5gIxBbEMsRq7w/nrPnUKdGIBo+TRGfc5p7LKi7AhPOZuQCz4d2y5q+uVz1zYfHmnhPYzu2JBwow5Kwi25Z7/q2NTrmSLRAwUh3hKM=
Message-ID: <2787e08a0706270643i1f9e04fdp6929120db37b9236@mail.gmail.com>
Date: Wed, 27 Jun 2007 09:43:11 -0400
From: "Adam Lally" <alally@alum.rpi.edu>
Sender: lally.adam@gmail.com
To: uima-user@incubator.apache.org
Subject: Re: UIMA document loading strategy
In-Reply-To: <1268e4410706250348k2b9844a7m7c98828a2c419666@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
References: <1268e4410706250348k2b9844a7m7c98828a2c419666@mail.gmail.com>

On 6/25/07, Arthit Suriyawongkul <arthit@gmail.com> wrote:
> Hi,
>
> How UIMA load document to memory ?
> Does it load the whole document at once, or it only read document
> partially (sometime stream-like).
>
> Now I'm using GATE and sometimes got a problem if my document is very large,
> as GATE trying to load the whole document into the memory first and
> convert it to
> its own representation.
> My application doesn't need a knowledge of the whole document (like DOM),
> but only takes data from a small-size window (e.g. less than 100
> characters) at a time.
>
> cheers,
> Art
>

Hi Art,

UIMA is flexible with respect to this.  You can provide a
CollectionReader that populates a CAS with however much text is
appropriate for your application.  So a single document could be split
across many CASes in order to decrease the overall memory
requirements.

It's also possible to split a CAS into smaller CASes, do annotation on
each, and then merge the results.  The kind of component that does the
split and merge is called a "CAS Multiplier".  There's an example of
this in the uimaj-examples project that comes with the download - see
descriptors/cas_multiplier/Segment_Annotate_Merge_AE.  This is
described in the "CAS Multiplier Developer's Guide" section of the
documentation.

Another option is to consider using a "remote Sofa" (Sofa = subject of
analysis).  In this case the CAS just contains a URL to where the
actual document lives, not the document text itself.  See the
"Annotations, Artifacts, and Sofas" section of the documentaiton.

-Adam