Mailing-List: contact uima-user-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: uima-user@incubator.apache.org
Received-SPF: pass (athena.apache.org: local policy)
Message-Id: <6.2.1.2.2.20071112080719.02eec988@mail.llnl.gov>
Date: Mon, 12 Nov 2007 08:12:20 -0800
To: uima-user@incubator.apache.org,uima-user@incubator.apache.org
From: Steve Suppe <ssuppe@llnl.gov>
Subject: Re: (one other) Multithreading question
In-Reply-To: <4738425F.5040600@schor.com>
References: 
 <OF3422A941.DF8A4D3B-ONC2257390.002F8ED6-C2257390.002FF977@il.ibm.com>
 <4738111D.2030102@michael-baessler.de>
 <4738425F.5040600@schor.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"; format=flowed

A slightly different (but related question):

I've been playing around with this type of computation.  We are loading 
data into a DB.  We have a small Linux cluster (15 multi-core nodes at the 
moment) that we have scaled up to run 224+ instances of our pipeline.  I've 
noticed for most of our calculations, it's really Oracle that is holding us 
back.

In some instances, a 'computation' is classified as one 'medium to long' 
data pull from Oracle, a bunch of analysis, and then 'small to large' 
insertions of results.  I've dabbled in placing static DB connections and 
mutexes through the code to guarantee that the instances on a machine only 
access the DB one at a time, but are free to run analysis simultaneously 
otherwise.

I have also toyed with the idea of locks that allow N number of connections 
(instead of only  the mutual exclusion one at a time) so that I can 
increase the connections to a point, but not overload the system.

Has anyone tried anything like this?  Or is anyone else at least running a 
similar hardware set-up?  It would be great to compare note.

Thanks,
Steve

At 04:09 AM 11/12/2007, Marshall Schor wrote:
>This may not be quite precise enough.  Your Annotators will be
>instantiated multiple times,  so that a single *instance* of an
>annotator will not be run on multiple threads at once.  So - if you have
>non-static fields in your annotator, they do not need to be accessed
>with threading in mind. But if you make use of "static" fields, there is
>only one instance of these, so access to them must be thread-safe.
>
>If your *application*  (not your annotator) is multi-threaded, it will
>need to be thread-safe. You can find relevant information about this in
>the tutorial and reference docs for UIMA (search for "thread").
>
>-Marshall
>
>Michael Baessler wrote:
> > Benjamin Sznajder wrote:
> >> Hi all,
> >>
> >> I am interested in using multi-threading in UIMA.
> >> My aim is that the flow runs several annotators in parallel.
> >> One of my annotators is not thread-safe. My question is, then,
> >> Does the UIMA parallelism ( setting "MultipleDeployment=true") requires
> >> that annotators on which this flag is set, are thread-safe?
> >>
> >> Regards,
> >> Benjamin.
> >>
> >>
> > Yes, your annotator have to be thread-safe when you want to run them
> > multi-threaded.
> >
> > -- Michael
> >
> >