Mailing-List: contact user-help@uima.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@uima.apache.org
Received-SPF: pass (athena.apache.org: local policy)
Message-ID: <4F9AFABC.4020503@cs.cmu.edu>
Date: Fri, 27 Apr 2012 15:59:56 -0400
From: Eric Riebling <er1k@cs.cmu.edu>
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64;
 rv:11.0) Gecko/20120327 Thunderbird/11.0.1
MIME-Version: 1.0
To: user@uima.apache.org
Subject: Re: Running UIMA on a cluster
References: <CBC058B1.8754%ozborn@uab.edu>
In-Reply-To: <CBC058B1.8754%ozborn@uab.edu>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

We've had success deploying annotators on cluster nodes (using UIMA-AS
deployment descriptors) registered to a UIMA-AS broker running on the
head node.  If the cluster use shared data folders, you only need to
put the code in one place for it to 'appear' on all nodes.

Then we run a collection reader and CAS consumer on the head node,
with the amount of scale-out specified on the command line of
runRemoteAsyncAE.sh, something like this:

   $UIMA_HOME/bin/runRemoteAsyncAE.sh -c (path.to)XmiCollectionReader.xml tcp://localhost:6
1616 (name of deployed service) -p (number of nodes) -o output_foldername

With enough scale-out, the limiting factor becomes the speed of the CR
and CC on the head node.  This is the briefest explanation I can give,
not sure it's a 'best practice' but it works. :)

On 4/27/2012 3:35 PM, John David Osborne wrote:
> Hello,
>
> Is there any best practice documentation out there for running
> UIMA/UIMA-AS on a cluster? I have only run single machine instances of
> UIMA (mostly through Eclipse) and have not investigated the ability to
> perform multiple simultaneous analyses in order to process large document
> collections.
>
> It's not clear to me how UIMA would operate in a cluster environment, do
> people really do message passing using JMI? I'm guessing this is the case
> as I seeing references to MPICH, SGE or other things I am more used to.
> I've looked through some of the documentation (including all the Overview
> &  SDK setup) but am not finding anything helpful. I've also tried googling
> but I am not getting much except this:
> http://comments.gmane.org/gmane.comp.apache.uima.general/2131 which makes
> me think it is possible.
>
> Currently with my level of confusion I think it may be best to have
> multiple instances of UIMA on a cluster and just submit jobs processing
> discrete document sets to our SGE cluster and ignore whatever scaling
> features are actually present in UIMA since the document processing I plan
> to do is data parallel.
>
> -John
>
>

-- 
Eric Riebling                 Senior Systems Programmer
http://ericriebling.com       CMU Language Technologies Institute