Mailing-List: contact user-help@uima.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@uima.apache.org
Received-SPF: pass (athena.apache.org: domain of eaepstein@gmail.com
 designates 209.85.128.182 as permitted sender)
MIME-Version: 1.0
In-Reply-To: <5332DAB3.6030808@orkash.com>
References: <532BD841.2010705@orkash.com>
	<5332A701.6070301@orkash.com>
	<CAGRYgEO3jO8+vh-dpCg69_9ond6kSU8Jfj6UB+Aq2XWAqKYcuQ@mail.gmail.com>
	<5332DAB3.6030808@orkash.com>
Date: Wed, 26 Mar 2014 14:04:27 -0400
Message-ID: 
 <CAGRYgEMhGqzKkTSqwH1_bdTnvtXOe0VuOgA5m6PRJh8ve1TPBw@mail.gmail.com>
Subject: Re: Ducc Problems
From: Eddie Epstein <eaepstein@gmail.com>
To: user@uima.apache.org
Content-Type: multipart/alternative; boundary=20cf307ac141a57b8f04f5864cec

--20cf307ac141a57b8f04f5864cec
Content-Type: text/plain; charset=ISO-8859-1

Hi Reshu,

The collectionProcessingComplete() method in UIMA-AS has a limitation: a
Collection Processing Complete request sent to the UIMA-AS Analysis Service
is cascaded down to all delegates; however, if a particular delegate is
scaled-out, only one of the instances of the delegate will get this call.

Since DUCC is using UIMA-AS to scale out the Job processes, it has no way
to deliver a CPC to all instances.

The applications we have been running on DUCC have used the Work Item CAS
as a signal to CAS consumers to do CPC level processing. That is discussed
in the first reference above, in the paragraph "Flushing Cached Data".

Eddie


On Wed, Mar 26, 2014 at 9:48 AM, reshu.agarwal <reshu.agarwal@orkash.com>wrote:

> On 03/26/2014 06:43 PM, Eddie Epstein wrote:
>
>> Are you using standard UIMA interface code to Solr? If so, which Cas
>> Consumer?
>>
>> Taking at quick look at the source code for SolrCASConsumer, the batch and
>> collection process complete methods appear to do nothing.
>>
>> Thanks,
>> Eddie
>>
>>
>> On Wed, Mar 26, 2014 at 6:08 AM, reshu.agarwal <reshu.agarwal@orkash.com>
>> wrote:
>>
>>  On 03/21/2014 11:42 AM, reshu.agarwal wrote:
>>>
>>>  Hence we can not attempt batch processing in cas consumer and it
>>>> increases our process timing. Is there any other option for that or is
>>>> it a
>>>> bug in DUCC?
>>>>
>>>>  Please reply on this problem as if I am sending document in solr one by
>>> one by cas consumer without using batch process and committing solr. It
>>> is
>>> not optimum way to use this. Why ducc is not calling collection Process
>>> Complete method of Cas Consumer? And If I want to do that then What is
>>> the
>>> way to do this?
>>>
>>> I am not able to find any thing about this in DUCC book.
>>>
>>> Thanks in Advanced.
>>>
>>> --
>>> Thanks,
>>> Reshu Agarwal
>>>
>>>
>>>  Hi Eddie,
>
> I am not using standard UIMA interface code to Solr. I create my own Cas
> Consumer. I will take a look on that too. But the problem is not for
> particularly to use solr, I can use any source to store my output. I want
> to do batch processing and want to use collectionProcessComplete. Why DUCC
> is not calling it? I check it with UIMA AS also and my cas consumer is
> working fine with it and also performing batch processing.
>
> --
> Thanks,
> Reshu Agarwal
>
>

--20cf307ac141a57b8f04f5864cec--