uima-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eddie Epstein <eaepst...@gmail.com>
Subject Re: Ducc Problems
Date Wed, 26 Mar 2014 18:04:27 GMT
Hi Reshu,

The collectionProcessingComplete() method in UIMA-AS has a limitation: a
Collection Processing Complete request sent to the UIMA-AS Analysis Service
is cascaded down to all delegates; however, if a particular delegate is
scaled-out, only one of the instances of the delegate will get this call.

Since DUCC is using UIMA-AS to scale out the Job processes, it has no way
to deliver a CPC to all instances.

The applications we have been running on DUCC have used the Work Item CAS
as a signal to CAS consumers to do CPC level processing. That is discussed
in the first reference above, in the paragraph "Flushing Cached Data".

Eddie



On Wed, Mar 26, 2014 at 9:48 AM, reshu.agarwal <reshu.agarwal@orkash.com>wrote:

> On 03/26/2014 06:43 PM, Eddie Epstein wrote:
>
>> Are you using standard UIMA interface code to Solr? If so, which Cas
>> Consumer?
>>
>> Taking at quick look at the source code for SolrCASConsumer, the batch and
>> collection process complete methods appear to do nothing.
>>
>> Thanks,
>> Eddie
>>
>>
>> On Wed, Mar 26, 2014 at 6:08 AM, reshu.agarwal <reshu.agarwal@orkash.com>
>> wrote:
>>
>>  On 03/21/2014 11:42 AM, reshu.agarwal wrote:
>>>
>>>  Hence we can not attempt batch processing in cas consumer and it
>>>> increases our process timing. Is there any other option for that or is
>>>> it a
>>>> bug in DUCC?
>>>>
>>>>  Please reply on this problem as if I am sending document in solr one by
>>> one by cas consumer without using batch process and committing solr. It
>>> is
>>> not optimum way to use this. Why ducc is not calling collection Process
>>> Complete method of Cas Consumer? And If I want to do that then What is
>>> the
>>> way to do this?
>>>
>>> I am not able to find any thing about this in DUCC book.
>>>
>>> Thanks in Advanced.
>>>
>>> --
>>> Thanks,
>>> Reshu Agarwal
>>>
>>>
>>>  Hi Eddie,
>
> I am not using standard UIMA interface code to Solr. I create my own Cas
> Consumer. I will take a look on that too. But the problem is not for
> particularly to use solr, I can use any source to store my output. I want
> to do batch processing and want to use collectionProcessComplete. Why DUCC
> is not calling it? I check it with UIMA AS also and my cas consumer is
> working fine with it and also performing batch processing.
>
> --
> Thanks,
> Reshu Agarwal
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message