Hi Reshu,
The collectionProcessingComplete() method in UIMA-AS has a limitation: a
Collection Processing Complete request sent to the UIMA-AS Analysis Service
is cascaded down to all delegates; however, if a particular delegate is
scaled-out, only one of the instances of the delegate will get this call.
Since DUCC is using UIMA-AS to scale out the Job processes, it has no way
to deliver a CPC to all instances.
The applications we have been running on DUCC have used the Work Item CAS
as a signal to CAS consumers to do CPC level processing. That is discussed
in the first reference above, in the paragraph "Flushing Cached Data".
Eddie
On Wed, Mar 26, 2014 at 9:48 AM, reshu.agarwal <reshu.agarwal@orkash.com>wrote:
> On 03/26/2014 06:43 PM, Eddie Epstein wrote:
>
>> Are you using standard UIMA interface code to Solr? If so, which Cas
>> Consumer?
>>
>> Taking at quick look at the source code for SolrCASConsumer, the batch and
>> collection process complete methods appear to do nothing.
>>
>> Thanks,
>> Eddie
>>
>>
>> On Wed, Mar 26, 2014 at 6:08 AM, reshu.agarwal <reshu.agarwal@orkash.com>
>> wrote:
>>
>> On 03/21/2014 11:42 AM, reshu.agarwal wrote:
>>>
>>> Hence we can not attempt batch processing in cas consumer and it
>>>> increases our process timing. Is there any other option for that or is
>>>> it a
>>>> bug in DUCC?
>>>>
>>>> Please reply on this problem as if I am sending document in solr one by
>>> one by cas consumer without using batch process and committing solr. It
>>> is
>>> not optimum way to use this. Why ducc is not calling collection Process
>>> Complete method of Cas Consumer? And If I want to do that then What is
>>> the
>>> way to do this?
>>>
>>> I am not able to find any thing about this in DUCC book.
>>>
>>> Thanks in Advanced.
>>>
>>> --
>>> Thanks,
>>> Reshu Agarwal
>>>
>>>
>>> Hi Eddie,
>
> I am not using standard UIMA interface code to Solr. I create my own Cas
> Consumer. I will take a look on that too. But the problem is not for
> particularly to use solr, I can use any source to store my output. I want
> to do batch processing and want to use collectionProcessComplete. Why DUCC
> is not calling it? I check it with UIMA AS also and my cas consumer is
> working fine with it and also performing batch processing.
>
> --
> Thanks,
> Reshu Agarwal
>
>
|