uima-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Burn Lewis" <burnle...@gmail.com>
Subject Re: UimaASProcessCasTimeout exception hangs application
Date Tue, 07 Oct 2008 21:36:01 GMT
I meant that if you used:

runRemoteAsyncAE tcp://localhost:61616 RoomNumberAnnotatorQueue –c
<DIR>FileSystemCollectionReader.xml -p 1 -t 6 -i

then CASes would be sent one at a time (-p 1) and the 6 sec timeout would
never be reached since each CAS takes 5 secs to process.  But with a larger
CAS pool, say -p 3, the -t value would have to be > 3 x 5, say -p 16, to
ensure that the 3rd CAS does not timeout while waiting for the 1st two to
complete.  All this just to avoid errors since as Jerry said there is a bug
in the handling of errors when -i is specified.

Yes, a crash in a service looks just the same as a slow service ... no
result returned in the allotted time.  Timeouts are usually used to prevent
the application from hanging when a service dies, so are often set much
larger than the expected processing time to avoid abandoning those CASes
that take longer than the average.

When your application is an asynchronous aggregate with multiple remote
delegates, you can set queue-specific timeouts in the deployment descriptor,
e.g.

          <remoteAnalysisEngine key="RoomNumber">
            <inputQueue endpoint="RoomNumberAnnotatorQueue"
brokerURL="tcp://localhost:61616"/>
            <serializer method="xmi"/>
            <asyncAggregateErrorConfiguration>
              <getMetadataErrors maxRetries="0" timeout="60000"
errorAction="disable"/>
              <processCasErrors maxRetries="0" timeout="300000"
continueOnRetryFailure="true" thresholdCount="1" thresholdWindow="0"
thresholdAction="disable"/>
              <collectionProcessCompleteErrors timeout="60000"
additionalErrorAction="terminate"/>
            </asyncAggregateErrorConfiguration>
          </remoteAnalysisEngine>

See section 3.10 in the uima_async_scaleout document.  You can use the CDE
in Eclipse to edit deployment descriptors.

- Burn.

On Tue, Oct 7, 2008 at 4:22 PM, Charles Proefrock <chas.pro@hotmail.com>wrote:

> I don't have ready access to the test environment, so I'll have to go back
> and play with the -t value.  It isn't clear if you are saying that a longer
> timeout "will succeed" in avoiding the hanging problem, or "will succeed" in
> repeating the problem we've observed.  Clarification on that would help,
> especially if some min or max timeout avoids the problem.
>
> Yes, the queue name should match the name configured in the deploy
> descriptor.  I think we had a more complex example running and then
> simplified it before posting, so the queue name documented here was probably
> just a mistype.
> To be a little more specific, we are testing both the natural timeout when
> a deployed process runs too long, and also the scenario where a deployed
> process crashes/exits with ^C.  Both seem to generate the same exception
> when the client timeout occurs, so the nature of the remote failure does not
> seem to be in play.  This makes sense since the remote is on the other side
> of a message queue, the client does not see a direct connection dropped on
> the crash.  It must also mean that the remote is not connected to the
> anything while it is running, so nothing detects the crash itself.
>  Confirmation of this understanding would help as well. Are there any
> options on the message queue (initiation or response queues) that would
> allow for configuring timeouts for specific named queues?
> - Charles
>
>
>
>
> > Date: Tue, 7 Oct 2008 10:10:21 -0400> From: burnlewis@gmail.com> To:
> uima-user@incubator.apache.org> Subject: Re: UimaASProcessCasTimeout
> exception hangs application> > I also see that the -t process timeout value
> has to be > 10 for this to> succeed ... I'd have assumed anything > 5 secs
> would be safe, but the> default value of -p 2 means that the client's
> timeout must be 2x the service> process time when 2 CASes are in play, since
> the 2nd is queued immediately> behind the first. Perhaps the default should
> be -p 1 for simplicity.> > P.S. I assume you meant RoomNumberAnnotatorQueue
> as the 2nd arg in step 4.> > On Tue, Oct 7, 2008 at 9:34 AM, Jaroslaw
> Cwiklik <cwiklik@us.ibm.com> wrote:> > > Charles, this appears to be a
bug
> in the UIMA AS client code. The only work> > around that comes to mind right
> now> > is to run without -i. Without -i the runRemoteAE should terminate on>
> > exception.> >> > We will investigate this problem and post a patch on
> apache.> >> > Thanks> >> > Jerry> >
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++> > Jerry
> Cwiklik> > UIMA Extensions> > IBM T.J. Watson Research Center> > Hawtorne,
> NY, 10532> > Tel: 914-784-7665, T/L: 863-7665> > Email: cwiklik@us.ibm.com>
> >> > [image: Inactive hide details for Charles Proefrock <chas.pro@
> hotmail.com>]Charles> > Proefrock <chas.pro@hotmail.com>> >>
>> >> >
> *Charles Proefrock <chas.pro@hotmail.com>*> >> > 10/06/2008 05:59 PM>
>
> Please respond to> > uima-user@incubator.apache.org> >> >> >
To> >> > UIMA
> User <uima-user@incubator.apache.org>> > cc> >> >> > Subject>
>> >
> UimaASProcessCasTimeout exception hangs application> >> >> >> >
When
> experimenting with the UIMA-AS examples and how the error handling> >
> mechanisms work in terms of timeouts due to AS Aggregates taking too long
> or> > going offline, we came across a situation in which the
> RunRemoteAsyncAE> > hangs and never returns after receiving and processing
> a> > UimaASProcessCasTimeout exception. Ultimately, we simply want the
> system to> > recover and try the next CAS or return gracefully without
> having to call a> > hard System.exit(1).> >> > Our tests are based
on the
> Deploy_MeetingDetectorTAE_RemoteRoomNumber.xml> > example. All we did was
> the following:> > (1) added "Thread.sleep(5000);" to
> RoomNumberAnnotator.java process to> > simulate a longer process.> > (2)
> Executed: startBroker.bat> > (3) Executed: deployAsyncService
> <DIR>\Deploy_RoomNumberAnnotator.xml> > (4) Executed: runRemoteAsyncAE
> tcp://localhost:61616> > MeetingDetectorTaeQueue –c
> <DIR>\FileSystemCollectionReader.xml -t 4 -i> >> > In summary: when
setting
> and triggering the time outs on the RoomNumber> > remoteAnalysisEngine, the
> exception is thrown and caught, each additional> > CAS is tried, and the
> system exists gracefully. If the> > UimaAsynchronousEngine within the
> runRemoteAsyncAE times out, then the> > system hangs and never returns.> >>
> > Is there a trick that we're missing? Is this expected?> >> > - Charles>
>>
> >> > _________________________________________________________________> >
> See how Windows Mobile brings your life together—at home, work, or on the> >
> go.> > http://clk.atdmt.com/MRT/go/msnnkwxp1020093182mrt/direct/01/> >
> _________________________________________________________________
> See how Windows Mobile brings your life together—at home, work, or on the
> go.
> http://clk.atdmt.com/MRT/go/msnnkwxp1020093182mrt/direct/01/
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message