camel-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Christian Schneider <ch...@die-schneider.net>
Subject Re: long-running requests over jms
Date Wed, 22 Dec 2010 07:37:47 GMT
I think on JMS you never can be sure that the consumer is working on a 
request. (When using simple request / reply).

So my suggestion is to simply offer a decoupled service that can be 
asked about the requests the consumer processes at the moment. I am not 
sure if this can be generically supported by camel.
Perhaps camel could offer a way to look into a route and see which 
requests it processes via JMX. As this would be quite camel specific I 
guess the better way would be to write the monitoring service on your
own.

Another simple solution would be to let the consumer write keep alive 
messages an a separate topic that also contain information on the 
requests it works on. So you could also monitor the progress.

The good thing in writing this on your own is that you could even send a 
progress in percent if this is possible in your case. Then the client 
could display how far the processing went.

Best regards

Christian


Am 22.12.2010 00:49, schrieb Jim Newsham:
>
> Thanks for the responses Claus and Willem.  I couldn't find a JIRA 
> issue related to jms exchange store, etc.  I opened a new JIRA issue 
> for long-running jms requests:  
> https://issues.apache.org/jira/browse/CAMEL-3456.
>
> I'm trying to understand the pluggable store you're referring to.  I 
> think what you're saying is that the pluggable store is purely 
> consumer-side, where the consumer would notice a long-running request, 
> park the exchange in the store, and release the consumer thread, then 
> when the result becomes available, it would pick up the exchange again 
> and send it on.  Did I understand correctly?  And the purpose would be 
> to free the consumer thread?
>
> I can see the potential problem that if a lot of long-running requests 
> are made, some thread pool might be exhausted.  For our own purposes, 
> I'm not too worried about this problem as the long-running requests 
> will be rare and exceptional cases and we have small enough load and 
> can afford to configure a large enough thread pool so that it's not an 
> issue.  So for us, the primary issue is that there's no way for the 
> producer to be aware that the consumer is still processing the request 
> (since setting requestTimeout to a large value is not a good idea).
>
> Regards,
> Jim
>
> On 12/21/2010 4:43 AM, Claus Ibsen wrote:
>> Hi
>>
>> We have some thoughts of letting camel-jms support using a pluggable
>> store so it can persist the in flight Exchange. And then on the reply
>> consumer side, it can peek in the store to find the correlated
>> Exchange and load it from the store, and continue routing the
>> Exchange.
>>
>> Then we can support long use-cases. Also if Camel crashes or you 
>> shutdown Camel.
>>
>> I think we have created a JIRA ticket for it, I may be mistaken. Fell
>> free to see if you can dig out the JIRA.
>> If not create a new one and refer to this thread (using nabble etc.)
>>
>>
>> On Tue, Dec 21, 2010 at 12:27 AM, Jim 
>> Newsham<jnewsham@referentia.com>  wrote:
>>> Hi everyone,
>>>
>>> We are using Camel + ActiveMQ, with InOut messages and bean() 
>>> routes, as a
>>> form of flexible remoting (remote service invocation).  This has been
>>> working out quite well for us so far.  One issue that we've run into 
>>> is that
>>> while most service requests complete very quickly, some particular 
>>> service
>>> requests can take a long time to execute (due to the processing they 
>>> must
>>> perform) -- perhaps many minutes, or in extreme cases possibly an 
>>> hour or
>>> more.  However, if the request exceeds the configured jms 
>>> "requestTimeout"
>>> parameter, then the requester will receive a timeout exception.
>>>
>>> I feel that the requestTimeout parameter alone is not flexible 
>>> enough to do
>>> what we need.  requestTimeout should be a somewhat small value so 
>>> that the
>>> application is responsive to disconnects (we certainly can't set it 
>>> to an
>>> hour; the 20 sec default seems reasonable).  By contrast, there 
>>> doesn't seem
>>> to be a very well-defined, reasonable upper bound on the long-running
>>> requests -- they could take an hour, perhaps more.
>>>
>>> I feel that what we need to ensure responsiveness while supporting
>>> long-running requests is some form of periodic, pending request 
>>> heartbeat.
>>>   Coding this on an ad-hoc basis per request would be tedious and 
>>> cumbersome.
>>>   It would be great if I could set a "requestTimeout" on the producer
>>> endpoint (let's say 20s), and configure the consumer endpoint with a
>>> heartbeat "requestKeepalive" parameter (let's say 15s), and the 
>>> consumer
>>> would send periodic messages to the producer which would reset its 
>>> timeout
>>> counter, until the consumer finally sends the result.
>>>
>>> What do you think?  Is such a proposal feasible?  Any alternative 
>>> ideas?  I
>>> took a look at the jms component code, but I'm not quite sure where the
>>> producer timeout happens, or where a consumer keepalive processor 
>>> would go.
>>>
>>> Thanks,
>>> Jim
>>>
>>
>>
>
>

-- 
----
http://www.liquid-reality.de


Mime
View raw message