beam-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jean-Baptiste Onofré ...@nanthrax.net>
Subject Re: Handling errors in IOs
Date Sun, 11 Feb 2018 06:56:26 GMT
Hi Motty,

For JMS, it depends if you are using queues or topics.

Using queues, JmsIO create several readers (concurrent consumer) on the same
queue. The checkpoint used is based on the ACK (it's a client ACK, and the
source send the ACK when the checkpoint is finalized). If you close a connection
for one source, the other sources should continue to consume.

Can you explain exactly your scenario (runner, pipeline, broker) ?

Regards
JB

On 02/11/2018 07:43 AM, Motty Gruda wrote:
> Hey,
> 
> How errors in the IOs can be treated (for example connection errors)? 
> 
> I've tested few scenarios with the JmsIO. When I read from two different jms
> connections and I closed only one of them, the entire pipeline failed/froze.
> I would expect it to continue running with one source and try to reconnect to
> the second source until it's available again.
> 
> Is this a bug in the IO itself? In the SDK? In the runner (I've tested with the
> direct runner and the spark runner)?

-- 
Jean-Baptiste Onofré
jbonofre@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com

Mime
View raw message