flume-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Endre Major (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLUME-3274) KafkaSource and JMSSource stucks at startup connection exception
Date Thu, 04 Oct 2018 14:43:00 GMT

    [ https://issues.apache.org/jira/browse/FLUME-3274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16638313#comment-16638313
] 

Endre Major commented on FLUME-3274:
------------------------------------

The JMSSource only does JMS related things in the startup that it does not retry in the normal
doProcess() operation. The consumer initialization is handled during the normal doProcess()
call. So the error handling seems appropriate, the errors that happen during startup cannot
be recovered later during processing events.

> KafkaSource and JMSSource stucks at startup connection exception
> ----------------------------------------------------------------
>
>                 Key: FLUME-3274
>                 URL: https://issues.apache.org/jira/browse/FLUME-3274
>             Project: Flume
>          Issue Type: Bug
>            Reporter: Endre Major
>            Assignee: Endre Major
>            Priority: Major
>
> This problem can happen in sources that are derived from BasicSourceSemantics. BasicSourceSemantics
at startup if an exception occurs, it stores it. This behavior only makes sense if the start
only throws an exception for unrecoverable configuration problems. Both KafkaSource and JMSSource
try to connect to a Kafka or JMS server at startup, if the server is not available the flume
agent needs to be restarted. They both can recover after connection errors after a successful
startup.
> The start() should only throw an exception for unrecoverable problems.
> We need to make sure that the reconnect mechanism is safe and even if there is some configuration
problem it does not start  a DOS attack.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@flume.apache.org
For additional commands, e-mail: issues-help@flume.apache.org


Mime
View raw message