flume-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jeff Holoman (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLUME-3044) KafkaSink should avoid to call method without timeout param
Date Sat, 04 Feb 2017 03:59:52 GMT

    [ https://issues.apache.org/jira/browse/FLUME-3044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15852565#comment-15852565

Jeff Holoman commented on FLUME-3044:

Thanks. I will take a look this weekend.


> KafkaSink should avoid to call method without timeout param
> -----------------------------------------------------------
>                 Key: FLUME-3044
>                 URL: https://issues.apache.org/jira/browse/FLUME-3044
>             Project: Flume
>          Issue Type: Improvement
>          Components: Sinks+Sources
>    Affects Versions: v1.7.0
>            Reporter: dengkai
>            Assignee: dengkai
>         Attachments: FLUME-3044.patch
> There are several method call in kafka sink with no timeout params, in some cases, kafka
sink will await forever if no interruption. 
> For example´╝î
> case 1 : When we stop kafka sink, it will call producer.close() without timeout param.
This will cause another close method in producer with timeout param as (Long.MAX_VALUE, TimeUnit.MILLISECONDS).
Then it will wait producer's IOThread exited, if the IOThread can't exit normally,  kafka
sink will block all the time and there is no effect of kill -TERM.
> case 2 : When an uncaught exception happened such as Correlation ID does not match in
Kafka Producer Sender thread, the sender will lost this request. If we call producer.flush(),
there is a await() without timeout param, kafka sink will block all the time except restart
> case 3 : Like case 2, when we call future.get() without timeout param but the request
in Kafka Producer lost, kafka sink will also block all the time.

This message was sent by Atlassian JIRA

View raw message