flume-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "dengkai (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLUME-3044) KafkaSink should avoid to call method without timeout param
Date Sun, 22 Jan 2017 09:28:26 GMT

    [ https://issues.apache.org/jira/browse/FLUME-3044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15833410#comment-15833410

dengkai commented on FLUME-3044:

Review Board request created.

> KafkaSink should avoid to call method without timeout param
> -----------------------------------------------------------
>                 Key: FLUME-3044
>                 URL: https://issues.apache.org/jira/browse/FLUME-3044
>             Project: Flume
>          Issue Type: Improvement
>          Components: Sinks+Sources
>    Affects Versions: v1.7.0
>            Reporter: dengkai
>            Assignee: dengkai
>         Attachments: FLUME-3044.patch
> There are several method call in kafka sink with no timeout params, in some cases, kafka
sink will await forever if no interruption. 
> For example´╝î
> case 1 : When we stop kafka sink, it will call producer.close() without timeout param.
This will cause another close method in producer with timeout param as (Long.MAX_VALUE, TimeUnit.MILLISECONDS).
Then it will wait producer's IOThread exited, if the IOThread can't exit normally,  kafka
sink will block all the time and there is no effect of kill -TERM.
> case 2 : When an uncaught exception happened such as Correlation ID does not match in
Kafka Producer Sender thread, the sender will lost this request. If we call producer.flush(),
there is a await() without timeout param, kafka sink will block all the time except restart
> case 3 : Like case 2, when we call future.get() without timeout param but the request
in Kafka Producer lost, kafka sink will also block all the time.

This message was sent by Atlassian JIRA

View raw message