kafka-jira mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Randall Hauch (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-5741) Prioritize threads in Connect distributed worker process
Date Wed, 06 Sep 2017 21:30:02 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-5741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16156062#comment-16156062

Randall Hauch commented on KAFKA-5741:

Thanks for replying, [~ewencp]. After looking into this, it's not clear to me that this is
an appropriate approach to take. Will investigate more whether this is actually a problem
or simply a symptom of another issue.

> Prioritize threads in Connect distributed worker process
> --------------------------------------------------------
>                 Key: KAFKA-5741
>                 URL: https://issues.apache.org/jira/browse/KAFKA-5741
>             Project: Kafka
>          Issue Type: Improvement
>          Components: KafkaConnect
>    Affects Versions:
>            Reporter: Randall Hauch
>            Priority: Critical
> Connect's distributed worker process uses the {{DistributedHerder}} to perform all administrative
operations, including: starting, stopping, pausing, resuming, reconfiguring connectors; rebalancing;
etc. The {{DistributedHerder}} uses a single threaded executor service to do all this work
and to do it sequentially. If this thread gets preempted for any reason (e.g., connector tasks
are bogging down the process, DoS, etc.), then the herder's membership in the group may be
dropped, causing a rebalance.
> This herder thread should be run at a much higher priority than all of the other threads
in the system.

This message was sent by Atlassian JIRA

View raw message