kafka-jira mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Randall Hauch (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-6208) Reduce startup time for Kafka Connect workers
Date Fri, 09 Feb 2018 16:10:00 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-6208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16358600#comment-16358600

Randall Hauch commented on KAFKA-6208:

[~Satyajit], I would argue that anything with the scanning approach should be fixed as part
of KAFKA-6503, for which there is a simple fix. This issue, OTOH, is more about moving to
a non-scanning approach, and this requires a longer-term phase in of the new mechanism and
phase-out of the existing scanning mechanism.

Regarding your specific suggestion, I don't believe that loading the different components
asynchronously would be of any benefit over the much simpler suggestion in KAFKA-6503, which
already is going to utilize multiple threads.

> Reduce startup time for Kafka Connect workers
> ---------------------------------------------
>                 Key: KAFKA-6208
>                 URL: https://issues.apache.org/jira/browse/KAFKA-6208
>             Project: Kafka
>          Issue Type: Improvement
>          Components: KafkaConnect
>    Affects Versions: 1.0.0
>            Reporter: Randall Hauch
>            Priority: Major
> Kafka Connect startup times are excessive with a handful of connectors on the plugin
path or classpath. We should not be scanning three times (once for connectors, once for SMTs,
and once for converters), and hopefully we can avoid scanning directories that are clearly
not plugin directories. 
> We should also consider using Java's Service Loader to quickly identify connectors. The
latter would require a KIP and would require time to for connectors to migrate, but we could
be smarter about only scanning plugin directories that need to be scanned.

This message was sent by Atlassian JIRA

View raw message