kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ewen Cheslack-Postava (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (KAFKA-3487) KIP-146: Support per-connector/per-task classloaders in Connect
Date Thu, 18 May 2017 17:40:04 GMT

     [ https://issues.apache.org/jira/browse/KAFKA-3487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Ewen Cheslack-Postava updated KAFKA-3487:
-----------------------------------------
    Resolution: Fixed
        Status: Resolved  (was: Patch Available)

Issue resolved by pull request 3028
[https://github.com/apache/kafka/pull/3028]

> KIP-146: Support per-connector/per-task classloaders in Connect
> ---------------------------------------------------------------
>
>                 Key: KAFKA-3487
>                 URL: https://issues.apache.org/jira/browse/KAFKA-3487
>             Project: Kafka
>          Issue Type: New Feature
>          Components: KafkaConnect
>    Affects Versions: 0.10.0.0
>            Reporter: Ewen Cheslack-Postava
>            Assignee: Konstantine Karantasis
>            Priority: Critical
>              Labels: needs-kip
>             Fix For: 0.11.0.0
>
>
> Currently we just use the default ClassLoader in Connect. However, this limits how we
can compatibly load conflicting connector plugins. Ideally we would use a separate class loader
per connector/task that is instantiated to avoid potential conflicts.
> Note that this also opens up options for other ways to provide jars to instantiate connectors.
For example, Spark uses this to dynamically publish classes defined in the REPL and load them
via URL: https://ardoris.wordpress.com/2014/03/30/how-spark-does-class-loading/ But much simpler
examples (include URL in the connector class instead of just class name) are also possible
and could be a nice way to more support dynamic sets of connectors, multiple versions of the
same connector, etc.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message