kafka-jira mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dustin Cote (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-6417) plugin.path pointing at a plugin directory causes ClassNotFoundException
Date Tue, 27 Mar 2018 19:43:00 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-6417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16416143#comment-16416143
] 

Dustin Cote commented on KAFKA-6417:
------------------------------------

[~ChrisEgerton] I think the problem with this is that in practice you could have so many jars
that also match this criteria that the log becomes chatty and the message gets missed (this
happens a lot with the ProducerConfig and ConsumerConfig warnings that pop out). There would
probably be a lot of false positives here. 

One thought I have is to enforce an easy to understand policy that no jars are allowed at
all directly underĀ {{plugin.path}} and just crash the worker saying so to "fail fast" in
the event someone isn't following convention. That would require uber jar users to make a
top level directory to contain their jar which may be a little annoying but would be easy
to understand. This doesn't seem like a severe penalty in general considering the real usage
pattern but would absolutely have compatibility issues with what exists out there today (on
upgrade you may see trouble). Alternatively, failing an individual connector might work too,
but I don't know how that would go in practice.

> plugin.path pointing at a plugin directory causes ClassNotFoundException
> ------------------------------------------------------------------------
>
>                 Key: KAFKA-6417
>                 URL: https://issues.apache.org/jira/browse/KAFKA-6417
>             Project: Kafka
>          Issue Type: Bug
>          Components: KafkaConnect
>    Affects Versions: 1.0.0
>            Reporter: Dustin Cote
>            Priority: Major
>
> When using the {{plugin.path}} configuration for the Connect workers, the user is expected
to specify a list containing the following per the docs:
> {quote}
> The list should consist of top level directories that include any combination of: a)
directories immediately containing jars with plugins and their dependencies b) uber-jars with
plugins and their dependencies c) directories immediately containing the package directory
structure of classes of plugins and their dependencies 
> {quote}
> This means we would expect {{plugin.path=/usr/share/plugins}} for a structure like {{/usr/share/plugins/myplugin1}},{{/usr/share/plugins/myplugin2}},
etc. However if you specify {{plugin.path=/usr/share/plugins/myplugin1}} the resulting behavior
is that dependencies for {{myplugin1}} are not properly loaded. This causes a {{ClassNotFoundException}}
that is not intuitive to debug. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message