kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Guozhang Wang (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (KAFKA-2295) Dynamically loaded classes (encoders, etc.) may not be found by Kafka Producer
Date Fri, 03 Jul 2015 01:47:04 GMT

     [ https://issues.apache.org/jira/browse/KAFKA-2295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Guozhang Wang updated KAFKA-2295:
         Reviewer: Guozhang Wang
    Fix Version/s: 0.9.0

> Dynamically loaded classes (encoders, etc.) may not be found by Kafka Producer 
> -------------------------------------------------------------------------------
>                 Key: KAFKA-2295
>                 URL: https://issues.apache.org/jira/browse/KAFKA-2295
>             Project: Kafka
>          Issue Type: Bug
>          Components: producer 
>            Reporter: Tathagata Das
>            Assignee: Manikumar Reddy
>             Fix For: 0.9.0
>         Attachments: KAFKA-2295.patch
> Kafka Producer (via CoreUtils.createObject) effectively uses Class.forName to load encoder
classes. Class.forName is by design finds classes only in the defining classloader of the
enclosing class (which is often the bootstrap class loader). It does not use the current thread
context class loader. This can lead to problems in environments where classes are dynamically
loaded and therefore may not be present in the bootstrap classloader.
> This leads to ClassNotFound Exceptions in environments like Spark where classes are loaded
dynamically using custom classloaders. Issues like this have reported. E.g. - 
> https://www.mail-archive.com/user@spark.apache.org/msg30951.html
> Other references regarding this issue with Class.forName 
> http://stackoverflow.com/questions/21749741/though-my-class-was-loaded-class-forname-throws-classnotfoundexception
> This is a problem we have faced repeatedly in Apache Spark and we solved it by explicitly
specifying the class loader to use. See 
> https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/util/Utils.scala#L178

This message was sent by Atlassian JIRA

View raw message