flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-6965) Avro is missing snappy dependency
Date Wed, 21 Jun 2017 16:27:00 GMT

    [ https://issues.apache.org/jira/browse/FLINK-6965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16057789#comment-16057789
] 

ASF GitHub Bot commented on FLINK-6965:
---------------------------------------

Github user tzulitai commented on a diff in the pull request:

    https://github.com/apache/flink/pull/4160#discussion_r123298637
  
    --- Diff: flink-core/pom.xml ---
    @@ -79,13 +79,12 @@ under the License.
     		<dependency>
     			<groupId>org.apache.avro</groupId>
     			<artifactId>avro</artifactId>
    -			<!-- managed version -->
    -			<exclusions>
    -				<exclusion>
    -					<groupId>org.xerial.snappy</groupId>
    -					<artifactId>snappy-java</artifactId>
    -				</exclusion>
    -			</exclusions>
    +		</dependency>
    +
    +		<!-- We explicitly depend on snappy since connectors that require it load it through
the system class loader -->
    --- End diff --
    
    Side question: is a fix also required for connectors to correctly use the user classloader
in such cases?


> Avro is missing snappy dependency
> ---------------------------------
>
>                 Key: FLINK-6965
>                 URL: https://issues.apache.org/jira/browse/FLINK-6965
>             Project: Flink
>          Issue Type: Bug
>          Components: Type Serialization System
>    Affects Versions: 1.3.0
>            Reporter: Chesnay Schepler
>            Assignee: Chesnay Schepler
>             Fix For: 1.3.2
>
>
> The shading rework made before 1.3 removed a snappy dependency that was accidentally
pulled in through hadoop. This is technically alright, until class-loaders rear their ugly
heads.
> Our kafka connector can read avro records, which may or may not require snappy. Usually
this _should_ be solvable by including the snappy dependency in the user-jar if necessary,
however since the kafka connector loads classes that it requires using the system class loader
this doesn't work.
> As such we have to add a separate snappy dependency to flink-core.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message