flume-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lior Zeno (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLUME-2921) Support Elasticsearch 2.0+
Date Wed, 13 Jul 2016 17:58:20 GMT

    [ https://issues.apache.org/jira/browse/FLUME-2921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15375442#comment-15375442

Lior Zeno commented on FLUME-2921:

I don't think that is possible. The client JARs for those systems would have to shade the
dependency and ship it. Once there is a com.google.guava import in a compiled JAR (i.e. the
HDFS or ES client JAR), we have to classload classes that match that package name. If one
requires Guava 11, and the other requires Guava 18, then it simply isn't possible to run them
at the same time in the same classloader. However, the workaround is pretty simple: Run multiple

The idea is that the user will not use the original elasticsearch.jar, for instance. But,
instead, he will create a shaded jar that shades guava. This way, we will only have to load
one version of Guava and avoid collisions.

I agree with you. It's a very sticky problem at this point.

It's a bit disappointing, but it is what it is. We will revisit in 2.0.

> Support Elasticsearch 2.0+
> --------------------------
>                 Key: FLUME-2921
>                 URL: https://issues.apache.org/jira/browse/FLUME-2921
>             Project: Flume
>          Issue Type: Improvement
>          Components: Sinks+Sources
>            Reporter: Lior Zeno
>            Assignee: Lior Zeno
>             Fix For: v1.7.0
>         Attachments: FLUME-2921-0.patch, FLUME-2921-1.patch
> Elasticsearch sink supports an ancient version of ES. We should make the sink work with
newer versions of Elasticsearch.
> I attached a patch for that. Please note that this involves upgrading netty and guava.

This message was sent by Atlassian JIRA

View raw message