Mailing-List: contact issues-help@flink.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@flink.apache.org
Date: Sat, 8 Jul 2017 04:10:00 +0000 (UTC)
From: "Tzu-Li (Gordon) Tai (JIRA)" <jira@apache.org>
To: issues@flink.apache.org
Message-ID: <JIRA.13085515.1499436123000.198477.1499487000170@Atlassian.JIRA>
In-Reply-To: <JIRA.13085515.1499436123000@Atlassian.JIRA>
References: <JIRA.13085515.1499436123000@Atlassian.JIRA> <JIRA.13085515.1499436123831@jira-lw-us.apache.org>
Subject: [jira] [Comment Edited] (FLINK-7133) Fix Elasticsearch version
 interference
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Sat, 08 Jul 2017 04:10:06 -0000


    [ https://issues.apache.org/jira/browse/FLINK-7133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16078938#comment-16078938 ] 

Tzu-Li (Gordon) Tai edited comment on FLINK-7133 at 7/8/17 4:09 AM:
--------------------------------------------------------------------

[~Adebski] [~aljoscha] thanks for the explanation :)

I assumed that the problem was local only because the last time I tried it with cluster execution for the release testing, it worked without issues.
I also tried it in local mode just now, however cannot reproduce. I wonder if it may be something specific to the IDE setup, or only surfaces when some other dependency is included.

I don't really like the idea of removing the `elasticsearch-base` module and duplicating the base classes. Apart from the duplication which isn't nice, if we're solving it this way, then the problem would still exist for the `elasticsearch1` module.

As for another possible resolution, I was checking out this thread: https://github.com/elastic/elasticsearch/issues/7959, and wondering if we can simply exclude the asm dependency from ES.
From the comments in that thread, it seems like if we're just using ES client, it is safe to exclude the asm dependencies from `elasticsearch-base`. i.e.,
{code}
   <dependency>
        <groupId>org.elasticsearch</groupId>
        <artifactId>elasticsearch</artifactId>
        <exclusions>
            <exclusion>
                <artifactId>asm</artifactId>
                <groupId>org.ow2.asm</groupId>
            </exclusion>
        </exclusions>
    </dependency>
{code}

[~Adebski] have you tried that? Btw, thanks a lot for digging into this :)


was (Author: tzulitai):
[~Adebski] [~aljoscha] thanks for the explanation :)

I assumed that the problem was local only because the last time I tried it with cluster execution for the release testing, it worked without issues.
I also tried it in local mode just now, however cannot reproduce. I wonder if it may be something specific to the IDE setup, or only surfaces when some other dependency is included.

As for the possible resolution, I was checking out this thread: https://github.com/elastic/elasticsearch/issues/7959, and wondering if we can simply exclude the asm dependency from ES.
From the comments in that thread, it seems like if we're just using ES client, it is safe to exclude the asm dependencies from `elasticsearch-base`. i.e.,
{code}
   <dependency>
        <groupId>org.elasticsearch</groupId>
        <artifactId>elasticsearch</artifactId>
        <exclusions>
            <exclusion>
                <artifactId>asm</artifactId>
                <groupId>org.ow2.asm</groupId>
            </exclusion>
        </exclusions>
    </dependency>
{code}

[~Adebski] have you tried that? Btw, thanks a lot for digging into this :)

> Fix Elasticsearch version interference
> --------------------------------------
>
>                 Key: FLINK-7133
>                 URL: https://issues.apache.org/jira/browse/FLINK-7133
>             Project: Flink
>          Issue Type: Bug
>          Components: Streaming Connectors
>    Affects Versions: 1.3.0, 1.3.1
>            Reporter: Aljoscha Krettek
>            Priority: Blocker
>             Fix For: 1.3.2
>
>
> At least two users have encountered problems with shading in the Elasticsearch connector:
>  - https://lists.apache.org/thread.html/b5bc1f690dc894ea9a8b69e82c89eb89ba6dfc2fec2588d2ccacee2c@%3Cuser.flink.apache.org%3E
>  - https://lists.apache.org/thread.html/2356670d168f61c20e34611e3c4aeb9c9b3f959f23a9833f631da1ba@%3Cuser.flink.apache.org%3E
> The problem seems to be (quote from the second mail):
> {code}
> I've found out the source of the problem when I build flink locally.
> elastic-search base depends on (by default) ES version 1.7.1 that depends on
> asm 4.1 and that version is shaded to elasticsearch-base-jar. I tried to set
> elasticsearch.version property in Maven to 5.1.2 (the same as elasticsearch5
> connector) but then elasticsearch-base does not compile:
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-compiler-plugin:3.1:testCompile
> (default-testCompile) on project flink-connector-elasticsearch-base_2.11:
> Compilation failure
> [ERROR]
> /home/adebski/Downloads/flink-release-1.3.1/flink-connectors/flink-connector-elasticsearch-base/src/test/java/org/apache/flink/streaming/connectors/elasticsearch/ElasticsearchSinkBaseTest.java:[491,92]
> no suitable constructor found for
> BulkItemResponse(int,java.lang.String,org.elasticsearch.action.ActionResponse)
> [ERROR] constructor
> org.elasticsearch.action.bulk.BulkItemResponse.BulkItemResponse(int,java.lang.String,org.elasticsearch.action.DocWriteResponse)
> is not applicable
> [ERROR] (argument mismatch; org.elasticsearch.action.ActionResponse cannot
> be converted to org.elasticsearch.action.DocWriteResponse)
> [ERROR] constructor
> org.elasticsearch.action.bulk.BulkItemResponse.BulkItemResponse(int,java.lang.String,org.elasticsearch.action.bulk.BulkItemResponse.Failure)
> is not applicable
> [ERROR] (argument mismatch; org.elasticsearch.action.ActionResponse cannot
> be converted to org.elasticsearch.action.bulk.BulkItemResponse.Failure)
> {code}
> To me, it seems like we have to get rid of the "base" package and have two completely separate packages.


--
This message was sent by Atlassian JIRA
(v6.4.14#64029)