Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id E5D5C200CA4 for ; Wed, 7 Jun 2017 17:42:25 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id E318F160BD0; Wed, 7 Jun 2017 15:42:25 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 0ECCB160BB6 for ; Wed, 7 Jun 2017 17:42:24 +0200 (CEST) Received: (qmail 90744 invoked by uid 500); 7 Jun 2017 15:42:24 -0000 Mailing-List: contact user-help@flink.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list user@flink.apache.org Received: (qmail 90733 invoked by uid 99); 7 Jun 2017 15:42:24 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 07 Jun 2017 15:42:24 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id 956C91AFED6 for ; Wed, 7 Jun 2017 15:42:23 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 2.48 X-Spam-Level: ** X-Spam-Status: No, score=2.48 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RCVD_IN_SORBS_SPAM=0.5] autolearn=disabled Authentication-Results: spamd2-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=okkam-it.20150623.gappssmtp.com Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id M-IxjIlE7RWe for ; Wed, 7 Jun 2017 15:42:20 +0000 (UTC) Received: from mail-vk0-f54.google.com (mail-vk0-f54.google.com [209.85.213.54]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTPS id 341765F6C0 for ; Wed, 7 Jun 2017 15:42:20 +0000 (UTC) Received: by mail-vk0-f54.google.com with SMTP id g66so6727106vki.1 for ; Wed, 07 Jun 2017 08:42:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=okkam-it.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=F+hqNhXlikqqJRhuCq5MtExygV2glVhh2FNgPKoj0lM=; b=p0wYqOZSFR63hUspieEWQ2ChphjjdHI/hC9kEwbX7mGaqiuWAG71pPhRBltg6ne4Gy SPKtV+W8YIjsVeAyc6oNDx+Tsnn/hh0XTeq7ax1R3L69Uveyb6aFOK++iVEFE9sOMpa6 NpVh8SRY5T3AWFf9M285HVD97XPyBIlHxjNLPyRkJS608+ulEZjP+eHzv7ksuj5ekkV9 Lcea3T76hliOHnDrrKJ2U2eaoqet8h/8rTdzsFWSIKnNUBwngqYe0udrL60Kb95jxys3 Zi9F46BBMCqrBgurJo1Q8dGn63iomjhy/8FqkjCk1Sov59F08iK21KwT34MWFeG3V6bk md/w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=F+hqNhXlikqqJRhuCq5MtExygV2glVhh2FNgPKoj0lM=; b=Ju/SzUgIO/r/8o0nqoG4VRGAv2QeMSlgBL//Ovq0HrmB/sm54nSxot2M1JJsCKjMDa 2PkPiTe1ljmdTEm3yH94Jkq2Z+uogqaqB5QT2P31TfMTIy75L4ikZO92T5oU77kbfU31 zLnPAlXuEQNfLafMi40kmSV3RQ5q2D5EHGskj5T+ZmRdABTlKqUHrHKPMG2+9YCc0fEt igSD8JRaEPUIGBj2kNttdvfS+kjC4uhdWvWc2E+LY5tdbb1+ymMchlgLDc2hRn2GNOs4 Cd0HSDmpnqhV+TH+cylgO0j/3E5i3jg+MzwClDPU5Vds61+TXB6reAuSrtmUdz2F67i6 F3+Q== X-Gm-Message-State: AODbwcBoWh6oHTrXGH0QbPe0+sadaLRLpzT8yZRGyK25/0TUe6Fsbv6T 9SnPyuTQB+E5as8K8nSKfmHyj1KH1Kg2 X-Received: by 10.31.188.193 with SMTP id m184mr9497651vkf.81.1496850139142; Wed, 07 Jun 2017 08:42:19 -0700 (PDT) MIME-Version: 1.0 Received: by 10.31.52.145 with HTTP; Wed, 7 Jun 2017 08:41:58 -0700 (PDT) X-Originating-IP: [77.43.114.114] In-Reply-To: References: From: Flavio Pompermaier Date: Wed, 7 Jun 2017 17:41:58 +0200 Message-ID: Subject: Re: Guava version conflict To: "Tzu-Li (Gordon) Tai" Cc: user Content-Type: multipart/alternative; boundary="001a11438dfcd377f4055160947a" archived-at: Wed, 07 Jun 2017 15:42:26 -0000 --001a11438dfcd377f4055160947a Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable I shaded the Elasticsearch dependency [1] and now the job works. So I cannot run a job that needs guava 18 on Flink 1.2.1... [1] https://www.elastic.co/blog/to-shade-or-not-to-shade On Wed, Jun 7, 2017 at 5:33 PM, Tzu-Li (Gordon) Tai wrote: > Hi Flavio, > > Could there be another dependency in your job that requires a conflicting > version (w.r.t. ES 2.4.1) of Guava? > I=E2=80=99ve just double checked the flink-dist jar, there doesn=E2=80=99= t seem to be any > non-shaded Guava dependencies there, so the conflict should not have been > caused by Flink. > > Cheers, > Gordon > > > On 7 June 2017 at 4:12:04 PM, Flavio Pompermaier (pompermaier@okkam.it) > wrote: > > Hi to all, > I'm trying to use the new ES connector to index data from Flink (with ES > 2.4.1). > When I try to run it from Eclipse everything is ok, when I run it from th= e > cluster I get the following exception: > > java.lang.NoSuchMethodError: com.google.common.util.concurr > ent.MoreExecutors.directExecutor()Ljava/util/concurrent/Executor; > at org.elasticsearch.threadpool.ThreadPool.(ThreadPool. > java:192) > at org.elasticsearch.client.transport.TransportClient$Builder. > build(TransportClient.java:131) > > In my fat jar there are the classes of guava 18 (ES requires that > version), Flink runs on CDH 5.9 (that use guava 11), in flink-dist jar I > think that there's guava 11 classes while in flink-hadoop-compatibility > there are shade guava 18 dependencies. > > How can I make the job successfully run on the cluster? > > Best, > Flavio > > --001a11438dfcd377f4055160947a Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
I shaded the Elasticsearch dependency [1] and now the job = works.
So I cannot run a job that needs guava 18 on Flink 1.2.1...


On Wed, J= un 7, 2017 at 5:33 PM, Tzu-Li (Gordon) Tai <tzulitai@apache.org><= /span> wrote:
Hi Flavio,

Could there be another dependen= cy in your job that requires a conflicting version (w.r.t. ES 2.4.1) of Gua= va?
I=E2=80=99ve just double checked the flink-dist jar, there does= n=E2=80=99t seem to be any non-shaded Guava dependencies there, so the conf= lict should not have been caused by Flink.

Cheers,
Gordon


On 7 June 2017 at= 4:12:04 PM, Flavio Pompermaier (pompermaier@okkam.it) wrote:

Hi to all,
I'm trying to use the new ES connector to index data from Flink (with ES 2.4.1).
When I try to run it from Eclipse everything is ok, when I run it from the cluster I get the following exception:

java.lang.NoSuchMethodError: com.google.common.util.concurr= ent.MoreExecutors.directExecut= or()Ljava/util/concurrent/Exec= utor;
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= at org.elasticsearch.threadpool.T= hreadPool.<clinit>(ThreadPool= .java:192)
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= at org.elasticsearch.client.trans= port.TransportClient$Builder.b= uild(TransportClient.java:131)=

In my fat jar there are the classes of guava 18 (ES requires that version), Flink runs on CDH 5.9 (that use guava 11), in flink-dist jar I think that there's guava 11 classes while in flink-hadoop-compatibility there are shade guava 18 dependencies.

How can I make the job successfully run on the cluster?

Best,
Flavio

--001a11438dfcd377f4055160947a--