From user-return-32653-archive-asf-public=cust-asf.ponee.io@flink.apache.org Mon Feb 17 03:12:57 2020 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [207.244.88.153]) by mx-eu-01.ponee.io (Postfix) with SMTP id 17C6C180181 for ; Mon, 17 Feb 2020 04:12:56 +0100 (CET) Received: (qmail 22130 invoked by uid 500); 17 Feb 2020 03:12:54 -0000 Mailing-List: contact user-help@flink.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list user@flink.apache.org Received: (qmail 22118 invoked by uid 99); 17 Feb 2020 03:12:54 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 17 Feb 2020 03:12:54 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id 9000118136F for ; Mon, 17 Feb 2020 03:12:53 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 0 X-Spam-Level: X-Spam-Status: No, score=0 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=0.2, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd3-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-he-de.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id 1qVmS-YbLpjn for ; Mon, 17 Feb 2020 03:12:52 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=2a00:1450:4864:20::442; helo=mail-wr1-x442.google.com; envelope-from=danrtsey.wy@gmail.com; receiver= Received: from mail-wr1-x442.google.com (mail-wr1-x442.google.com [IPv6:2a00:1450:4864:20::442]) by mx1-he-de.apache.org (ASF Mail Server at mx1-he-de.apache.org) with ESMTPS id E60577E135 for ; Mon, 17 Feb 2020 03:12:51 +0000 (UTC) Received: by mail-wr1-x442.google.com with SMTP id w15so17834363wru.4 for ; Sun, 16 Feb 2020 19:12:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=eqdq7nqLDhpRcMVpij7sZMGcoZ4bkt6b5KTXBsAQpVo=; b=AlXdHbg/9jyJI2l22jfPTQg3DoFqMWTeZXNfOAJM3VvSpT75V/1GVE/+XdJHdCjIP1 OCnZKHRi0cZJWHwH7AFNms/IEMau1AokGooDb8dCyllrdTLCsIvTWde9AMacfofxbA7y WzrGLfPoilNjL8UZ8F+GvsTv2G+5dUdGLaYxYIMzNL4Gef9M1AoXxr8lrGZ3BRJ8NVeB pmrjhgVh4ZmR7019x8lWvlzEQpPtguQ5gXcoC45O52or3e0YzV6nEDGQgMsIUA4S8jtZ QUMJI5R1mj35t/6FFd9G11PbOQVi5SFxfqbBv01/Giv+weHd2XiHm6OD3KVLH/LzehxZ C66A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=eqdq7nqLDhpRcMVpij7sZMGcoZ4bkt6b5KTXBsAQpVo=; b=R0Z5WWxizUXNktpkD5xFctwUUyX5i7gsPxwDOpNGOJ06T2Y2gv1xWJcE69YCM5ljDm 9CVQCNlSSNaNZi4d3uU0SA4tvPs5c3FhiQgJPKoxKJZneMToJGGaSpGjY+jrfnNx3TKL WTkWxHqaVMhdaV7aiSahwK8VsY4BkXm53WE9g7kNm5SK4EZignMskWBPDqHTv3ac5P9Y jmHCfDLAdb5ldBAX0FnGpNi9ZOeveOpsTJ6LZpSqv4fskdX5+wh9QhBRYSS6anaU5Pjm 3AE96TGPCyqBAIF0OPH9q66xELFbMH/+V5OGHgDWwFe9Qfuhmlrk8zHcL0kzK68b7nY2 8B8A== X-Gm-Message-State: APjAAAUm8Hy7/rHIBjyvBa9ktTtUxp/vKNcC56ZySlKYK7kUBb3ujCyO 8ZT0YjIA0szoKIcRAp6g964qu5/TYUskq7n9V+Q= X-Google-Smtp-Source: APXvYqzc3ysNMFjhVabhrx75EYxzLV3MDU0tTrJPutQeneCcZS1wrZrLniUXx6+Xnglk9W0iKkkazaqxDtiiivLwyzE= X-Received: by 2002:a5d:534b:: with SMTP id t11mr18775099wrv.120.1581909165753; Sun, 16 Feb 2020 19:12:45 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: Yang Wang Date: Mon, 17 Feb 2020 11:12:34 +0800 Message-ID: Subject: Re: [Flink 1.10] Classpath doesn't include custom files in lib/ To: Maxim Parkachov Cc: user Content-Type: multipart/alternative; boundary="000000000000e4497d059ebced77" --000000000000e4497d059ebced77 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hi Maxim Parkachov, The users files also have been shipped to JobManager and TaskManager. However, it is not directly added to the classpath. Instead, the parent directory is added to the classpath. This changes are to make resource classloading work. You could check more information here[1]. [1]. https://issues.apache.org/jira/browse/FLINK-13127 Best, Yang Maxim Parkachov =E4=BA=8E2020=E5=B9=B42=E6=9C=8815= =E6=97=A5=E5=91=A8=E5=85=AD =E4=B8=8A=E5=8D=8812:58=E5=86=99=E9=81=93=EF=BC= =9A > Hi everyone, > > I'm trying to run my job with flink 1.10 with YARN cluster per-job mode. > In the previous versions all files in lib/ folder were automatically > included in classpath. Now, with 1.10 I see only *.jar files are included > in classpath. but not "other" files. Is this deliberate change or bug ? > > Generally, what is recommended way to include custom files in classpath > and ship it during start to all containers ? > > Thank > --000000000000e4497d059ebced77 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi=C2=A0Maxim Parkachov,

The users file= s also have been shipped to JobManager and TaskManager. However, it
is not directly added to the classpath. Instead, the parent directory is= added to the
classpath. This changes are to=C2=A0make resource c= lassloading work. You could check more
information here[1].
=




Best,
Yang

Maxim Parkachov <lazy.gopher@= gmail.com> =E4=BA=8E2020=E5=B9=B42=E6=9C=8815=E6=97=A5=E5=91=A8=E5= =85=AD =E4=B8=8A=E5=8D=8812:58=E5=86=99=E9=81=93=EF=BC=9A
Hi everyone,
I'm trying=C2=A0to run my job with flink 1.10 with YARN= cluster per-job mode. In the previous versions all files in lib/ folder we= re automatically included in classpath. Now, with 1.10 I see only *.jar fil= es are included in classpath. but not "other" files. Is this deli= berate change or bug ?

Generally, what is recommen= ded way to include custom files in classpath and ship it during start to al= l containers ?

Thank
--000000000000e4497d059ebced77--