Mailing-List: contact user-help@flink.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@flink.apache.org
MIME-Version: 1.0
In-Reply-To: 
 <CAGr9p8BSLXHBsx2ru-oHQz45WaX4OXEMStQhOjGEPUHL8DobMA@mail.gmail.com>
References: 
 <CABsXpV1ooJp9+QXhRudyC6FvwZE4H7=_Lbm39KUwXSX6m1FCtw@mail.gmail.com>
	<CAKiyyaHQYShdtQa2jzHFddB14APrp6reVUdmD=SLm29cwK=q+w@mail.gmail.com>
	<CABsXpV3DA4Np8Ea1g9KYrAomCzcMzEP5hRGoVaq4bPP7VCRYdQ@mail.gmail.com>
	<CAGr9p8BSLXHBsx2ru-oHQz45WaX4OXEMStQhOjGEPUHL8DobMA@mail.gmail.com>
Date: Mon, 14 Mar 2016 15:00:50 +0100
Message-ID: 
 <CABsXpV3O2iCNtsZmGTWa0xAV2VWzd9V1o259MjFTO6yL47a3XQ@mail.gmail.com>
Subject: Re: Flink and YARN ship folder
From: Andrea Sella <andrea.sella@radicalbit.io>
To: user@flink.apache.org
Content-Type: multipart/alternative; boundary=001a1130d21853bf3d052e02b5e4

--001a1130d21853bf3d052e02b5e4
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Hi Robert,

Ok, thank you.

2016-03-14 11:13 GMT+01:00 Robert Metzger <rmetzger@apache.org>:

> Hi Andrea,
>
> You don't have to manually replicate any operations on the slaves. All
> files in the lib/ folder are transferred to all containers (Jobmanagers a=
nd
> TaskManagers).
>
>
> On Sat, Mar 12, 2016 at 3:25 PM, Andrea Sella <andrea.sella@radicalbit.io=
>
> wrote:
>
>> Hi Ufuk,
>>
>> I'm trying to execute the WordCount batch example with input and output
>> on Alluxio, i followed Running Flink on Alluxio
>> <http://www.alluxio.org/documentation/en/Running-Flink-on-Alluxio.html> =
and
>> added the library to lib folder. Have I to replicate this operation on t=
he
>> slaves or YARN manage that and I must have the library just where I laun=
ch
>> the job?
>>
>> Thanks,
>> Andrea
>>
>> 2016-03-11 19:23 GMT+01:00 Ufuk Celebi <uce@apache.org>:
>>
>>> Everything in the lib folder should be added to the classpath. Can you
>>> check the YARN client logs that the files are uploaded? Furthermore,
>>> you can check the classpath of the JVM in the YARN logs of the
>>> JobManager/TaskManager processes.
>>>
>>> =E2=80=93 Ufuk
>>>
>>>
>>> On Fri, Mar 11, 2016 at 5:33 PM, Andrea Sella
>>> <andrea.sella@radicalbit.io> wrote:
>>> > Hi,
>>> >
>>> > There is a way to add external dependencies to Flink Job,  running on
>>> YARN,
>>> > not using HADOOP_CLASSPATH?
>>> > I am looking for a similar idea to standalone mode using lib folder.
>>> >
>>> > BR,
>>> > Andrea
>>>
>>
>>
>

--001a1130d21853bf3d052e02b5e4
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Hi Robert,<br><br>Ok, thank you.<br></div><div class=3D"gm=
ail_extra"><br><div class=3D"gmail_quote">2016-03-14 11:13 GMT+01:00 Robert=
 Metzger <span dir=3D"ltr">&lt;<a href=3D"mailto:rmetzger@apache.org" targe=
t=3D"_blank">rmetzger@apache.org</a>&gt;</span>:<br><blockquote class=3D"gm=
ail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-le=
ft:1ex"><div dir=3D"ltr">Hi Andrea,<div><br></div><div>You don&#39;t have t=
o manually replicate any operations on the slaves. All files in the lib/ fo=
lder are transferred to all containers (Jobmanagers and TaskManagers).</div=
><div><br></div></div><div class=3D"HOEnZb"><div class=3D"h5"><div class=3D=
"gmail_extra"><br><div class=3D"gmail_quote">On Sat, Mar 12, 2016 at 3:25 P=
M, Andrea Sella <span dir=3D"ltr">&lt;<a href=3D"mailto:andrea.sella@radica=
lbit.io" target=3D"_blank">andrea.sella@radicalbit.io</a>&gt;</span> wrote:=
<br><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-lef=
t:1px #ccc solid;padding-left:1ex"><div dir=3D"ltr">Hi Ufuk,<br><br>I&#39;m=
 trying to execute the WordCount batch example with input and output on All=
uxio, i followed=C2=A0<a href=3D"http://www.alluxio.org/documentation/en/Ru=
nning-Flink-on-Alluxio.html" target=3D"_blank">Running Flink on Alluxio</a>=
=C2=A0and added the library to lib folder. Have I to replicate this operati=
on on the slaves or YARN manage that and I must have the library just where=
 I launch the job?<br><br>Thanks,<br>Andrea</div><div><div><div class=3D"gm=
ail_extra"><br><div class=3D"gmail_quote">2016-03-11 19:23 GMT+01:00 Ufuk C=
elebi <span dir=3D"ltr">&lt;<a href=3D"mailto:uce@apache.org" target=3D"_bl=
ank">uce@apache.org</a>&gt;</span>:<br><blockquote class=3D"gmail_quote" st=
yle=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Every=
thing in the lib folder should be added to the classpath. Can you<br>
check the YARN client logs that the files are uploaded? Furthermore,<br>
you can check the classpath of the JVM in the YARN logs of the<br>
JobManager/TaskManager processes.<br>
<span><font color=3D"#888888"><br>
=E2=80=93 Ufuk<br>
</font></span><div><div><br>
<br>
On Fri, Mar 11, 2016 at 5:33 PM, Andrea Sella<br>
&lt;<a href=3D"mailto:andrea.sella@radicalbit.io" target=3D"_blank">andrea.=
sella@radicalbit.io</a>&gt; wrote:<br>
&gt; Hi,<br>
&gt;<br>
&gt; There is a way to add external dependencies to Flink Job,=C2=A0 runnin=
g on YARN,<br>
&gt; not using HADOOP_CLASSPATH?<br>
&gt; I am looking for a similar idea to standalone mode using lib folder.<b=
r>
&gt;<br>
&gt; BR,<br>
&gt; Andrea<br>
</div></div></blockquote></div><br></div>
</div></div></blockquote></div><br></div>
</div></div></blockquote></div><br></div>

--001a1130d21853bf3d052e02b5e4--