Mailing-List: contact user-help@flink.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@flink.apache.org
MIME-Version: 1.0
In-Reply-To: <CAGr9p8DXQsxCHyJPk7gtpAWXVomLdnU8qaQZqiMdmeESSjZEyg@mail.gmail.com>
References: <CAFgrA0KJE+mZKi7kWYXLn=hQ7Li4o1a7=SYCHAnYAHx30trFJw@mail.gmail.com>
 <CAGr9p8DXQsxCHyJPk7gtpAWXVomLdnU8qaQZqiMdmeESSjZEyg@mail.gmail.com>
From: "Chawla,Sumit " <sumitkchawla@gmail.com>
Date: Thu, 14 Jul 2016 09:49:27 -0700
Message-ID: <CAFgrA0L8SUOi+xu5TdeFVAVwfpATy94Czdxv7g+tcZMK58FATw@mail.gmail.com>
Subject: Re: Ability to partition logs per pipeline
To: user@flink.apache.org
Content-Type: multipart/alternative; boundary=001a1140b4000a5c2705379b498b
archived-at: Thu, 14 Jul 2016 16:49:33 -0000

--001a1140b4000a5c2705379b498b
Content-Type: text/plain; charset=UTF-8

Hi Robert

I actually mean both.  Scenarios where multiple jobs are running on
cluster, and same job could  be running on multiple task managers.  How can
we make sure that each job logs to a different file so that Logs are not
mixed, and its easy to debug a particular job.  Something like Hadoop Yarn,
where each attempt of a task produces a different log file.

Regards
Sumit Chawla


On Thu, Jul 14, 2016 at 6:11 AM, Robert Metzger <rmetzger@apache.org> wrote:

> Hi Sumit,
>
> What exactly do you mean by pipeline?
> Are you talking about cases were multiple jobs are running concurrently on
> the same TaskManager, or are you referring to parallel instances of a Flink
> job?
>
> On Wed, Jul 13, 2016 at 9:49 PM, Chawla,Sumit <sumitkchawla@gmail.com>
> wrote:
>
>> Hi All
>>
>> Does flink provide any ability to streamline logs being generated from a
>>  pipeline.  How can we keep the logs from two pipelines separate so that
>> its easy to debug the pipeline execution (something dynamic to
>> automatically partition the logs per pipeline)
>> Regards
>> Sumit Chawla
>>
>>
>

--001a1140b4000a5c2705379b498b
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Hi Robert<div><br></div><div>I actually mean both.=C2=A0 S=
cenarios where multiple jobs are running on cluster, and same job could =C2=
=A0be running on multiple task managers.=C2=A0 How can we make sure that ea=
ch job logs to a different file so that Logs are not mixed, and its easy to=
 debug a particular job.=C2=A0 Something like Hadoop Yarn, where each attem=
pt of a task produces a different log file.</div></div><div class=3D"gmail_=
extra"><br clear=3D"all"><div><div class=3D"gmail_signature" data-smartmail=
=3D"gmail_signature">Regards<br>Sumit Chawla<br><br></div></div>
<br><div class=3D"gmail_quote">On Thu, Jul 14, 2016 at 6:11 AM, Robert Metz=
ger <span dir=3D"ltr">&lt;<a href=3D"mailto:rmetzger@apache.org" target=3D"=
_blank">rmetzger@apache.org</a>&gt;</span> wrote:<br><blockquote class=3D"g=
mail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-l=
eft:1ex"><div dir=3D"ltr">Hi Sumit,<div><br></div><div>What exactly do you =
mean by pipeline?</div><div>Are you talking about cases were multiple jobs =
are running concurrently on the same TaskManager, or are you referring to p=
arallel instances of a Flink job?</div></div><div class=3D"HOEnZb"><div cla=
ss=3D"h5"><div class=3D"gmail_extra"><br><div class=3D"gmail_quote">On Wed,=
 Jul 13, 2016 at 9:49 PM, Chawla,Sumit <span dir=3D"ltr">&lt;<a href=3D"mai=
lto:sumitkchawla@gmail.com" target=3D"_blank">sumitkchawla@gmail.com</a>&gt=
;</span> wrote:<br><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 =
.8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir=3D"ltr">


<p><span>Hi All</span></p><p><span>Does flink provide any ability to stream=
line logs being generated from a =C2=A0pipeline.=C2=A0 How can we keep the =
logs from two pipelines separate so that its easy to debug the pipeline exe=
cution (something dynamic to automatically partition the logs per pipeline)=
=C2=A0</span></p><div><div data-smartmail=3D"gmail_signature">Regards<span>=
<font color=3D"#888888"><br>Sumit Chawla<br><br></font></span></div></div>
</div>
</blockquote></div><br></div>
</div></div></blockquote></div><br></div>

--001a1140b4000a5c2705379b498b--