Mailing-List: contact user-help@flink.apache.org; run by ezmlm
Precedence: bulk
MIME-Version: 1.0
In-Reply-To: <94A3D65C-8BBE-4C72-9901-F37596FF8754@icloud.com>
References: <94A3D65C-8BBE-4C72-9901-F37596FF8754@icloud.com>
From: Till Rohrmann <trohrmann@apache.org>
Date: Tue, 23 May 2017 16:20:11 +0200
Message-ID: <CAC27z=NPMT96Kh+P1_7OWms6Zpq6Ma_f9F8W=6d_i+KjdYBX5A@mail.gmail.com>
Subject: Re: Need help debugging back pressure job
To: Fritz Budiyanto <fbudiyan@icloud.com>
Cc: user <user@flink.apache.org>
Content-Type: multipart/alternative; boundary="94eb2c19ecb4e0f018055031b141"
archived-at: Tue, 23 May 2017 14:20:58 -0000

--94eb2c19ecb4e0f018055031b141
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Hi Fritz,

you're right that back pressure should propagate upstream to the sources.
Thus, the cause of the back pressure should be the operator following the
last operator with back pressure.

In order to debug it you could take a look at the stack trace of the TM.
Simply go to the machine on which the TM runs, find out the process id via
jps and then call jstack with the respective process id.

Alternatively, you can try to debug the cluster remotely [1].

[1]
https://cwiki.apache.org/confluence/display/FLINK/Remote+Debugging+of+Flink=
+Clusters

Cheers,
Till

On Tue, May 23, 2017 at 7:14 AM, Fritz Budiyanto <fbudiyan@icloud.com>
wrote:

> Hi All,
>
> Any tips on debugging back pressure ? I have a workload where it get stuc=
k
> after it ran for a couple of hours.
> I assume the cause of the back pressure is the block next to the one
> showing as having the back pressure, is this right ?
>
> Any idea on how to get the backtrace ? (I=E2=80=99m using standalone comb=
ined
> jm/tm with parallelism of 1, and the suspected block is doing
> ProcessFunction with event timers)
>
> =E2=80=94
> Fritz
>
>
>

--94eb2c19ecb4e0f018055031b141
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Hi Fritz,<div><br></div><div>you&#39;re right that back pr=
essure should propagate upstream to the sources. Thus, the cause of the bac=
k pressure should be the operator following the last operator with back pre=
ssure.</div><div><br></div><div>In order to debug it you could take a look =
at the stack trace of the TM. Simply go to the machine on which the TM runs=
, find out the process id via jps and then call jstack with the respective =
process id.</div><div><br></div><div>Alternatively, you can try to debug th=
e cluster remotely [1].</div><div><br></div><div>[1]=C2=A0<a href=3D"https:=
//cwiki.apache.org/confluence/display/FLINK/Remote+Debugging+of+Flink+Clust=
ers">https://cwiki.apache.org/confluence/display/FLINK/Remote+Debugging+of+=
Flink+Clusters</a></div><div><br></div><div>Cheers,</div><div>Till</div></d=
iv><div class=3D"gmail_extra"><br><div class=3D"gmail_quote">On Tue, May 23=
, 2017 at 7:14 AM, Fritz Budiyanto <span dir=3D"ltr">&lt;<a href=3D"mailto:=
fbudiyan@icloud.com" target=3D"_blank">fbudiyan@icloud.com</a>&gt;</span> w=
rote:<br><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;borde=
r-left:1px #ccc solid;padding-left:1ex">Hi All,<br>
<br>
Any tips on debugging back pressure ? I have a workload where it get stuck =
after it ran for a couple of hours.<br>
I assume the cause of the back pressure is the block next to the one showin=
g as having the back pressure, is this right ?<br>
<br>
Any idea on how to get the backtrace ? (I=E2=80=99m using standalone combin=
ed jm/tm with parallelism of 1, and the suspected block is doing ProcessFun=
ction with event timers)<br>
<br>
=E2=80=94<br>
<span class=3D"HOEnZb"><font color=3D"#888888">Fritz<br>
<br>
<br>
</font></span></blockquote></div><br></div>

--94eb2c19ecb4e0f018055031b141--