Mailing-List: contact user-help@flink.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@flink.apache.org
MIME-Version: 1.0
Sender: ewenstephan@gmail.com
In-Reply-To: 
 <CAPVL+b4HSqRrDMQJGC1r-gR_HvVvauRt0SfnP9Ca+xCMJKOW=g@mail.gmail.com>
References: 
 <CAPVL+b4HSqRrDMQJGC1r-gR_HvVvauRt0SfnP9Ca+xCMJKOW=g@mail.gmail.com>
Date: Tue, 23 Jun 2015 13:53:50 +0200
Message-ID: 
 <CANC1h_saLdeRACZtBHmvHvPGWRMPL7=n7FPUH5RQ95r4W7ef=Q@mail.gmail.com>
Subject: Re: memory flush on cluster
From: Stephan Ewen <sewen@apache.org>
To: user@flink.apache.org
Content-Type: multipart/alternative; boundary=20cf3071cb282a171905192e0b4c

--20cf3071cb282a171905192e0b4c
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Currently, Flink does not cache anything across runs, except JAR files on
the workers.

The reason the first run is slower may be:
 - Because in the first run, code is distributed in the cluster. In
subsequent runs, the JAR files need not be redistributed.
 - Because the JIT takes a bit to kick in and compile code in the first
run. In subsequent runs, the code is already JIT-ted.


The system should not freeze after 100 runs. Can you tell us a bit more of
what you see? Can you identify which process hangs and send us a
stack-trace of that one? Then we could look into this...


On Tue, Jun 23, 2015 at 10:56 AM, Pa R=C3=B6 <paul.roewer1990@googlemail.co=
m>
wrote:

> hi flink community,
>
> to time i test my flink app with a benchmark on an hadoop cluster (flink
> on yarn).
> my results show me that flink need for the first round more time as all
> other rounds. maybe flink cache something in memory? and if i run the
> benchmark 100 rounds my system freeze, i think the memory is full. give i=
t
> a way to flush the memory after the execution?
>
> best regards,
> paul
>

--20cf3071cb282a171905192e0b4c
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Currently, Flink does not cache anything across runs, exce=
pt JAR files on the workers.<div><br></div><div>The reason the first run is=
 slower may be:</div><div>=C2=A0- Because in the first run, code is distrib=
uted in the cluster. In subsequent runs, the JAR files need not be redistri=
buted.</div><div>=C2=A0- Because the JIT takes a bit to kick in and compile=
 code in the first run. In subsequent runs, the code is already JIT-ted.</d=
iv><div><br></div><div><br></div><div>The system should not freeze after 10=
0 runs. Can you tell us a bit more of what you see? Can you identify which =
process hangs and send us a stack-trace of that one? Then we could look int=
o this...</div><div><br></div><div><br></div></div><div class=3D"gmail_extr=
a"><br><div class=3D"gmail_quote">On Tue, Jun 23, 2015 at 10:56 AM, Pa R=C3=
=B6 <span dir=3D"ltr">&lt;<a href=3D"mailto:paul.roewer1990@googlemail.com"=
 target=3D"_blank">paul.roewer1990@googlemail.com</a>&gt;</span> wrote:<br>=
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex"><div dir=3D"ltr"><div><div><div><div>hi flin=
k community,<br><br></div>to time i test my flink app with a benchmark on a=
n hadoop cluster (flink on yarn).<br></div>my results show me that flink ne=
ed for the first round more time as all other rounds. maybe flink cache som=
ething in memory? and if i run the benchmark 100 rounds my system freeze, i=
 think the memory is full. give it a way to flush the memory after the exec=
ution?<br><br></div>best regards,<br></div>paul<br></div>
</blockquote></div><br></div>

--20cf3071cb282a171905192e0b4c--