flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stephan Ewen <se...@apache.org>
Subject Re: memory flush on cluster
Date Tue, 23 Jun 2015 11:53:50 GMT
Currently, Flink does not cache anything across runs, except JAR files on
the workers.

The reason the first run is slower may be:
 - Because in the first run, code is distributed in the cluster. In
subsequent runs, the JAR files need not be redistributed.
 - Because the JIT takes a bit to kick in and compile code in the first
run. In subsequent runs, the code is already JIT-ted.

The system should not freeze after 100 runs. Can you tell us a bit more of
what you see? Can you identify which process hangs and send us a
stack-trace of that one? Then we could look into this...

On Tue, Jun 23, 2015 at 10:56 AM, Pa Rö <paul.roewer1990@googlemail.com>

> hi flink community,
> to time i test my flink app with a benchmark on an hadoop cluster (flink
> on yarn).
> my results show me that flink need for the first round more time as all
> other rounds. maybe flink cache something in memory? and if i run the
> benchmark 100 rounds my system freeze, i think the memory is full. give it
> a way to flush the memory after the execution?
> best regards,
> paul

View raw message