flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Viktor Rosenfeld <viktor.rosenf...@tu-berlin.de>
Subject Re: Does Flink use all of the available memory?
Date Sat, 14 Feb 2015 14:13:34 GMT
Hi Fabian,

thanks for the reply. I increased the TM heap to 20GB (on a 32GB machine) and indeed got a
performance boost of more than a minute (about 20%).

Cheers,
Viktor

> Am 14.02.2015 um 14:23 schrieb Fabian Hueske-2 [via Apache Flink (Incubator) User Mailing
List archive.] <ml-node+s2336050n731h85@n4.nabble.com>:
> 
> Hi,
> 
> Flink uses only the memory which is configured to the JobManager and TaskManager JVMs.
> By default this is 256MB for the JM and 512MB for the TM (see [1] for details).
> The TM memory is split into equally large chunks for each configured slot of the TM.
> 
> You should definitely configure the TM JVM memory for your setup! See common options
in [1] for other important configuration parameters such as numberOfTaskSlots.
> 
> In Yarn setups the TM memory is automatically sized with respect to the Yarn container
size, if I remember correctly.
> 
> [1] http://flink.apache.org/docs/0.8/config.html <http://flink.apache.org/docs/0.8/config.html>
> 
> 2015-02-14 13:58 GMT+01:00 Viktor Rosenfeld <[hidden email] <x-msg://43/user/SendEmail.jtp?type=node&node=731&i=0>>:
> Hi,
> 
> does the default configuration of Flink use all of the available memory? I
> mean the physical memory installed in the computer, not whatever amount of
> memory the JVM allocates by default.
> 
> Cheers,
> Viktor
> 
> 
> 
> --
> View this message in context: http://apache-flink-incubator-user-mailing-list-archive.2336050.n4.nabble.com/Does-Flink-use-all-of-the-available-memory-tp730.html
<http://apache-flink-incubator-user-mailing-list-archive.2336050.n4.nabble.com/Does-Flink-use-all-of-the-available-memory-tp730.html>
> Sent from the Apache Flink (Incubator) User Mailing List archive. mailing list archive
at Nabble.com.
> 
> 
> 
> If you reply to this email, your message will be added to the discussion below:
> http://apache-flink-incubator-user-mailing-list-archive.2336050.n4.nabble.com/Does-Flink-use-all-of-the-available-memory-tp730p731.html
<http://apache-flink-incubator-user-mailing-list-archive.2336050.n4.nabble.com/Does-Flink-use-all-of-the-available-memory-tp730p731.html>
> To unsubscribe from Does Flink use all of the available memory?, click here <http://apache-flink-incubator-user-mailing-list-archive.2336050.n4.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=730&code=dmlrdG9yLnJvc2VuZmVsZEB0dS1iZXJsaW4uZGV8NzMwfDQ1NTg2ODM4Ng==>.
> NAML <http://apache-flink-incubator-user-mailing-list-archive.2336050.n4.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>


signature.asc (161 bytes) <http://apache-flink-incubator-user-mailing-list-archive.2336050.n4.nabble.com/attachment/732/0/signature.asc>




--
View this message in context: http://apache-flink-incubator-user-mailing-list-archive.2336050.n4.nabble.com/Does-Flink-use-all-of-the-available-memory-tp730p732.html
Sent from the Apache Flink (Incubator) User Mailing List archive. mailing list archive at
Nabble.com.
Mime
View raw message