flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robert Metzger <rmetz...@apache.org>
Subject Re: EMR vCores and slot allocation
Date Mon, 02 May 2016 08:12:35 GMT
Hi Ken,
sorry for the late response. The number of CPU cores we show in the web
interface is based on what the JVM tells us from
"Runtime.getRuntime().availableProcessors();". I'm not sure how tthe
processor count behaves on Amazon VMs.

Given that each of your servers has 8 vCores, I would set the slot count to
8 per Taskmanager.


On Fri, Apr 29, 2016 at 1:58 AM, Ken Krugler <kkrugler_lists@transpac.com>
wrote:

> Based on what Flink reports in the JobManager GUI, it looks like it thinks
> that the EC2 instances I’m using for my EMR jobs only have 4 physical cores.
>
> Which would make sense, as Amazon describes these servers as having 8
> vCores.
>
> From
> https://ci.apache.org/projects/flink/flink-docs-master/setup/config.html,
> the recommended configuration would then be 4 slots/TaskManager, yes?
>
> Thanks,
>
> — Ken
>
> --------------------------
> Ken Krugler
> +1 530-210-6378
> http://www.scaleunlimited.com
> custom big data solutions & training
> Hadoop, Cascading, Cassandra & Solr
>
>
>
>

Mime
View raw message