hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harsh J <ha...@cloudera.com>
Subject Re: total vcores per node containers in yarn
Date Sun, 19 Jul 2015 08:57:05 GMT
That doesn't sound right, if you are indeed making use of the new feature.
First off, this is not the default behaviour in the actual Apache Hadoop
2.5.0 release. Are you using a different distribution based on that release
instead? Secondly, if it were using the auto-calculate feature, it'd only
count primary cores as vcores by default.

Are you certain you've looked at the right yarn-site.xml that pertains to
the NodeManager, and not a client-purpose file which may simply lack the
config mentioned in it?

On Sat, Jul 18, 2015 at 7:50 PM Shushant Arora <shushantarora09@gmail.com>

> Its hadoop 2.5.0.
> Whats the logic of default using hardware detection. Say My node has 8
> actual core and 32 virtual cores. Its taking 26 as value of vcores
> available of this node on RM UI.
> On Sat, Jul 18, 2015 at 7:22 PM, Harsh J <harsh@cloudera.com> wrote:
>> What version of Apache Hadoop are you running? Recent changes have made
>> YARN to auto-compute this via hardware detection, by default (rather than
>> the 8 default).
>> On Fri, Jul 17, 2015 at 11:31 PM Shushant Arora <
>> shushantarora09@gmail.com> wrote:
>>> In Yarn there is a setting to specify no of vcores that can be allocated
>>> to containers.
>>> yarn.nodemanager.resource.cpu-vcores
>>> In my cluster's nodes yarn-site.xml, this property is not specified. But
>>> total vcores displayed on RM's web page for the nodes are different than
>>> the default(8). Is there any other place to control this number ?

View raw message