hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Smita Deshpande <smita.deshpa...@cumulus-systems.com>
Subject CPU usage of a container.
Date Wed, 05 Nov 2014 06:03:12 GMT


From: Smita Deshpande
Sent: Wednesday, November 05, 2014 10:52 AM
To: user@hadoop.apache.org
Subject: CPU usage of a container.

Hi All,
                I am facing sort of a weird issue in YARN. I am running a single container
on a cluster whose cpu configuration is as follows:
                NODEMANAGER1 : 4 cpu cores
                NODEMANAGER2 : 4 cpu cores
                NODEMANAGER3 : 16 cpu cores
                All processors are Hyperthreaded ones. So if I am using 1 cpu core then max
usage could be 200%.
                When I am running different number of threads in that container(basically
cpu intensive calculation), its showing cpu usage more than allotted number of cores to it.
Please refer to below table for different test cases. Highlighted values in Red seem to have
crossed its usage. I am using DominantResourceCalculator in CS.
                PFA the screenshot for the same.
                Any help would be appreciated.

Resource Ask

%cpu Usage (from htop command)

# of Threads launched in container

<1024,1>

176.8

4

108

1

177

2

291

3

342

4

337

4    [container launched on NODEMANAGER3]

<1024,2>

177

3

182.6

9

336

4    [container launched on NODEMANAGER3]

189

2   [container launched on NODEMANAGER2]

291

3

337

4

<1024,3>

283

3

329.7

9

343

4  [container launched on NODEMANAGER3]

122

1

216

2

290

3

<1024,4>

289

3

123

1

217

2

292

3

338

4

177.3

32



Regards,
Smita

Mime
View raw message