hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From 임정택 <kabh...@gmail.com>
Subject Question about YARN Memory allocation
Date Wed, 28 Jan 2015 07:47:26 GMT
Hello all!

I'm new to YARN, so it could be beginner question.
(I've been used MRv1 and changed just now.)

I'm using HBase with 3 masters and 10 slaves - CDH 5.2 (Hadoop 2.5.0).
In order to migrate MRv1 to YARN, I read several docs, and change
configrations.

```
yarn.nodemanager.resource.memory-mb: 12288
yarn.scheduler.minimum-allocation-mb: 512
mapreduce.map.memory.mb: 1536
mapreduce.reduce.memory.mb: 1536
mapreduce.map.java.opts: -Xmx1024m -Dfile.encoding=UTF-8
-Dfile.client.encoding=UTF-8 -Dclient.encoding.override=UTF-8
mapreduce.reduce.java.opts: -Xmx1024m -Dfile.encoding=UTF-8
-Dfile.client.encoding=UTF-8 -Dclient.encoding.override=UTF-8
```

I'm expecting that it will be 80 containers running concurrently, but in
real it's 60 containers. (59 maps ran concurrently, maybe 1 is
ApplicationManager.)

All YarnChilds' VIRT are higher than 1.5G and lower than 2G now, so I'm
suspecting it.
But it's better to make clear, to understand YARN clearer.

Any helps & explanations are really appreciated.
Thanks!

Best regards.
Jungtaek Lim (HeartSaVioR)

Mime
View raw message