hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Narayanan K <knarayana...@gmail.com>
Subject Memory settings in hadoop YARN
Date Wed, 20 Aug 2014 04:08:35 GMT
Hi

We run our Pig jobs in Hadoop 0.23 which has the new YARN architecture.

I had few questions on memory used by the jobs :

We have following settings for memory.

mapred.child.java.opts

mapreduce.map.memory.mb

mapreduce.reduce.memory.mb

yarn.app.mapreduce.am.resource.mb

yarn.app.mapreduce.am.command-opts


1. I want to understand these settings to make better use of Hadoop cluster.
2. How is Memory Allocated to Container ? Does any of the above
settings result in change in container ?
3. Any other memory settings we need to be aware of ?
4. I heard there was a virtual memory and physical memory involved. Is
there any propery documentation/guide that can make the memory
management easy.

Thanks
Narayanan

Mime
View raw message