Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hadoop.apache.org
Received-SPF: pass (athena.apache.org: domain of regestrer@gmail.com
 designates 209.85.212.172 as permitted sender)
Message-ID: <54415EEC.2070302@gmail.com>
Date: Fri, 17 Oct 2014 21:24:44 +0300
From: peter 2 <regestrer@gmail.com>
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64;
 rv:24.0) Gecko/20100101 Thunderbird/24.6.0
MIME-Version: 1.0
To: user@hadoop.apache.org
Subject: Dynamically set map / reducer memory
References: 
 <CACeqxwQ7sMmo92RG=SFuuOhCHn2x1Rf=WvE1_9Hs38q-EBX+Kw@mail.gmail.com>
	<CALr1C9oYpmFZUnc1LK-aEtJxPGMBvqQZA7kxFbMk9+0egK=9qw@mail.gmail.com>
	<CACeqxwTcFqN6PJtz8kB9NHJts9L3a=HH-JDwjS+TL1cAWFRO3Q@mail.gmail.com>
	<COL126-DS146CAD630D36FF9B04F93A98A80@phx.gbl>
 <CAEo-6+TQA3=Yd1rw5gZJ1xi7YbtJY4Xkt_bM3Rs7zjKKZPOnsw@mail.gmail.com>
In-Reply-To: 
 <CAEo-6+TQA3=Yd1rw5gZJ1xi7YbtJY4Xkt_bM3Rs7zjKKZPOnsw@mail.gmail.com>
Content-Type: multipart/alternative;
 boundary="------------010904040206090208090306"

This is a multi-part message in MIME format.
--------------010904040206090208090306
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

HI Guys,
I am trying to run a few MR jobs in a succession, some of the jobs don't 
need that much memory and others do. I want to be able to tell hadoop 
how much memory should be allocated  for the mappers of each job.
I know how to increase the memory for a mapper JVM, through the mapred xml.
I tried manually setting the mapreduce.reduce.java.opts= 
-Xmx<someNumber>m , but wasn't picked up by the mapper jvm, the global 
setting was always been picked up .

In summation
Job 1 - Mappers need only 250 Mg of Ram
Job2 - Mapper
            Reducer need around - 2Gb

I don't want to be able to set those restrictions prior to submitting 
the job to my hadoop cluster.

--------------010904040206090208090306
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: 8bit

<html>
  <head>
    <meta content="text/html; charset=UTF-8" http-equiv="Content-Type">
  </head>
  <body bgcolor="#FFFFFF" text="#000000">
    HI Guys,<br>
    I am trying to run a few MR jobs in a succession, some of the jobs
    don't need that much memory and others do. I want to be able to tell
    hadoop how much memory should be allocated  for the mappers of each
    job. <br>
    I know how to increase the memory for a mapper JVM, through the
    mapred xml. <br>
    I tried manually setting the  <span class="nt" style="box-sizing:
      border-box; color: rgb(0, 0, 128);"></span>mapreduce.reduce.java.opts<span
      class="nt" style="box-sizing: border-box; color: rgb(0, 0, 128);"></span>
    = <span class="nt" style="box-sizing: border-box; color: rgb(0, 0,
      128);"></span>-Xmx&lt;someNumber&gt;m , but wasn't picked up by
    the mapper jvm, the global setting was always been picked up . <br>
    <br>
    In summation <br>
    Job 1 - Mappers need only 250 Mg of Ram <br>
    Job2 - Mapper <br>
               Reducer need around - 2Gb <br>
    <br>
    I don't want to be able to set those restrictions prior to
    submitting the job to my hadoop cluster. <br>
  </body>
</html>

--------------010904040206090208090306--