hadoop-hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Joydeep Sen Sarma (JIRA)" <j...@apache.org>
Subject [jira] Created: (HIVE-1520) hive.mapred.local.mem should only be used in case of local mode job submissions
Date Mon, 09 Aug 2010 19:38:17 GMT
hive.mapred.local.mem should only be used in case of local mode job submissions
-------------------------------------------------------------------------------

                 Key: HIVE-1520
                 URL: https://issues.apache.org/jira/browse/HIVE-1520
             Project: Hadoop Hive
          Issue Type: Bug
            Reporter: Joydeep Sen Sarma


Currently - whenever we submit a map-reduce job via a child jvm process, hive sets HADOOP_HEAPSIZE
to hive.mapred.local.mem (thereby limiting the max heap memory of the child jvm). the assumption
being that we are submitting a job for local mode execution and different memory limits apply
for that.

however - one can submit jobs via a child jvm for non local mode execution as well. This is
useful, for example, if hive wants to submit jobs via different hadoop clients (for sending
jobs to different hadoop clusters). in such case, we can use the 'hive.exec.submitviachild'
and 'hadoop.bin.path' to dispatch job via an alternate hadoop client install point. however
in such case, we don't need to set HADOOP_HEAPSIZE. all we are using the child jvm is to run
the small bit of hive code that submits the job (and not for local mode execution).

in this case - we shouldn't be setting the child jvm's memory limit and should leave it to
what the parent's value is.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message