hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Hsieh (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-8258) Make mapreduce tests pass on hadoop2
Date Tue, 09 Apr 2013 14:14:17 GMT

    [ https://issues.apache.org/jira/browse/HBASE-8258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13626635#comment-13626635
] 

Jonathan Hsieh commented on HBASE-8258:
---------------------------------------

I only changed two lines that could modify all other tests 

{code}
+    // Tests were failing because this process used 6GB of virtual memory and was getting
killed.
+    // we up the VM usable so that processes don't get killed.
+    conf.setInt("yarn.nodemanager.resource.memory-mb", 8 * 1024);
+    conf.setFloat("yarn.nodemanager.vmem-pmem-ratio", 8.0f);
+
{code}

doing the math this could mean allocating 64gb of vmem which may be excessive.  I removed
the *.memory-mb line (I believe the default is 1.5gb -- maxing out vmem at 12gb).

                
> Make mapreduce tests pass on hadoop2
> ------------------------------------
>
>                 Key: HBASE-8258
>                 URL: https://issues.apache.org/jira/browse/HBASE-8258
>             Project: HBase
>          Issue Type: Bug
>          Components: mapreduce
>            Reporter: stack
>            Priority: Blocker
>             Fix For: 0.95.1
>
>         Attachments: 8258-plain.txt, 8258-v1-hadoop-2.0.txt, 8258-v2-hadoop-2.0.txt,
8258-v4-hadoop-2.0.txt, hbase-8258-simple.patch, hbase-8258.simple.v2.patch, hbase-8258.simple.v3.patch,
hbase-8258.simple.v4.patch
>
>
> HBASE-7904 was a first attempt at making this work but it got lost in the weeds.
> This is a new attempt at making hbase mapreduce jobs run on hadoop2 (w/o breaking mapreduce
on hadoop1)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message