hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Hsieh (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-8258) Make mapreduce tests pass on hadoop2
Date Fri, 05 Apr 2013 23:33:17 GMT

    [ https://issues.apache.org/jira/browse/HBASE-8258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13624173#comment-13624173
] 

Jonathan Hsieh commented on HBASE-8258:
---------------------------------------

Hm.. On my second try, it looks like hadoop1 has got some failures, but the hadoop2 build
is good. 

On my first run against hadoop2 I had problems with vmem overuse:
{code}
2013-04-05 14:57:45,983 WARN  [AsyncDispatcher event handler] resourcemanager.RMAuditLogger(255):
USER=jenkins	OPERATION=Application Finished - Failed	TARGET=RMAppManager	RESULT=FAILURE	DESCRIPTION=App
failed with state: FAILED	PERMISSIONS=Application application_1365199040071_0001 failed 1
times due to AM Container for appattempt_1365199040071_0001_000001 exited with  exitCode:
143 due to: Container [pid=24860,containerID=container_1365199040071_0001_01_000001] is running
beyond virtual memory limits. Current usage: 269.3mb of 2.0gb physical memory used; 6.3gb
of 4.2gb virtual memory used. Killing container.
{code}

I've uncommented the pmem/vmem lines in the patch to make hadoop2 happy.
{code}
    // Tests were failing because this process used 6GB of virtual memory and was getting
killed.
    // we up the VM usable so that processes don't get killed.
    conf.setInt("yarn.nodemanager.resource.memory-mb", 8 * 1024);
    conf.setFloat("yarn.nodemanager.vmem-pmem-ratio", 8.0f);
{code}

Digging into the hadoop 1 issues now.



                
> Make mapreduce tests pass on hadoop2
> ------------------------------------
>
>                 Key: HBASE-8258
>                 URL: https://issues.apache.org/jira/browse/HBASE-8258
>             Project: HBase
>          Issue Type: Bug
>          Components: mapreduce
>            Reporter: stack
>            Priority: Blocker
>             Fix For: 0.95.1
>
>         Attachments: 8258-plain.txt, 8258-v1-hadoop-2.0.txt, 8258-v2-hadoop-2.0.txt,
8258-v4-hadoop-2.0.txt, hbase-8258-simple.patch
>
>
> HBASE-7904 was a first attempt at making this work but it got lost in the weeds.
> This is a new attempt at making hbase mapreduce jobs run on hadoop2 (w/o breaking mapreduce
on hadoop1)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message