hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Hsieh (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HBASE-8258) Make mapreduce tests pass on hadoop2
Date Thu, 04 Apr 2013 14:35:17 GMT

    [ https://issues.apache.org/jira/browse/HBASE-8258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13622371#comment-13622371
] 

Jonathan Hsieh edited comment on HBASE-8258 at 4/4/13 2:33 PM:
---------------------------------------------------------------

The old theory I mentioned above is not relevant.  Here's the new idea I'm working on.

* Certain mapreduce on hadoop2 minicluster jobs are failing, because 
* we are getting an UndeclaredThrowableException when waiting for the MR job to complete,
because
* we get a ConnectException from the mr client trying to talk to 0.0.0.0:8032 (the resource
manager port), because
* ... there is a gap here between the local runs and the jenkins runs ... 
* the Yarn MRAppMaster for the job is getting killed the Yarn nodemanager[1], (separate processes)
because
* virtual memory is being over provisioned in the MRAppMaster

[1] 
{code}
2013-04-04 04:46:18,895 WARN  [AsyncDispatcher event handler] resourcemanager.RMAuditLogger(255):
USER=jenkins	OPERATION=Application Finished - Failed	TARGET=RMAppManager	RESULT=FAILURE	DESCRIPTION=App
failed with state: FAILED	PERMISSIONS=Application application_1365075961471_0001 failed 1
times due to AM Container for appattempt_1365075961471_0001_000001 exited with  exitCode:
143 due to: Container [pid=31600,containerID=container_1365075961471_0001_01_000001] is running
beyond virtual memory limits. Current usage: 259.7mb of 2.0gb physical memory used; 5.6gb
of 4.2gb virtual memory used. Killing container.
{code}
                
      was (Author: jmhsieh):
    The old theory I mentioned above is not relevant.  Here's the new idea I'm working on.

* We are getting an UndeclaredThrowableException when waiting for the MR job to complete,
because
* we get a ConnectException from the mr client trying to talk to 0.0.0.0:8032 (the resource
manager port), because
* ... there is a gap here between the local runs and the jenkins runs ... 
* the Yarn MRAppMaster for the job is getting killed the Yarn nodemanager[1], (separate processes)
because
* virtual memory is being over provisioned in the MRAppMaster

[1] 
{code}
2013-04-04 04:46:18,895 WARN  [AsyncDispatcher event handler] resourcemanager.RMAuditLogger(255):
USER=jenkins	OPERATION=Application Finished - Failed	TARGET=RMAppManager	RESULT=FAILURE	DESCRIPTION=App
failed with state: FAILED	PERMISSIONS=Application application_1365075961471_0001 failed 1
times due to AM Container for appattempt_1365075961471_0001_000001 exited with  exitCode:
143 due to: Container [pid=31600,containerID=container_1365075961471_0001_01_000001] is running
beyond virtual memory limits. Current usage: 259.7mb of 2.0gb physical memory used; 5.6gb
of 4.2gb virtual memory used. Killing container.
{code}
                  
> Make mapreduce tests pass on hadoop2
> ------------------------------------
>
>                 Key: HBASE-8258
>                 URL: https://issues.apache.org/jira/browse/HBASE-8258
>             Project: HBase
>          Issue Type: Bug
>          Components: mapreduce
>            Reporter: stack
>            Priority: Blocker
>             Fix For: 0.95.1
>
>         Attachments: 8258-plain.txt
>
>
> HBASE-7904 was a first attempt at making this work but it got lost in the weeds.
> This is a new attempt at making hbase mapreduce jobs run on hadoop2 (w/o breaking mapreduce
on hadoop1)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message