hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Siddharth Seth (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-449) MRAppMaster classpath not set properly for unit tests in downstream projects
Date Wed, 06 Mar 2013 04:00:14 GMT

    [ https://issues.apache.org/jira/browse/YARN-449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13594332#comment-13594332
] 

Siddharth Seth commented on YARN-449:
-------------------------------------

Ted, good catch on the isMiniYARNCluster property not being set. I assumed that would be in
place.

The MiniCluster, as part of it's startup process, sets parameters like the RM address in the
configuration. 
This is then available via createJobConf in MiniMRCluster, or getConfig in MiniMRClientCluster.
Instead of selectively copying out parameters, downstream projects should really be using
the configuration objects returned by these APIs to submit jobs. That would allow things to
keep working if parameters were changed.

Looked at the PIG code, and that's exactly what it is doing - so the tests passing is expected.
I'm not sure if Hive unit tests will work. In the test command you pasted, I believe TestCliDriver
needs to be replaced with TestMinimrCliDriver to actually get it to use the MiniMRCluster.

IAC, does it make sense for HBase to make use of config objects returned by the "getConfig"
objects so that similar changes in the future don't break unit tests ?
                
> MRAppMaster classpath not set properly for unit tests in downstream projects
> ----------------------------------------------------------------------------
>
>                 Key: YARN-449
>                 URL: https://issues.apache.org/jira/browse/YARN-449
>             Project: Hadoop YARN
>          Issue Type: Bug
>    Affects Versions: 2.0.3-alpha
>            Reporter: Siddharth Seth
>            Priority: Blocker
>         Attachments: hbase-TestHFileOutputFormat-wip.txt, hbase-TestingUtility-wip.txt
>
>
> Post YARN-429, unit tests for HBase continue to fail since the classpath for the MRAppMaster
is not being set correctly.
> Reverting YARN-129 may fix this, but I'm not sure that's the correct solution. My guess
is, as Alexandro pointed out in YARN-129, maven classloader magic is messing up java.class.path.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message