hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ted Yu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-7904) Upgrade hadoop 2.0 dependency to 2.0.4-alpha
Date Sat, 09 Mar 2013 05:33:12 GMT

    [ https://issues.apache.org/jira/browse/HBASE-7904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13597849#comment-13597849
] 

Ted Yu commented on HBASE-7904:
-------------------------------

bq. We are not picking up the yarn defaults?
That was the case.
>From MiniYARNCluster#ResourceManagerWrapper :
{code}
    public synchronized void start() {
      try {
        getConfig().setBoolean(YarnConfiguration.IS_MINI_YARN_CLUSTER, true);
{code}
YARN-129 depended on the above - see MRApps#setMRFrameworkClasspath():
{code}
+    if (conf.getBoolean(YarnConfiguration.IS_MINI_YARN_CLUSTER, false)) {
+      Apps.addToEnvironment(environment, Environment.CLASSPATH.name(),
+          System.getProperty("java.class.path"));
{code}
Here is how our tests are written (excerpt from TestMultiTableInputFormat):
{code}
    // start MR cluster
    TEST_UTIL.startMiniMapReduceCluster();
...
    Configuration c = new Configuration(TEST_UTIL.getConfiguration());
    
    c.set(KEY_STARTROW, start != null ? start : "");
    c.set(KEY_LASTROW, last != null ? last : "");
...
    Job job = new Job(c, jobName);
{code}
Since TEST_UTIL.getConfiguration() didn't contain YarnConfiguration.IS_MINI_YARN_CLUSTER,
classpath wasn't adjusted by MRApps#setMRFrameworkClasspath() which led to job failure.

Between patches v4 and v5, looks like v4 is a bit cleaner.
                
> Upgrade hadoop 2.0 dependency to 2.0.4-alpha
> --------------------------------------------
>
>                 Key: HBASE-7904
>                 URL: https://issues.apache.org/jira/browse/HBASE-7904
>             Project: HBase
>          Issue Type: Task
>            Reporter: Ted Yu
>            Assignee: Ted Yu
>            Priority: Critical
>             Fix For: 0.95.0
>
>         Attachments: 7904.txt, 7904-v2.txt, 7904-v4.txt, 7904-v4.txt, 7904-v5.txt, hbase-7904-v3.txt
>
>
> 2.0.3-alpha has been released.
> We should upgrade the dependency.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message