hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ted Yu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-7904) Make mapreduce jobs pass based on 2.0.4-alpha
Date Fri, 29 Mar 2013 14:08:21 GMT

    [ https://issues.apache.org/jira/browse/HBASE-7904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13617363#comment-13617363

Ted Yu commented on HBASE-7904:

bq. Why this change?
When SecureBulkLoadEndpoint tries to access hdfs, org.apache.hadoop.ipc.RPC.setProtocolEngine()
would call Configuration.setClass(). Without the change, we would get:
2013-03-25 22:36:19,493 ERROR [IPC Server handler 2 on 40900] access.SecureBulkLoadEndpoint$1(240):
Failed to complete bulk load
java.lang.UnsupportedOperationException: Immutable Configuration
  at org.apache.hadoop.hbase.CompoundConfiguration.setClass(CompoundConfiguration.java:474)
  at org.apache.hadoop.ipc.RPC.setProtocolEngine(RPC.java:193)
bq. This seems to be only real addition
I followed your suggestion. Since debugging output from copyNewConfEntries() is no longer
needed, I removed that method.

bq. Did something change in the hadoop2 snapshot?
I would have to go into recent checkins in more detail. This may be answered by a YARN developer.

bq. Why we have to do this now?
We have been doing that for the remaining tests. See one example from testSimpleCase():
    Configuration conf = opts.getConfiguration();

    // copy or add the necessary configuration values from the map reduce config to the hbase
    copyConfigurationValues(UTIL.getConfiguration(), conf);
    args = opts.getRemainingArgs();
bq. Why create a new HBaseConfiguration instance?
I don't see reference to HBaseConfiguration in TestImportExport. The patch doesn't introduce
HBaseConfiguration either.

bq. How does cluster config relate to UTIL.getConfiguration?
cluster is a MiniHBaseCluster instance. UTIL is an HBaseTestingUtility where we store all
yarn-related conf parameters. So they're not the same.
copyConfigurationValues() passes all yarn-related conf parameters from UTIL.getConfiguration()
to conf.
> Make mapreduce jobs pass based on 2.0.4-alpha
> ---------------------------------------------
>                 Key: HBASE-7904
>                 URL: https://issues.apache.org/jira/browse/HBASE-7904
>             Project: HBase
>          Issue Type: Task
>            Reporter: Ted Yu
>            Assignee: Ted Yu
>            Priority: Critical
>             Fix For: 0.95.0
>         Attachments: 7904.txt, 7904-v2-hadoop-2.0.txt, 7904-v2.txt, 7904-v4-hadoop-2.0.txt,
7904-v4.txt, 7904-v4.txt, 7904-v5-hadoop-2.0.txt, 7904-v5.txt, 7904-v6-hadoop-2.0.txt, 7904-v7-hadoop-2.0.txt,
7904-v8-hadoop-2.0.txt, 7904-v8.txt, 7904-v9-hadoop-2.0.txt, 7904-v9.txt, hbase-7904-v3.txt
> 2.0.3-alpha has been released.
> We should upgrade the dependency.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message