hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Takuya Ueshin (Updated) (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HBASE-4398) If HRegionPartitioner is used in MapReduce, client side configurations are overwritten by hbase-site.xml.
Date Thu, 29 Mar 2012 14:26:27 GMT

     [ https://issues.apache.org/jira/browse/HBASE-4398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Takuya Ueshin updated HBASE-4398:
---------------------------------

    Attachment: HBASE-4398.patch

I attatched a patch file.
Tnis includes a patch for HRegionPartitioner and a test case.
                
> If HRegionPartitioner is used in MapReduce, client side configurations are overwritten
by hbase-site.xml.
> ---------------------------------------------------------------------------------------------------------
>
>                 Key: HBASE-4398
>                 URL: https://issues.apache.org/jira/browse/HBASE-4398
>             Project: HBase
>          Issue Type: Bug
>          Components: mapreduce
>    Affects Versions: 0.90.4
>            Reporter: Takuya Ueshin
>         Attachments: HBASE-4398.patch
>
>
> If HRegionPartitioner is used in MapReduce, client side configurations are overwritten
by hbase-site.xml.
> We can reproduce the problem by the following instructions:
> {noformat}
> - Add HRegionPartitioner.class to the 4th argument of
> TableMapReduceUtil#initTableReducerJob()
> at line around 133
> in src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableMapReduce.java
> - Change or remove "hbase.zookeeper.property.clientPort" property
> in hbase-site.xml ( for example, changed to 12345 ).
> - run testMultiRegionTable()
> {noformat}
> Then I got error messages as following:
> {noformat}
> 2011-09-12 22:28:51,020 DEBUG [Thread-832] zookeeper.ZKUtil(93): hconnection opening
connection to ZooKeeper with ensemble (localhost:12345)
> 2011-09-12 22:28:51,022 INFO  [Thread-832] zookeeper.RecoverableZooKeeper(89): The identifier
of this process is 43200@imac.local
> 2011-09-12 22:28:51,123 WARN  [Thread-832] zookeeper.RecoverableZooKeeper(161): Possibly
transient ZooKeeper exception: org.apache.zookeeper.KeeperException$ConnectionLossException:
KeeperErrorCode = ConnectionLoss for /hbase/master
> 2011-09-12 22:28:51,123 INFO  [Thread-832] zookeeper.RecoverableZooKeeper(173): The 1
times to retry ZooKeeper after sleeping 1000 ms
>  =====
> 2011-09-12 22:29:02,418 ERROR [Thread-832] mapreduce.HRegionPartitioner(125): java.io.IOException:
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@2e54e48d closed
> 2011-09-12 22:29:02,422 WARN  [Thread-832] mapred.LocalJobRunner$Job(256): job_local_0001
> java.lang.NullPointerException
>        at org.apache.hadoop.hbase.mapreduce.HRegionPartitioner.setConf(HRegionPartitioner.java:128)
>        at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:62)
>        at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
>        at org.apache.hadoop.mapred.MapTask$NewOutputCollector.<init>(MapTask.java:527)
>        at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:613)
>        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
>        at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:177)
> {noformat}
> I think HTable should connect to ZooKeeper at port 21818 configured at client side instead
of 12345 in hbase-site.xml
> and It might be caused by "HBaseConfiguration.addHbaseResources(conf);" in HRegionPartitioner#setConf(Configuration).
> And this might mean that all of client side configurations, also configured in hbase-site.xml,
are overwritten caused by this problem.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message