hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kireet <kir...@feedly.com>
Subject Re: HConnectionManager$HConnectionImplementation.locateRegionInMeta
Date Thu, 30 May 2013 13:12:45 GMT


Would there be a problem if our cluster is 0.94 and we use a 0.95 client?

I am not familiar with the HBase code base, but I did a dump of the 
thread that is actually running (below). It seems like it is related to 
the issue you mentioned as the running thread is doing the prefetch 
logic. Would pre-splitting tables help here? We are doing some 
performance tests and essentially starting from an empty instance.

java.lang.Thread.State: WAITING (on object monitor)
  at java.lang.Object.wait(Native Method)
  at java.lang.Object.wait(Object.java:503)
  at org.apache.zookeeper.ClientCnxn.submitRequest(ClientCnxn.java:1309)
- locked <0x00000000e10cf830> (a org.apache.zookeeper.ClientCnxn$Packet)
  at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1036)
  at 
org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:172)
  at org.apache.hadoop.hbase.zookeeper.ZKUtil.checkExists(ZKUtil.java:450)
  at 
org.apache.hadoop.hbase.zookeeper.ZooKeeperNodeTracker.checkIfBaseNodeAvailable(ZooKeeperNodeTracker.java:208)
  at 
org.apache.hadoop.hbase.zookeeper.RootRegionTracker.waitRootRegionLocation(RootRegionTracker.java:77)
  at 
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:874)
  at 
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:987)
  at 
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:885)
  at 
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:846)
  at org.apache.hadoop.hbase.client.HTable.finishSetup(HTable.java:234)
  at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:174)
  at 
org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:160)
  at 
org.apache.hadoop.hbase.client.MetaScanner.access$000(MetaScanner.java:54)
  at 
org.apache.hadoop.hbase.client.MetaScanner$1.connect(MetaScanner.java:133)
  at 
org.apache.hadoop.hbase.client.MetaScanner$1.connect(MetaScanner.java:130)
  at 
org.apache.hadoop.hbase.client.HConnectionManager.execute(HConnectionManager.java:383)
  at 
org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:130)
  at 
org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:105)
  at 
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.prefetchRegionCache(HConnectionManager.java:947)
  at 
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1002)
- locked <0x00000000e882f778> (a java.lang.Object)
  at 
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:889)
  at 
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:846)
  at 
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchCallback(HConnectionManager.java:1533)
  at 
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1418)
  at org.apache.hadoop.hbase.client.HTable.batch(HTable.java:702)
  at org.apache.hadoop.hbase.client.HTable.get(HTable.java:671)
  at 
org.apache.hadoop.hbase.client.HTablePool$PooledHTable.get(HTablePool.java:367)

Thanks
Kireet

On 5/30/13 12:16 AM, Anoop John wrote:
> Can you have a look at issue HBASE-8476?  Seems related?  A fix is
> available in HBASE-8346's patch..
>
> -Anoop-
>
> On Thu, May 30, 2013 at 9:21 AM, Kireet <kireet-Teh5dPVPL8nQT0dZR+AlfA@public.gmane.org>
wrote:
>
>> We are running hbase 0.94.6 in a concurrent environment and we are seeing
>> the majority of our code stuck in this method at the synchronized block
>> (line 997). Is there some configuration we can change to avoid it? It seems
>> like we could disable prefetching, but that sounds like it could cause
>> other problems?
>>
>>   - org.apache.hadoop.hbase.**client.HConnectionManager$**
>> HConnectionImplementation.**locateRegionInMeta(byte[], byte[], byte[],
>> boolean, java.lang.Object, boolean) @bci=149, line=997 (Compiled frame)
>>   - org.apache.hadoop.hbase.**client.HConnectionManager$**
>> HConnectionImplementation.**locateRegion(byte[], byte[], boolean,
>> boolean) @bci=212, line=889 (Compiled frame)
>>   - org.apache.hadoop.hbase.**client.HConnectionManager$**
>> HConnectionImplementation.**locateRegion(byte[], byte[]) @bci=5, line=846
>> (Compiled frame)
>>   - org.apache.hadoop.hbase.**client.HConnectionManager$**
>> HConnectionImplementation.**processBatchCallback(java.**util.List,
>> byte[], java.util.concurrent.**ExecutorService, java.lang.Object[],
>> org.apache.hadoop.hbase.**client.coprocessor.Batch$**Callback) @bci=194,
>> line=1533 (Compiled frame)
>>   - org.apache.hadoop.hbase.**client.HConnectionManager$**
>> HConnectionImplementation.**processBatch(java.util.List, byte[],
>> java.util.concurrent.**ExecutorService, java.lang.Object[]) @bci=30,
>> line=1418 (Compiled frame)
>>   - org.apache.hadoop.hbase.**client.HTable.batch(java.util.**List)
>> @bci=24, line=702 (Compiled frame)
>>   - org.apache.hadoop.hbase.**client.HTable.get(java.util.**List) @bci=2,
>> line=671 (Compiled frame)
>>   - org.apache.hadoop.hbase.**client.HTablePool$**
>> PooledHTable.get(java.util.**List) @bci=5, line=367 (Compiled frame)
>>
>>
>


Mime
View raw message