hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From bijieshan <bijies...@huawei.com>
Subject A concurrency issue on SoftValueSortedMap?
Date Wed, 21 Dec 2011 10:48:17 GMT
Hi all,

We use thousand of threads doing the scan operations. One thread got blocked at the position
of "java.util.TreeMap.fixAfterDeletion". It can't come out of the loop of "TreeMap.fixAfterDeletion":

So I think it maybe a concurrency issue. Has someone encountered this?
Thank you.

Jieshan.

Here's the thread dump:

"Thread-923" prio=10 tid=0x00007f3d40553000 nid=0x3ed6 runnable [0x00007f3d05c1b000]
   java.lang.Thread.State: RUNNABLE
                at java.util.TreeMap.fixAfterDeletion(TreeMap.java:2176)
                at java.util.TreeMap.deleteEntry(TreeMap.java:2151)
                at java.util.TreeMap.remove(TreeMap.java:585)
                at java.util.TreeMap$NavigableSubMap.remove(TreeMap.java:1395)
                at org.apache.hadoop.hbase.util.SoftValueSortedMap.get(SoftValueSortedMap.java:101)
                - locked <0x00007f3d92937748> (a org.apache.hadoop.hbase.util.SoftValueSortedMap)
                at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getCachedLocation(HConnectionManager.java:846)
                at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:668)
                at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:594)
                at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:559)
                at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionLocation(HConnectionManager.java:416)
                at org.apache.hadoop.hbase.client.ServerCallable.instantiateServer(ServerCallable.java:57)
                at org.apache.hadoop.hbase.client.ScannerCallable.instantiateServer(ScannerCallable.java:63)
                at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionServerWithRetries(HConnectionManager.java:1018)
                at org.apache.hadoop.hbase.client.HTable$ClientScanner.nextScanner(HTable.java:1104)
                at org.apache.hadoop.hbase.client.HTable$ClientScanner.initialize(HTable.java:1027)
                at org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:535)
                at com.huawei.icbc.query.SingleTabQuery.querybatch(SingleTabQuery.java:197)
                at com.huawei.icbc.benchmark.SingleTabQueryAction.query(SingleTabQueryAction.java:181)
                at framework.QueryThread.run(QueryThread.java:47)

"Thread-923" prio=10 tid=0x00007f3d40553000 nid=0x3ed6 runnable [0x00007f3d05c1a000]
   java.lang.Thread.State: RUNNABLE
                at java.util.TreeMap.fixAfterDeletion(TreeMap.java:2193)
                at java.util.TreeMap.deleteEntry(TreeMap.java:2151)
                at java.util.TreeMap.remove(TreeMap.java:585)
                at java.util.TreeMap$NavigableSubMap.remove(TreeMap.java:1395)
                at org.apache.hadoop.hbase.util.SoftValueSortedMap.get(SoftValueSortedMap.java:101)
                - locked <0x00007f3d94f24f70> (a org.apache.hadoop.hbase.util.SoftValueSortedMap)
                at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getCachedLocation(HConnectionManager.java:846)
                at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:668)
                at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:594)
                at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:559)
                at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionLocation(HConnectionManager.java:416)
                at org.apache.hadoop.hbase.client.ServerCallable.instantiateServer(ServerCallable.java:57)
                at org.apache.hadoop.hbase.client.ScannerCallable.instantiateServer(ScannerCallable.java:63)
                at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionServerWithRetries(HConnectionManager.java:1018)
                at org.apache.hadoop.hbase.client.HTable$ClientScanner.nextScanner(HTable.java:1104)
                at org.apache.hadoop.hbase.client.HTable$ClientScanner.initialize(HTable.java:1027)
                at org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:535)
                at com.huawei.icbc.query.SingleTabQuery.querybatch(SingleTabQuery.java:197)
                at com.huawei.icbc.benchmark.SingleTabQueryAction.query(SingleTabQueryAction.java:181)
                at framework.QueryThread.run(QueryThread.java:47)



Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message