Return-Path: X-Original-To: apmail-hbase-dev-archive@www.apache.org Delivered-To: apmail-hbase-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 001E11042E for ; Sat, 23 Nov 2013 20:07:36 +0000 (UTC) Received: (qmail 46979 invoked by uid 500); 23 Nov 2013 20:07:36 -0000 Delivered-To: apmail-hbase-dev-archive@hbase.apache.org Received: (qmail 46916 invoked by uid 500); 23 Nov 2013 20:07:36 -0000 Mailing-List: contact dev-help@hbase.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@hbase.apache.org Delivered-To: mailing list dev@hbase.apache.org Received: (qmail 46906 invoked by uid 99); 23 Nov 2013 20:07:36 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 23 Nov 2013 20:07:35 +0000 Date: Sat, 23 Nov 2013 20:07:35 +0000 (UTC) From: "stack (JIRA)" To: dev@hbase.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Resolved] (HBASE-9322) Contention on the SoftValueSortedMap with multiple clients MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HBASE-9322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack resolved HBASE-9322. -------------------------- Resolution: Won't Fix SoftValueSortedMap was removed by HBASE-9869 so we won't be fixing this one. > Contention on the SoftValueSortedMap with multiple clients > ---------------------------------------------------------- > > Key: HBASE-9322 > URL: https://issues.apache.org/jira/browse/HBASE-9322 > Project: HBase > Issue Type: Bug > Affects Versions: 0.95.2 > Reporter: Jean-Daniel Cryans > Fix For: 0.98.0, 0.96.1 > > > I've been running tests on clusters with "lots" of regions, about 400, and I'm seeing weird contention in the client. > This one shows up a lot around the SoftValueSortedMap. > First I have this blocked thread on I'm not sure what: > {noformat} > "TestClient-12" prio=10 tid=0x00007fb268872000 nid=0x3add waiting for monitor entry [0x00007fb251416000] > java.lang.Thread.State: BLOCKED (on object monitor) > at org.apache.hadoop.hbase.util.SoftValueSortedMap.isEmpty(SoftValueSortedMap.java:210) > - locked <0x00000000c1b70318> (a java.util.TreeMap) > at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getCachedLocation(HConnectionManager.java:1263) > at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1103) > at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:1036) > at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:993) > {noformat} > This client waits on it: > {noformat} > "TestClient-14" prio=10 tid=0x00007fb268876000 nid=0x3adf waiting for monitor entry [0x00007fb251214000] > java.lang.Thread.State: BLOCKED (on object monitor) > at org.apache.hadoop.hbase.util.SoftValueSortedMap.lowerValueByKey(SoftValueSortedMap.java:189) > - waiting to lock <0x00000000c1b70318> (a java.util.TreeMap) > - locked <0x00000000c1b82120> (a org.apache.hadoop.hbase.util.SoftValueSortedMap) > at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getCachedLocation(HConnectionManager.java:1272) > at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1103) > at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:1036) > at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:993) > {noformat} > As you can see it's also holding a lock, which I have 11 other clients waiting on: > {noformat} > "TestClient-13" prio=10 tid=0x00007fb268874000 nid=0x3ade waiting for monitor entry [0x00007fb251315000] > java.lang.Thread.State: BLOCKED (on object monitor) > at org.apache.hadoop.hbase.util.SoftValueSortedMap.lowerValueByKey(SoftValueSortedMap.java:189) > - waiting to lock <0x00000000c1b82120> (a org.apache.hadoop.hbase.util.SoftValueSortedMap) > at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getCachedLocation(HConnectionManager.java:1272) > at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1103) > at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:1036) > at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:993) > {noformat} -- This message was sent by Atlassian JIRA (v6.1#6144)