Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
Received-SPF: pass (athena.apache.org: domain of yuzhihong@gmail.com
 designates 209.85.160.41 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=references:in-reply-to:mime-version:content-transfer-encoding
         :content-type:message-id:cc:x-mailer:from:subject:date:to;
        b=G7W/d4s8o4lXYVL2EPTQE/Ganj4jmaGQlGTuqmtyNIxxcwtzbt5kDqcDE9rePZ++Lx
         fCYxfS78uCNYi0IPaAw/u2PEMLjdjdae/3gnKjxahM0yvRkpPrAJg2hcjPSCv4ct727e
         FwnEJzmA50tnJHxBNpIQ/SlJSJCVV/zVEjtEA=
References: <SNT141-w36BD7EC39CB76D8C48BADABF980@phx.gbl>
 <SNT141-w3492609E70EDE4BB9134A4BF980@phx.gbl>
 <BANLkTimmhaVZBky_LXbmGGPUOkn-QUyQsQ@mail.gmail.com>
 <SNT141-w140DB1359785A7D42AC700BF980@phx.gbl>
 <BANLkTikrrAg+bTmY7Qnfn73SdeAu7CU33w@mail.gmail.com>
 <BANLkTimMTFPKgQpgyRPG0JwH=qg2_ChqdQ@mail.gmail.com>
 <E41F4E6CFB766541888FC855399F07C502EBAD@SZXEML506-MBS.china.huawei.com>
 <BANLkTi=ws7aJEt7smEPMe6joY8ESG7itHA@mail.gmail.com>
 <E41F4E6CFB766541888FC855399F07C502EC72@SZXEML506-MBS.china.huawei.com>
 <BANLkTikMH5iytTRFhQbY4r5ofVyRXuez3A@mail.gmail.com>
 <BANLkTimnaD39Mdp7Yox7qeJJTc5ewAh=OA@mail.gmail.com>
 <E41F4E6CFB766541888FC855399F07C5031F56@szxeml506-mbx.china.huawei.com>
In-Reply-To: 
 <E41F4E6CFB766541888FC855399F07C5031F56@szxeml506-mbx.china.huawei.com>
Mime-Version: 1.0 (iPhone Mail 8E401)
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=utf-8
Message-Id: <4200ACF8-8478-4966-8A76-C2B43469E438@gmail.com>
Cc: "user@hbase.apache.org" <user@hbase.apache.org>
From: Ted Yu <yuzhihong@gmail.com>
Subject: Re: A question about client
Date: Thu, 12 May 2011 02:51:44 -0700
To: "user@hbase.apache.org" <user@hbase.apache.org>

Yes please.=20


On May 12, 2011, at 12:49 AM, Gaojinchao <gaojinchao@huawei.com> wrote:

> Does it need open a issue ?=20
>=20
> -----=E9=82=AE=E4=BB=B6=E5=8E=9F=E4=BB=B6-----
> =E5=8F=91=E4=BB=B6=E4=BA=BA: Ted Yu [mailto:yuzhihong@gmail.com]=20
> =E5=8F=91=E9=80=81=E6=97=B6=E9=97=B4: 2011=E5=B9=B45=E6=9C=8811=E6=97=A5 1=
0:31
> =E6=94=B6=E4=BB=B6=E4=BA=BA: user@hbase.apache.org
> =E4=B8=BB=E9=A2=98: Re: A question about client
>=20
> I think the second explanation is plausible.
> From
> http://download.oracle.com/javase/1.4.2/docs/api/java/lang/ref/SoftReferen=
ce.html
> :
> Soft reference objects, which are cleared at the discretion of the garbage=

> collector in response to memory demand.
>=20
> I think we should check whether possibleRegion is null in the code below.
>=20
> 2011/5/10 Jean-Daniel Cryans <jdcryans@apache.org>
>=20
>> Are you running a modified YCSB by any chance? Because last time I
>> looked at that code it didn't share the HTables between threads and it
>> looks like it's doing something like that.
>>=20
>> Looking more deeper at the code, the NoSuchElementException is thrown
>> because the map is empty. This is what that code looks like:
>>=20
>>     if (!matchingRegions.isEmpty()) {
>>       HRegionLocation possibleRegion =3D
>>         matchingRegions.get(matchingRegions.lastKey());
>>=20
>> So to me it seems that the only way you would get this exception is if
>> someone emptied the map between the isEmpty call and lastKey which
>> shouldn't happen if HTables aren't shared.
>>=20
>> The only other way it seems it could happen, and it's a stretch, is
>> that since the regions are kept in a SoftValueSortedMap then the GC
>> would have removed the elements you needed exactly between those two
>> lines...  Is it easy for you to recreate the issue?
>>=20
>> Thx a bunch,
>>=20
>> J-D
>>=20
>> On Mon, May 9, 2011 at 11:34 PM, Gaojinchao <gaojinchao@huawei.com> wrote=
:
>>> Hbase version: 0.90.2 .
>>> I merged patches:
>>> HBASE-3773      Set ZK max connections much higher in 0.90
>>> HBASE-3771      All jsp pages don't clean their HBA
>>> HBASE-3783      hbase-0.90.2.jar exists in hbase root and in 'lib/'
>>> HBASE-3756      Can't move META or ROOT from shell
>>> HBASE-3744      createTable blocks until all regions are out of
>> transition
>>> HBASE-3712      HTable.close() doesn't shutdown thread pool
>>> HBASE-3750      HTablePool.putTable() should call
>> tableFactory.releaseHTableInterface() for discarded table
>>> HBASE-3722      A lot of data is lost when name node crashed
>>> HBASE-3800      If HMaster is started after NN without starting DN in
>> Hbase 090.2 then HMaster is not able to start due to AlreadyCreatedExcept=
ion
>> for /hbase/hbase.version
>>> HBASE-3749      Master can't exit when open port failed
>>>=20
>>> -----=E9=82=AE=E4=BB=B6=E5=8E=9F=E4=BB=B6-----
>>> =E5=8F=91=E4=BB=B6=E4=BA=BA: jdcryans@gmail.com [mailto:jdcryans@gmail.c=
om] =E4=BB=A3=E8=A1=A8 Jean-Daniel
>> Cryans
>>> =E5=8F=91=E9=80=81=E6=97=B6=E9=97=B4: 2011=E5=B9=B45=E6=9C=8810=E6=97=A5=
 1:17
>>> =E6=94=B6=E4=BB=B6=E4=BA=BA: user@hbase.apache.org
>>> =E4=B8=BB=E9=A2=98: Re: A question about client
>>>=20
>>> TreeMap isn't concurrent and it seems it was used that way? I know you
>>> guys are testing a bunch of different things at the same time so which
>>> HBase version and which patches were you using when you got that?
>>>=20
>>> Thx,
>>>=20
>>> J-D
>>>=20
>>> On Mon, May 9, 2011 at 5:22 AM, Gaojinchao <gaojinchao@huawei.com>
>> wrote:
>>>>   I used ycsb to put data and threw exception.
>>>>   Who can give me some suggestion?
>>>>=20
>>>>  Hbase Code:
>>>>     // Cut the cache so that we only get the part that could contain
>>>>     // regions that match our key
>>>>     SoftValueSortedMap<byte[], HRegionLocation> matchingRegions =3D
>>>>       tableLocations.headMap(row);
>>>>=20
>>>>     // if that portion of the map is empty, then we're done. otherwise,=

>>>>     // we need to examine the cached location to verify that it is
>>>>     // a match by end key as well.
>>>>     if (!matchingRegions.isEmpty()) {
>>>>       HRegionLocation possibleRegion =3D
>>>>         matchingRegions.get(matchingRegions.lastKey());
>>>>=20
>>>>   ycsb client log:
>>>>=20
>>>>   [java] begin StatusThread run
>>>>    [java] java.util.NoSuchElementException
>>>>    [java]     at java.util.TreeMap.key(TreeMap.java:1206)
>>>>    [java]     at
>> java.util.TreeMap$NavigableSubMap.lastKey(TreeMap.java:1435)
>>>>    [java]     at
>> org.apache.hadoop.hbase.util.SoftValueSortedMap.lastKey(SoftValueSortedMa=
p.java:131)
>>>>    [java]     at
>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementati=
on.getCachedLocation(HConnectionManager.java:841)
>>>>    [java]     at
>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementati=
on.locateRegionInMeta(HConnectionManager.java:664)
>>>>    [java]     at
>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementati=
on.locateRegion(HConnectionManager.java:590)
>>>>    [java]     at
>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementati=
on.processBatch(HConnectionManager.java:1114)
>>>>    [java]     at
>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementati=
on.processBatchOfPuts(HConnectionManager.java:1234)
>>>>    [java]     at
>> org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:819)
>>>>    [java]     at
>> org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:675)
>>>>    [java]     at
>> org.apache.hadoop.hbase.client.HTable.put(HTable.java:665)
>>>>    [java]     at com.yahoo.ycsb.db.HBaseClient.update(Unknown Source)
>>>>    [java]     at com.yahoo.ycsb.db.HBaseClient.insert(Unknown Source)
>>>>    [java]     at com.yahoo.ycsb.DBWrapper.insert(Unknown Source)
>>>>    [java]     at com.yahoo.ycsb.workloads.MyWorkload.doInsert(Unknown
>> Source)
>>>>    [java]     at com.yahoo.ycsb.ClientThread.run(Unknown Source)
>>>>=20
>>>=20
>>=20