hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: A question about client
Date Thu, 12 May 2011 09:51:44 GMT
Yes please. 



On May 12, 2011, at 12:49 AM, Gaojinchao <gaojinchao@huawei.com> wrote:

> Does it need open a issue ? 
> 
> -----邮件原件-----
> 发件人: Ted Yu [mailto:yuzhihong@gmail.com] 
> 发送时间: 2011年5月11日 10:31
> 收件人: user@hbase.apache.org
> 主题: Re: A question about client
> 
> I think the second explanation is plausible.
> From
> http://download.oracle.com/javase/1.4.2/docs/api/java/lang/ref/SoftReference.html
> :
> Soft reference objects, which are cleared at the discretion of the garbage
> collector in response to memory demand.
> 
> I think we should check whether possibleRegion is null in the code below.
> 
> 2011/5/10 Jean-Daniel Cryans <jdcryans@apache.org>
> 
>> Are you running a modified YCSB by any chance? Because last time I
>> looked at that code it didn't share the HTables between threads and it
>> looks like it's doing something like that.
>> 
>> Looking more deeper at the code, the NoSuchElementException is thrown
>> because the map is empty. This is what that code looks like:
>> 
>>     if (!matchingRegions.isEmpty()) {
>>       HRegionLocation possibleRegion =
>>         matchingRegions.get(matchingRegions.lastKey());
>> 
>> So to me it seems that the only way you would get this exception is if
>> someone emptied the map between the isEmpty call and lastKey which
>> shouldn't happen if HTables aren't shared.
>> 
>> The only other way it seems it could happen, and it's a stretch, is
>> that since the regions are kept in a SoftValueSortedMap then the GC
>> would have removed the elements you needed exactly between those two
>> lines...  Is it easy for you to recreate the issue?
>> 
>> Thx a bunch,
>> 
>> J-D
>> 
>> On Mon, May 9, 2011 at 11:34 PM, Gaojinchao <gaojinchao@huawei.com> wrote:
>>> Hbase version: 0.90.2 .
>>> I merged patches:
>>> HBASE-3773      Set ZK max connections much higher in 0.90
>>> HBASE-3771      All jsp pages don't clean their HBA
>>> HBASE-3783      hbase-0.90.2.jar exists in hbase root and in 'lib/'
>>> HBASE-3756      Can't move META or ROOT from shell
>>> HBASE-3744      createTable blocks until all regions are out of
>> transition
>>> HBASE-3712      HTable.close() doesn't shutdown thread pool
>>> HBASE-3750      HTablePool.putTable() should call
>> tableFactory.releaseHTableInterface() for discarded table
>>> HBASE-3722      A lot of data is lost when name node crashed
>>> HBASE-3800      If HMaster is started after NN without starting DN in
>> Hbase 090.2 then HMaster is not able to start due to AlreadyCreatedException
>> for /hbase/hbase.version
>>> HBASE-3749      Master can't exit when open port failed
>>> 
>>> -----邮件原件-----
>>> 发件人: jdcryans@gmail.com [mailto:jdcryans@gmail.com] 代表 Jean-Daniel
>> Cryans
>>> 发送时间: 2011年5月10日 1:17
>>> 收件人: user@hbase.apache.org
>>> 主题: Re: A question about client
>>> 
>>> TreeMap isn't concurrent and it seems it was used that way? I know you
>>> guys are testing a bunch of different things at the same time so which
>>> HBase version and which patches were you using when you got that?
>>> 
>>> Thx,
>>> 
>>> J-D
>>> 
>>> On Mon, May 9, 2011 at 5:22 AM, Gaojinchao <gaojinchao@huawei.com>
>> wrote:
>>>>   I used ycsb to put data and threw exception.
>>>>   Who can give me some suggestion?
>>>> 
>>>>  Hbase Code:
>>>>     // Cut the cache so that we only get the part that could contain
>>>>     // regions that match our key
>>>>     SoftValueSortedMap<byte[], HRegionLocation> matchingRegions =
>>>>       tableLocations.headMap(row);
>>>> 
>>>>     // if that portion of the map is empty, then we're done. otherwise,
>>>>     // we need to examine the cached location to verify that it is
>>>>     // a match by end key as well.
>>>>     if (!matchingRegions.isEmpty()) {
>>>>       HRegionLocation possibleRegion =
>>>>         matchingRegions.get(matchingRegions.lastKey());
>>>> 
>>>>   ycsb client log:
>>>> 
>>>>   [java] begin StatusThread run
>>>>    [java] java.util.NoSuchElementException
>>>>    [java]     at java.util.TreeMap.key(TreeMap.java:1206)
>>>>    [java]     at
>> java.util.TreeMap$NavigableSubMap.lastKey(TreeMap.java:1435)
>>>>    [java]     at
>> org.apache.hadoop.hbase.util.SoftValueSortedMap.lastKey(SoftValueSortedMap.java:131)
>>>>    [java]     at
>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getCachedLocation(HConnectionManager.java:841)
>>>>    [java]     at
>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:664)
>>>>    [java]     at
>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:590)
>>>>    [java]     at
>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1114)
>>>>    [java]     at
>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchOfPuts(HConnectionManager.java:1234)
>>>>    [java]     at
>> org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:819)
>>>>    [java]     at
>> org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:675)
>>>>    [java]     at
>> org.apache.hadoop.hbase.client.HTable.put(HTable.java:665)
>>>>    [java]     at com.yahoo.ycsb.db.HBaseClient.update(Unknown Source)
>>>>    [java]     at com.yahoo.ycsb.db.HBaseClient.insert(Unknown Source)
>>>>    [java]     at com.yahoo.ycsb.DBWrapper.insert(Unknown Source)
>>>>    [java]     at com.yahoo.ycsb.workloads.MyWorkload.doInsert(Unknown
>> Source)
>>>>    [java]     at com.yahoo.ycsb.ClientThread.run(Unknown Source)
>>>> 
>>> 
>> 

Mime
View raw message