hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Peter Rietzler <peter.rietz...@smarter-ecommerce.com>
Subject Re: HTable.put() with Hbase 0.20.1
Date Wed, 11 Nov 2009 10:07:44 GMT

Hi Jean

I've managed to solve this issue without changing the configuration
properties. It seems that I've had another problem ... sorry. 

Puts are now ~ the same as in 0.19.3. Should that have been improved with
0.20.1 ?

I have not changed the hbase.regionserver.flushlogentries (this is set to
100 in hbase-default.xml shipped with 0.20.1). Did you mean that I should
change this back to 100000. What implications would such a change have ? We
had problems with data loss in previous versions - does this change affect
this issue or is data loss not a problem with 0.20.1 anymore ? 

Thanks!

Peter



Jean-Daniel Cryans-2 wrote:
> 
> Peter,
> 
> One thing that changed is the default number of entries we keep in the
> write ahead logs that went from 100 000 to 100 mainly because we
> wanted to be safe in the case of 1MB rows. This can easily slow down
> the inserts a lot. To be sure, try resetting
> hbase.regionserver.flushlogentries to 100 in hbase-site.xml and
> restart HBase.
> 
> J-D
> 
> On Mon, Nov 9, 2009 at 10:23 AM, Peter Rietzler
> <peter.rietzler@smarter-ecommerce.com> wrote:
>>
>> Hi
>>
>> We currently switch from HBase 0.19.3 to 0.20.1 and experience very slow
>> inserts in comparison to 0.19.3.
>>
>> Inserting 10.000 rows with around 100 columns (pretty small content) per
>> row
>> takes around 26 seconds. I am using HTable.put(List<Row>) with batches of
>> 1000 rows. I've already set HTable.setAutoCommit(false) and tried to set
>> the
>> write buffer size to around 12MB (as seen in some previous posts).
>>
>> Our test currently just uses a standalone instance of HBase. With 0.19.3
>> the
>> insert took a couple of seconds (using BatchUpdate).
>>
>> Can someone tell me what I'm doing wrong ?
>>
>> Thanks in advance!
>>
>> Cheers,
>> Peter
>> --
>> View this message in context:
>> http://old.nabble.com/HTable.put%28%29-with-Hbase-0.20.1-tp26271241p26271241.html
>> Sent from the HBase User mailing list archive at Nabble.com.
>>
>>
> 
> 

-- 
View this message in context: http://old.nabble.com/HTable.put%28%29-with-Hbase-0.20.1-tp26271241p26298872.html
Sent from the HBase User mailing list archive at Nabble.com.


Mime
View raw message