hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jean-Daniel Cryans <jdcry...@apache.org>
Subject Re: Performance become slower and slower during inserting
Date Thu, 19 Mar 2009 15:02:02 GMT
This is unfortunate but it's already better in the current trunk with
the new file format. It's still very unstable tho.

J-D

On Thu, Mar 19, 2009 at 10:55 AM, schubert zhang <zsongbo@gmail.com> wrote:
> My data loader MapReduce job like following:1. only use mapper, number of
> reducer is 0.
> 2. mapred.tasktracker.map.tasks.maximum=2
> 3. my input file is about 20MB each (50000 rows, each row have about 32
> column within one family).
> 3. each time the MapReduce job load 11 files (3regionserver * 2 *1.95 = 11)
>
> Yes, I think the META scanning and more region compactions and spliting will
> slow HBase.
>
> Schubert
>
> On Thu, Mar 19, 2009 at 9:07 PM, Jean-Daniel Cryans <jdcryans@apache.org>wrote:
>
>> How many tasks that are writing into HBase are being spawn? One thing
>> that sure explains some slow down is the fact that your HBase clients
>> must build up their META cache, which requires lookups in the META
>> table.
>>
>> J-D
>>
>> On Thu, Mar 19, 2009 at 1:54 AM, schubert zhang <zsongbo@gmail.com> wrote:
>> > I am testing the performance of HBase, after about one weeks's test.
>> > I found the HBase become more and more slow when inserting data.
>> >
>> > (3 regionserver, HBase 0.19.1 and hadoop 0.19.2)
>> >
>> > Each row have about 32 column (in one family), the row have about 400
>> bytes
>> > raw data.
>> >
>> > For example:
>> > 1. when there are only 10-32 regions, the inserting time about 550000
>> rows
>> > is about 3-4 minutes.
>> > 2. when there are about 64 regions, the inserting time about 550000 rows
>> is
>> > about 6-10 minutes.
>> > 3. and then more than 10 minutes.
>> > .....
>> >
>> > Schubert
>> >
>>
>

Mime
View raw message