hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From 某因幡 <tewil...@gmail.com>
Subject Re: Extremely slow when loading small amount of data from HBase
Date Thu, 06 Sep 2012 03:17:20 GMT
Yes hbase.hregion.max.filesize was set to default 256MB and it was too low.


2012/9/5 Jean-Marc Spaggiari <jean-marc@spaggiari.org>:
> But I think you should also look at why we have so many regions...
> Because even if you merge them manually now, you might face the same
> issu soon.
>
> 2012/9/5, n keywal <nkeywal@gmail.com>:
>> Hi,
>>
>> With 8 regionservers, yes, you can. Target a few hundreds by default imho.
>>
>> N.
>>
>> On Wed, Sep 5, 2012 at 4:55 AM, 某因幡 <tewilove@gmail.com> wrote:
>>
>>> +HBase users.
>>>
>>>
>>> ---------- Forwarded message ----------
>>> From: Dmitriy Ryaboy <dvryaboy@gmail.com>
>>> Date: 2012/9/4
>>> Subject: Re: Extremely slow when loading small amount of data from HBase
>>> To: "user@pig.apache.org" <user@pig.apache.org>
>>>
>>>
>>> I think the hbase folks recommend something like 40 regions per node
>>> per table, but I might be misremembering something. Have you tried
>>> emailing the hbase users list?
>>>
>>> On Sep 4, 2012, at 3:39 AM, 某因幡 <tewilove@gmail.com> wrote:
>>>
>>> > After merging ~8000 regions to ~4000 on an 8-node cluster the things
>>> > is getting better.
>>> > Should I continue merging?
>>> >
>>> >
>>> > 2012/8/29 Dmitriy Ryaboy <dvryaboy@gmail.com>:
>>> >> Can you try the same scans with a regular hbase mapreduce job? If you
>>> see the same problem, it's an hbase issue. Otherwise, we need to see the
>>> script and some facts about your table (how many regions, how many rows,
>>> how big a cluster, is the small range all on one region server, etc)
>>> >>
>>> >> On Aug 27, 2012, at 11:49 PM, 某因幡 <tewilove@gmail.com> wrote:
>>> >>
>>> >>> When I load a range of data from HBase simply using row key range
in
>>> >>> HBaseStorageHandler, I find that the speed is acceptable when I'm
>>> >>> trying to load some tens of millions rows or more, while the only
map
>>> >>> ends up in a timeout when it's some thousands of rows.
>>> >>> What is going wrong here? Tried both Pig-0.9.2 and Pig-0.10.0.
>>> >>>
>>> >>>
>>> >>> --
>>> >>> language: Chinese, Japanese, English
>>> >
>>> >
>>> >
>>> > --
>>> > language: Chinese, Japanese, English
>>>
>>>
>>> --
>>> language: Chinese, Japanese, English
>>>
>>



-- 
language: Chinese, Japanese, English

Mime
View raw message