hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stack <st...@duboce.net>
Subject Re: Slow MR data load to table
Date Wed, 22 Dec 2010 19:43:02 GMT
I took a look at your regionserver log.  As per your above comment,
boring.  More importantly, no blocking going on.

You have 4 column families going on.  Do you have to have this amount
of CFs?  This might explain some slow down.

If it was faster last week and this week its slow though 'nothing' has
changed, it smells environmental.

It looks like you have hooked your Map to TOF... so you should have a
nice little write buffer in HTable going on (You might check).

For sure, you are not swapping?  You have any monitoring of this
cluster going on?  Setting swappyness to zero from 60 is probably a
bit radical.  You want some swap if memory pressure.  60 is too loose.
 If you look at those killed map tasks... why they die?  Because
processes were killed by the kernel?


On Tue, Dec 21, 2010 at 10:10 PM, Bradford Stephens
<bradfordstephens@gmail.com> wrote:
> Unfortunately, changing swappiness didn't seem to help.
> On Tue, Dec 21, 2010 at 4:28 PM, Andrew Purtell <apurtell@apache.org> wrote:
>>> Yes, a good point. Swappiness is set to 60 -- suppose I should set it to 0?
>> Yes.
>> Best regards,
>>    - Andy
>> Problems worthy of attack prove their worth by hitting back.
>>  - Piet Hein (via Tom White)
> --
> Bradford Stephens,
> Founder, Drawn to Scale
> drawntoscalehq.com
> 727.697.7528
> http://www.drawntoscalehq.com --  The intuitive, cloud-scale data
> solution. Process, store, query, search, and serve all your data.
> http://www.roadtofailure.com -- The Fringes of Scalability, Social
> Media, and Computer Science

View raw message