hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daniel Leffel" <daniel.lef...@gmail.com>
Subject Re: Problem I'm having with Release Candidate 0.1.2
Date Tue, 29 Apr 2008 21:18:30 GMT
I don't know about idle java consuming a load of 0.2... Maybe all that load
goes into figuring out a way to store an int in a few hundred bytes. ;-)

Clearly, you were right about the hadoop-site.xml fix. I don't know where I
read that this was something to do with TableReduce. I kept looking through
the source code and couldn't find where it was being set. This makes a lot
more sense now.

Yep. Dev is all one machine and production (being built as we speak) will be
a more traditional deployment.

Trying the job again now and will let you know what kind of speed
improvement I get.

Thanks,

Danny


On Tue, Apr 29, 2008 at 2:03 PM, stack <stack@duboce.net> wrote:

> Daniel Leffel wrote:
>
>> The Reduce phase is a simple reduce using the TableReduceOutputFormat.
>>
>>
> Try upping the number of reducers -- mapred.reduce.tasks in your
> hadoop-site.xml.  Default is 1.  Try with 4/8/16?
>
> Is your hbase cluster all running on the same node too?
>
>  In this configuration, it writes 50-100 or so inserts per second (which
>> doesn't strike me as terrible), but given the low load factor of 0.20 on
>> the
>> machine, I don't understand what's keeping it from performing better.
>>
>>
>
> Yeah... doesn't java just idling consume a load of 0.2 (smile).  Is this a
> multicore box?
>
>
> St.Ack
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message