chukwa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eric Yang <eric...@gmail.com>
Subject Re: [DISCUSSION] Making HBaseWriter default
Date Sun, 21 Nov 2010 05:02:41 GMT
Hi James,

In my 10 nodes cluster, it used to take 7 minutes (3 minutes M/R + 4
minutes load to mysql) to process data and being able to visualize on
HICC UI.  Now, it takes 50 milliseconds.  For data aggregation, it
used to take 15-20 minutes to roll up data for 2000 nodes data daily,
now it takes <5 minutes.  The improvement is 2100 times better for
data load latency, and 3 times better for data analytics throughput
with pig+hbase.

regards,
Eric

On Sat, Nov 20, 2010 at 12:20 PM, James Seigel <james@tynt.com> wrote:
> Hello!
>
> As a high volume user, I was just wondering how the HbaseWriter compares with the current
one under load?  Better or worse and by how much?
>
> Cheers
> James.
>
>
> On 2010-11-20, at 1:15 PM, Eric Yang wrote:
>
>> Hi all,
>>
>> In order to use full features of Chukwa in trunk, HBase is required to
>> display data on HICC.  I am wondering if anyone has good success in
>> using HBase+HICC?  I am leaning toward making hbase the default data
>> storage for chukwa, and the default configuration for chukwa collector
>> will make use of HBaseWriter.  What do the community feel about
>> changing the default writer config?
>>
>> regards,
>> Eric
>
>

Mime
View raw message