chukwa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From James Seigel <ja...@tynt.com>
Subject Re: [DISCUSSION] Making HBaseWriter default
Date Mon, 22 Nov 2010 19:16:35 GMT
+1

On 2010-11-22, at 11:22 AM, Bill Graham wrote:

> Hi Eric,
> 
> I think we should have a default config that is easy to tweak to work
> with or without HBase. My inclination would be to not have HBase
> enabled by default, since it raises the barrier to entry for a basic
> set-up that might not otherwise need HBase.
> 
> When I first installed Chukwa 0.3.0 for evaluation I spent a lot of
> time setting up MySQL and HICC because I thought I had to, only to
> realize later that those components weren't needed for my use cases
> (this wasn't and still isn't clearly reflected in the quick start
> documentation). Hence I think it's better to require a few extra steps
> for people who have HBase, than to risk losing users to the extra
> steps required to get a basic setup running without HBase.
> 
> thanks,
> Bill
> 
> 
> 
> On Sat, Nov 20, 2010 at 9:02 PM, Eric Yang <eric818@gmail.com> wrote:
>> Hi James,
>> 
>> In my 10 nodes cluster, it used to take 7 minutes (3 minutes M/R + 4
>> minutes load to mysql) to process data and being able to visualize on
>> HICC UI.  Now, it takes 50 milliseconds.  For data aggregation, it
>> used to take 15-20 minutes to roll up data for 2000 nodes data daily,
>> now it takes <5 minutes.  The improvement is 2100 times better for
>> data load latency, and 3 times better for data analytics throughput
>> with pig+hbase.
>> 
>> regards,
>> Eric
>> 
>> On Sat, Nov 20, 2010 at 12:20 PM, James Seigel <james@tynt.com> wrote:
>>> Hello!
>>> 
>>> As a high volume user, I was just wondering how the HbaseWriter compares with
the current one under load?  Better or worse and by how much?
>>> 
>>> Cheers
>>> James.
>>> 
>>> 
>>> On 2010-11-20, at 1:15 PM, Eric Yang wrote:
>>> 
>>>> Hi all,
>>>> 
>>>> In order to use full features of Chukwa in trunk, HBase is required to
>>>> display data on HICC.  I am wondering if anyone has good success in
>>>> using HBase+HICC?  I am leaning toward making hbase the default data
>>>> storage for chukwa, and the default configuration for chukwa collector
>>>> will make use of HBaseWriter.  What do the community feel about
>>>> changing the default writer config?
>>>> 
>>>> regards,
>>>> Eric
>>> 
>>> 
>> 


Mime
View raw message