chukwa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eric Yang <>
Subject Re: Chukwa on Hbase
Date Mon, 01 Jun 2009 07:53:53 GMT
Hi Vijay,

Chukwa+Hbase is definitely in the realm of possibility.  I had been playing
with Hbase as one of the candidate for replacing mysql in Chukwa.  An
abstract class with load(fs,directory) method should be a good start point
for processDemuxOutput to load data to multiple pipelines (Hbase, mysql, or
pig).  It's fairly trivial to clean up the dataSource hashmap.  DataSource
hashmap should be moved into the DataLoader to decide what to do with the
sequence file(s).

Don't worry, if you don't plan to write the driver for HICC.  I am actually
interested to write a reference implementation to read data from Hbase, and
render on HICC for the fun of it.


On 5/31/09 9:13 PM, "Vijay" <> wrote:

> Hi,
> i would like to contribute hbase adapter for chukwa.... i am writing it
> extending the classes of postprocess as they are written for Mysql can we
> just separate it to make it and both Mysql and Hbase extend the abstract
> class or an interface? i will submit the hbase code once i have a working
> set of it.... i am not sure about hicc as we are planning to use our own
> display with flex, and already have a custom parser working.
> Regards,
> </VJ>

View raw message