chukwa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eric Yang <>
Subject Re: Simplify chukwa agent
Date Mon, 18 Oct 2010 06:14:47 GMT
Hi Ari,

The list of TODO from my side for Chukwa 0.5 is:

- Update demux parsers (New parser for parsing hadoop metrics, job
history 0.20+ log, and hadoop logs)
- Update data analytics scripts using pig 0.8
- Update documentation

I am waiting on pig to release 0.8 which contains support for using
HBase as storage.  This will enable chukwa data analytics on top of
After pig 0.8 is released, then Chukwa 0.5 release will be ready.  I
agree that refactoring the interface in Chukwa Agent could wait until
Chukwa 0.5 is released.


On Sun, Oct 17, 2010 at 10:54 PM, Ariel Rabkin <> wrote:
> Howdy.
> I agree that that part of the code is pretty snarly. Not sure how
> urgent a priority it is to fix.
> I think a better short-term goal might be doing a 0.5 release. How
> close are we to that?
> --Ari
> On Sun, Oct 17, 2010 at 3:31 PM, Eric Yang <> wrote:
>> Hi all,
>> Chukwa agent code is not intuitive to understand.  This is mainly the
>> interface in Chukwa agent is over complicating the implementation.  I
>> don't see a rationale for having each class to be an interface.
>> Connector and Chukwa Sender are two interfaces which are not very
>> useful to be interface.  It creates over complicated subsystem to
>> maintain collector list in agent, connector and sender.  Ideally,
>> there should be a single place for configuration source of truth.  I
>> am leaning toward making those interface abstract classes.  Connector
>> should be rename to something more meaningful like multiplexer or MUX
>> for short.  AgentControlSocketListener could be refactor into a jersey
>> like rest api, for easier to maintain the code base, and remove the
>> 9093 protocol.
>> This sounds like a major task.  I like to gather feedback to see if
>> this change is necessary for creating more traction on chukwa
>> development.
>> regards,
>> Eric
> --
> Ari Rabkin
> UC Berkeley Computer Science Department

View raw message