chukwa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eric Yang <eric...@gmail.com>
Subject Re: Simplify chukwa agent
Date Mon, 18 Oct 2010 06:14:47 GMT
Hi Ari,

The list of TODO from my side for Chukwa 0.5 is:

- Update demux parsers (New parser for parsing hadoop metrics, job
history 0.20+ log, and hadoop logs)
- Update data analytics scripts using pig 0.8
- Update documentation

I am waiting on pig to release 0.8 which contains support for using
HBase as storage.  This will enable chukwa data analytics on top of
pig+HBase.
After pig 0.8 is released, then Chukwa 0.5 release will be ready.  I
agree that refactoring the interface in Chukwa Agent could wait until
Chukwa 0.5 is released.

regards,
Eric

On Sun, Oct 17, 2010 at 10:54 PM, Ariel Rabkin <asrabkin@gmail.com> wrote:
> Howdy.
>
> I agree that that part of the code is pretty snarly. Not sure how
> urgent a priority it is to fix.
>
> I think a better short-term goal might be doing a 0.5 release. How
> close are we to that?
>
> --Ari
>
> On Sun, Oct 17, 2010 at 3:31 PM, Eric Yang <eric818@gmail.com> wrote:
>> Hi all,
>>
>> Chukwa agent code is not intuitive to understand.  This is mainly the
>> interface in Chukwa agent is over complicating the implementation.  I
>> don't see a rationale for having each class to be an interface.
>> Connector and Chukwa Sender are two interfaces which are not very
>> useful to be interface.  It creates over complicated subsystem to
>> maintain collector list in agent, connector and sender.  Ideally,
>> there should be a single place for configuration source of truth.  I
>> am leaning toward making those interface abstract classes.  Connector
>> should be rename to something more meaningful like multiplexer or MUX
>> for short.  AgentControlSocketListener could be refactor into a jersey
>> like rest api, for easier to maintain the code base, and remove the
>> 9093 protocol.
>> This sounds like a major task.  I like to gather feedback to see if
>> this change is necessary for creating more traction on chukwa
>> development.
>>
>> regards,
>> Eric
>>
>
>
>
> --
> Ari Rabkin asrabkin@gmail.com
> UC Berkeley Computer Science Department
>

Mime
View raw message