chukwa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ari Rabkin (JIRA)" <>
Subject [jira] Commented: (CHUKWA-330) controller should not create duplicates for adaptors with offsets
Date Wed, 24 Jun 2009 00:17:07 GMT


Ari Rabkin commented on CHUKWA-330:

My view is that adaptor names are a responsibility for *users*, not for Adaptor developers.
In practice, most users will use the controller, which is able to handle this for them. More
precisely, I think uniqueness is a problem for whoever is issuing the "add" command, which
is a handful of places in the Chukwa code, plus advanced users who presumably know what they're

I'm uneasy about an isDuplicate method. The Adaptor interface is already getting uncomfortably
complex and that makes me skittish.   Worse, it's not clear what isDuplicate should do.  What
does it mean, really, to say that one adaptor duplicates another?  It seems like this is pushing
an awful lot of policy into Adaptors, without being able to give adaptor authors any guidance
on what it should be.  

If I understand your proposal correctly, isDuplicate() need to be called for each existing
adaptor, every time we create one.  That might be a performance limitation at scale.

> controller should not create duplicates for adaptors with offsets
> -----------------------------------------------------------------
>                 Key: CHUKWA-330
>                 URL:
>             Project: Hadoop Chukwa
>          Issue Type: Bug
>          Components: input tools
>            Reporter: Ari Rabkin
>            Assignee: Ari Rabkin
>            Priority: Blocker
>   Original Estimate: 1.5h
>  Remaining Estimate: 1.5h
> The offset-of-first-byte param to FileTailingAdaptor is hashed into the default adaptor
name.  This changes the semantics of adaptor uniqueness, and broke metrics and the log4jappender.
> Proposed fix:
>   Add a new "addByName()" method to ChukwaAgentController that lets callers specify names.
>   Use this new method whenever finer control of adaptor uniqueness is needed.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message