chukwa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ari Rabkin (JIRA)" <>
Subject [jira] Commented: (CHUKWA-330) controller should not create duplicates for adaptors with offsets
Date Thu, 25 Jun 2009 00:04:07 GMT


Ari Rabkin commented on CHUKWA-330:

Yes.  The parseArgs() method returns exactly the part of the argument used for duplicate detection;
this was intended to carry out your suggestion.  I noticed that isDuplicate would basically
have to reproduce all the parsing logic from start(), and it occurred to me that that should
be factored out-- duplicating the code would be a significant maintainance burden.

The point about "optional parts" is a good one. I think this is a documentation-only fix --
instead of talking about optional parts, the comment should be more explicit that the string
returned is used for duplicate detection.
As to not keeping a registry -- object creation in Java is really efficient, as is garbage-collecting
short-lived objects so I think the overhead of creating an object, and throwing it away if
that adaptor already exists, is OK. (Particularly since it avoids a hash lookup) Adding more
shared datastructures is unappealing -- each new datastructure needs a synchronization scheme,
and introduces the posibility of subtle bugs.

> controller should not create duplicates for adaptors with offsets
> -----------------------------------------------------------------
>                 Key: CHUKWA-330
>                 URL:
>             Project: Hadoop Chukwa
>          Issue Type: Bug
>          Components: input tools
>            Reporter: Ari Rabkin
>            Assignee: Ari Rabkin
>            Priority: Blocker
>             Fix For: 0.2.0, 0.3.0
>         Attachments: CHUKWA-330-big.patch, CHUKWA-330.patch
>   Original Estimate: 1.5h
>  Remaining Estimate: 1.5h
> The offset-of-first-byte param to FileTailingAdaptor is hashed into the default adaptor
name.  This changes the semantics of adaptor uniqueness, and broke metrics and the log4jappender.
> Proposed fix:
>   Add a new "addByName()" method to ChukwaAgentController that lets callers specify names.
>   Use this new method whenever finer control of adaptor uniqueness is needed.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message