chukwa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jerome Boulon (JIRA)" <>
Subject [jira] Commented: (CHUKWA-330) controller should not create duplicates for adaptors with offsets
Date Wed, 24 Jun 2009 18:32:07 GMT


Jerome Boulon commented on CHUKWA-330:

For my proposal:

1- if the client provide a name, then this name overwrite all the AdaptorManager logic
2- If there's no name then the AdaptorManager rely on the underlying adaptor to provide one.

3- >> uniqueness is a problem for whoever is issuing the "add" command
I still think that if the one that is doing the add is automatically able to figure out a
unique name in order to avoid duplicate in a distributed environment without talking to anyone
else, then the that logic can be pushed down the adaptorManager, who is the only one to have
the global vision. 

4- >> isDuplicate() need to be called for each existing adaptor, every time we create
No. the AdaptorManager, receive a Add command then call a isDuplicate's method to the factory,

Here the workfow:

4.1- At the factory level, if there's no adaptor already registered for this AdaptorType then
there's no duplicate
4.1.1- if it's not a duplicate The AdaptorManger call the factory to create the adaptor The factory create the new adaptor and register that instance as the instance that
will provide all answers for the isDuplicate call for that specific AdaptorType (hash.put(AdaptorType,
adaptor) )

4.1.2- if there's already one adaptor registered for the AdaptorType then the factory return
hash.get(AdaptorType).isDuplicate(...) At the AdaptorManager level if it's a duplicate then it returns the existing adaptor
based on the name, otherwise call the factory

So there's no iteration over any adaptors, just a get from a hash based on AdaptorType + 1
call to isDuplicate(..) and therefore there's no performance issue

> controller should not create duplicates for adaptors with offsets
> -----------------------------------------------------------------
>                 Key: CHUKWA-330
>                 URL:
>             Project: Hadoop Chukwa
>          Issue Type: Bug
>          Components: input tools
>            Reporter: Ari Rabkin
>            Assignee: Ari Rabkin
>            Priority: Blocker
>             Fix For: 0.2.0, 0.3.0
>         Attachments: CHUKWA-330.patch
>   Original Estimate: 1.5h
>  Remaining Estimate: 1.5h
> The offset-of-first-byte param to FileTailingAdaptor is hashed into the default adaptor
name.  This changes the semantics of adaptor uniqueness, and broke metrics and the log4jappender.
> Proposed fix:
>   Add a new "addByName()" method to ChukwaAgentController that lets callers specify names.
>   Use this new method whenever finer control of adaptor uniqueness is needed.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message