chukwa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jerome Boulon (JIRA)" <j...@apache.org>
Subject [jira] Commented: (CHUKWA-330) controller should not create duplicates for adaptors with offsets
Date Wed, 24 Jun 2009 18:32:07 GMT

    [ https://issues.apache.org/jira/browse/CHUKWA-330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12723672#action_12723672
] 

Jerome Boulon commented on CHUKWA-330:
--------------------------------------

For my proposal:

1- if the client provide a name, then this name overwrite all the AdaptorManager logic
2- If there's no name then the AdaptorManager rely on the underlying adaptor to provide one.

3- >> uniqueness is a problem for whoever is issuing the "add" command
I still think that if the one that is doing the add is automatically able to figure out a
unique name in order to avoid duplicate in a distributed environment without talking to anyone
else, then the that logic can be pushed down the adaptorManager, who is the only one to have
the global vision. 

4- >> isDuplicate() need to be called for each existing adaptor, every time we create
one
No. the AdaptorManager, receive a Add command then call a isDuplicate's method to the factory,

Here the workfow:

4.1- At the factory level, if there's no adaptor already registered for this AdaptorType then
there's no duplicate
4.1.1- if it's not a duplicate The AdaptorManger call the factory to create the adaptor
4.1.1.1- The factory create the new adaptor and register that instance as the instance that
will provide all answers for the isDuplicate call for that specific AdaptorType (hash.put(AdaptorType,
adaptor) )

4.1.2- if there's already one adaptor registered for the AdaptorType then the factory return
hash.get(AdaptorType).isDuplicate(...)
4.1.2.1- At the AdaptorManager level if it's a duplicate then it returns the existing adaptor
based on the name, otherwise call the factory

 
So there's no iteration over any adaptors, just a get from a hash based on AdaptorType + 1
call to isDuplicate(..) and therefore there's no performance issue


> controller should not create duplicates for adaptors with offsets
> -----------------------------------------------------------------
>
>                 Key: CHUKWA-330
>                 URL: https://issues.apache.org/jira/browse/CHUKWA-330
>             Project: Hadoop Chukwa
>          Issue Type: Bug
>          Components: input tools
>            Reporter: Ari Rabkin
>            Assignee: Ari Rabkin
>            Priority: Blocker
>             Fix For: 0.2.0, 0.3.0
>
>         Attachments: CHUKWA-330.patch
>
>   Original Estimate: 1.5h
>  Remaining Estimate: 1.5h
>
> The offset-of-first-byte param to FileTailingAdaptor is hashed into the default adaptor
name.  This changes the semantics of adaptor uniqueness, and broke metrics and the log4jappender.
> Proposed fix:
>   Add a new "addByName()" method to ChukwaAgentController that lets callers specify names.
 
>   Use this new method whenever finer control of adaptor uniqueness is needed.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message