hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Junping Du (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-3039) [Aggregator wireup] Implement ATS app-appgregator service discovery
Date Wed, 04 Mar 2015 19:05:38 GMT

    [ https://issues.apache.org/jira/browse/YARN-3039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14347384#comment-14347384
] 

Junping Du commented on YARN-3039:
----------------------------------

Thanks [~Naganarasimha] for comments!
bq. Even if we are not able to have "AMRMClient can be wrapped into TimelineClient" i feel
other suggestion from vinod was right
to add a blocking call in AMRMClient to get aggregator address directly from RM. instead of
observer pattern @ the AM side. thoughts?
I am open for this way. However, more to treat this as an optimization (don't have to wait
heartbeat interval). Within this JIRA scope, I think we should have heartbeat in ApplicationMasterService
as basic mechanism because some applications (like MR) doesn't use AMRMClient for now. We
can have separated JIRA to address this optimization if necessary. BTW, what's your concern
for observer (listener) pattern in AM?

bq. Yes i have gone through 3033, but earlier was trying to mention as our current approach
was with NM AUX service. But anyway what i wanted was some kind of protocol between appAggregators
with either NM or RM. Protocol between NM and appAgregator should suffice all other ways to
launch AppAgregators.
Yes. Agree that not too much differences for aggregator talk to NM or RM. Just as demo patch
shows, I would prefer slightly for NM because it seems RM already host too many RPC services
today.

bq. Well if there is no logic/requirement to aggregate/consolidate all messages (events and
metrics) for an App, then in my opinion it better not to have additional instances of aggregators
and we can keep it similar to old Timeline service.
I am not sure on this but assume this is one part of motivation that we need new TimelineService
(not only for performance reasons)? 

bq. Thanks it will be more clear to implement if we have the proposals documented.
No problem. I will upload a new one when figuring out the demo patch which force me to address
more details.

> [Aggregator wireup] Implement ATS app-appgregator service discovery
> -------------------------------------------------------------------
>
>                 Key: YARN-3039
>                 URL: https://issues.apache.org/jira/browse/YARN-3039
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>            Reporter: Sangjin Lee
>            Assignee: Junping Du
>         Attachments: Service Binding for applicationaggregator of ATS (draft).pdf, YARN-3039-no-test.patch,
YARN-3039-v2-incomplete.patch
>
>
> Per design in YARN-2928, implement ATS writer service discovery. This is essential for
off-node clients to send writes to the right ATS writer. This should also handle the case
of AM failures.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message