hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Karthik Kambatla (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-2179) Initial cache manager structure and context
Date Wed, 17 Sep 2014 18:44:34 GMT

    [ https://issues.apache.org/jira/browse/YARN-2179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14137725#comment-14137725
] 

Karthik Kambatla commented on YARN-2179:
----------------------------------------

Review comments:
# Rename yarn.sharedcache.nested.level to yarn.sharedcache.nested-level? 
# Rename AppChecker#appIsActive to either isActive(AppId) or isAppActive.
# Nit (okay with not changing): Should AppChecker methods thrown YarnException instead of
IOException, since they are strictly used within SCM code?  
# CacheStructureUtil: remove empty line in class javadoc
# sharedcache-pom: my understand of maven is pretty sparse, so please correct me if I am wrong.
Looks like sharedcache depends on the RM. If we were to embed the sharedcache in the RM, wouldn't
that lead to circular dependency? How do we plan to solve it? 
# RemoteAppChecker: Just thinking out loud - in a non-embedded case, what happens if we upgrade
other daemons/clients but not the SCM and add a new completed state? There might not be a
solution here though, the worst case appears to be that we wouldn't clear the cache when apps
end up in that state. One alternative is to query the RM for active states or an app being
active. I am open to adding these APIs (Private for now) to the RM.
{code}
  private static final EnumSet<YarnApplicationState> ACTIVE_STATES =
      EnumSet.complementOf(EnumSet.of(YarnApplicationState.FINISHED,
        YarnApplicationState.FAILED,
        YarnApplicationState.KILLED));
{code}
# RemoteAppChecker#create should use ClientRMProxy instead of YarnRPC for it to work in an
HA-RM setting. 
# As per offline discussions, we don't need the SCMContext outside of the store implementations.
Can we move it out?

> Initial cache manager structure and context
> -------------------------------------------
>
>                 Key: YARN-2179
>                 URL: https://issues.apache.org/jira/browse/YARN-2179
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Chris Trezzo
>            Assignee: Chris Trezzo
>         Attachments: YARN-2179-trunk-v1.patch, YARN-2179-trunk-v2.patch, YARN-2179-trunk-v3.patch,
YARN-2179-trunk-v4.patch, YARN-2179-trunk-v5.patch
>
>
> Implement the initial shared cache manager structure and context. The SCMContext will
be used by a number of manager services (i.e. the backing store and the cleaner service).
The AppChecker is used to gather the currently running applications on SCM startup (necessary
for an scm that is backed by an in-memory store).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message