hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vinod Kumar Vavilapalli (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-2179) Initial cache manager structure and context
Date Sat, 20 Sep 2014 18:42:34 GMT

    [ https://issues.apache.org/jira/browse/YARN-2179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14142157#comment-14142157
] 

Vinod Kumar Vavilapalli commented on YARN-2179:
-----------------------------------------------

Some comments on the patch
 - Rename config yarn.sharedcache.root to root-path or root-dir?
 - I cannot see why sharedcachemanager depends on resourcemanager module, given I haven't
seen the entire feature related code yet. Ideally
    -- sharedcachemanager simply uses yarn-client
    -- scharedcachemanager is its own module
    -- ResourceManager can embed shared-cache-manager by making it a run-time dependency (and
thus not depend on it at compile time)
 - AppChecker.appIsActive() -> isApplicationActive() and getAllActiveApps() -> getActiveApplications()?
(I tend to favor controlled verbosity :) )
 - RemoteAppChecker won't work in when RM-failover is enabled. You are better off simply using
YarnClient instead of building all of that functionality from scratch again. Similarly, for
getAllActiveApps(), we can just use {{List<ApplicationReport> getApplications(EnumSet<YarnApplicationState>
applicationStates)}} from YarnClient.

bq. Would it make more sense to leverage getFinalApplicationStatus() instead of getYarnApplicationState()?
That way we can just say if the FinalApplicationStatus is undefined don't clean it up, otherwise
we are safe to delete the appId.
FinalApplicationStatus is filled in by user-APIs and some applications may chose to leave
it as UNDEFINED, so we cannot depend on it. I propose that we leave the usage of ApplicationState
and add an API in YarnClient/RM to detect active states in a follow up.

> Initial cache manager structure and context
> -------------------------------------------
>
>                 Key: YARN-2179
>                 URL: https://issues.apache.org/jira/browse/YARN-2179
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Chris Trezzo
>            Assignee: Chris Trezzo
>         Attachments: YARN-2179-trunk-v1.patch, YARN-2179-trunk-v2.patch, YARN-2179-trunk-v3.patch,
YARN-2179-trunk-v4.patch, YARN-2179-trunk-v5.patch
>
>
> Implement the initial shared cache manager structure and context. The SCMContext will
be used by a number of manager services (i.e. the backing store and the cleaner service).
The AppChecker is used to gather the currently running applications on SCM startup (necessary
for an scm that is backed by an in-memory store).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message