hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Robert Kanter (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (YARN-2131) Add a way to format the RMStateStore
Date Thu, 10 Jul 2014 21:13:06 GMT

     [ https://issues.apache.org/jira/browse/YARN-2131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Robert Kanter updated YARN-2131:

    Attachment: YARN-2131_addendum.patch

The addendum patch renames the command.  

However, I was looking into making the ZK change, and I'm not sure it makes sense to do that.
 To build up the list of delete Ops, we need to get all of the children, and there's no "get
_all_ children" call; so we have to recursively do this ourselves.  And we can't use another
list of Ops for this because it's a "discovery" operation.  That is, if the structure looks
like this:
- A
  | - B
      | - C
given that we start off only knowing A, we can't know that C exists until we know that B exists;
and these each require a call to ZK.  
Because we already have to recursively call ZK to discover the nodes to delete, we may as
well delete them at the same time, right?

Also, I agree with Karthik's earlier comment that it would be good to eventually replace all
of the ZooKeeper code with Curator code.  It handles most if not all of the connection stuff,
provides useful convenience methods, and implements a lot of useful recipes (e.g. leader latch,
locks, etc).  We've been using Curator extensively for Oozie HA.

> Add a way to format the RMStateStore
> ------------------------------------
>                 Key: YARN-2131
>                 URL: https://issues.apache.org/jira/browse/YARN-2131
>             Project: Hadoop YARN
>          Issue Type: New Feature
>          Components: resourcemanager
>    Affects Versions: 2.4.0
>            Reporter: Karthik Kambatla
>            Assignee: Robert Kanter
>             Fix For: 2.6.0
>         Attachments: YARN-2131.patch, YARN-2131.patch, YARN-2131_addendum.patch
> There are cases when we don't want to recover past applications, but recover applications
going forward. To do this, one has to clear the store. Today, there is no easy way to do this
and users should understand how each store works.

This message was sent by Atlassian JIRA

View raw message