hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Robert Kanter (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-2131) Add a way to format the RMStateStore
Date Fri, 11 Jul 2014 20:35:07 GMT

    [ https://issues.apache.org/jira/browse/YARN-2131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14059320#comment-14059320

Robert Kanter commented on YARN-2131:

Are you sure that the NameNode uses a lock file to do this?  I took a look at NameNode.java,
and from what I can tell, it doesn't actually do anything to prevent you from formatting a
running NameNode (there's a config property it checks where you can disable formatting, but
that's not really the same thing).  Just to make sure I wasn't missing something, I also just
tried it on a pseudo-cluster and it didn't stop me.  

I agree it would be nice to do that; I'm just not sure of a good way.  I don't think a lock
file will work because the command could be run from any node where the RM is installed, not
necessarily the currently running or active one; in which case, a lock file isn't accessible
unless it's in HDFS (and I don't think we want an HDFS dependency for this).  I suppose we
could use a lock on whatever is backing the store (e.g. Znode for ZKRMStateStore, file for
FSRMStateStore, etc), but that gets tricky: what happens if the RM crashes without releasing
the lock?  Then it's stuck.  Doing that would also mean we can't simply wipe out the entire
store space because we need to keep the lock there.  I'll keep thinking about it, but I'm
open to suggestions if you have any other ideas.  Unless you think we should make this another

> Add a way to format the RMStateStore
> ------------------------------------
>                 Key: YARN-2131
>                 URL: https://issues.apache.org/jira/browse/YARN-2131
>             Project: Hadoop YARN
>          Issue Type: New Feature
>          Components: resourcemanager
>    Affects Versions: 2.4.0
>            Reporter: Karthik Kambatla
>            Assignee: Robert Kanter
>             Fix For: 2.6.0
>         Attachments: YARN-2131.patch, YARN-2131.patch, YARN-2131_addendum.patch
> There are cases when we don't want to recover past applications, but recover applications
going forward. To do this, one has to clear the store. Today, there is no easy way to do this
and users should understand how each store works.

This message was sent by Atlassian JIRA

View raw message