hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Craig Welch (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-3165) Possible inconsistent queue state when queue reinitialization failed
Date Fri, 01 May 2015 20:01:06 GMT

    [ https://issues.apache.org/jira/browse/YARN-3165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14523807#comment-14523807
] 

Craig Welch commented on YARN-3165:
-----------------------------------

So, given that the old queue actually takes on the new values & thereby is "changed" during
the process, how would we do a rollback?  Is an option to instead have a "two stage commit"
where the validation occurs in one pass and then the taking on of the changes occurs in a
second pass?

> Possible inconsistent queue state when queue reinitialization failed
> --------------------------------------------------------------------
>
>                 Key: YARN-3165
>                 URL: https://issues.apache.org/jira/browse/YARN-3165
>             Project: Hadoop YARN
>          Issue Type: Bug
>            Reporter: Jian He
>            Assignee: Jian He
>
> This came up in a discussion with [~chris.douglas]. 
> If queue reinitialization failed in the middle, it is possible that queues are left in
an inconsistent state - some queues are already updated, but some are not.  One example is
below code in leafQueue:
> {code} 
>     if (newMax.getMemory() < oldMax.getMemory()
>         || newMax.getVirtualCores() < oldMax.getVirtualCores()) {
>       throw new IOException(
>           "Trying to reinitialize "
>               + getQueuePath()
>               + " the maximum allocation size can not be decreased!"
>               + " Current setting: " + oldMax
>               + ", trying to set it to: " + newMax);
>     }
> {code}
> If exception is thrown here, the previous queues are already updated, but latter queues
are not.
> So we should make queue reinitialization transactional. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message