hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arinto Murdopo (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-128) Resurrect RM Restart
Date Fri, 16 Nov 2012 13:22:16 GMT

    [ https://issues.apache.org/jira/browse/YARN-128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13498800#comment-13498800
] 

Arinto Murdopo commented on YARN-128:
-------------------------------------

1) Yes, I agree with your clarification. It works as what you state when we are using persistent
storage (not MemStore, but ZK, MySQL, file or other persistent storage)
However, when we are using MemStore, the stored object (appState) and app1 are referring to
the same instance since our "store" is memory. To test my argument, we can put breakpoint
in the assert statement that compares the ApplicationSubmissionContext, then use IDE feature
to change any value of appState's properties  i.e resource in ApplicationSubmissionContext.
The corresponding app1 value (in this case is the resource in app1's ApplicationSubmissionContext)
will also be updated to the same value.

3). Yes, it's in Resource in ApplicationSubmissionContext->ContainerLaunchContext. e 
If we are saving the original resource value requested by client, then the assert statement
that compare ApplicationSubmissionContext will not pass. 
Let's say Client request resource of memory with value of 200. We store this in our persistent
storage. After we store, scheduler updates the resource with value of 1024. In this case,
the resource in app1 instance will be 1024, but the resource that stored in our storage is
200. Hence, it will not pass when we compare them using current assert statement. Maybe we
need to keep storing our original resource request in ApplicationSubmissionContext.

Looking forward to your ZK and HDFS state store. The state store in our project is using MySQL
cluster.  
                
> Resurrect RM Restart 
> ---------------------
>
>                 Key: YARN-128
>                 URL: https://issues.apache.org/jira/browse/YARN-128
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: resourcemanager
>    Affects Versions: 2.0.0-alpha
>            Reporter: Arun C Murthy
>            Assignee: Bikas Saha
>         Attachments: MR-4343.1.patch, RM-recovery-initial-thoughts.txt, RMRestartPhase1.pdf,
YARN-128.full-code.3.patch, YARN-128.full-code-4.patch, YARN-128.new-code-added.3.patch, YARN-128.new-code-added-4.patch,
YARN-128.old-code-removed.3.patch, YARN-128.old-code-removed.4.patch, YARN-128.patch
>
>
> We should resurrect 'RM Restart' which we disabled sometime during the RM refactor.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message