hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Junping Du (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-2045) Data persisted in NM should be versioned
Date Sat, 12 Jul 2014 02:36:04 GMT

    [ https://issues.apache.org/jira/browse/YARN-2045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14059602#comment-14059602

Junping Du commented on YARN-2045:

Thanks both for your comments! 
[~vvasudev], I will fix the comments and consolidate methods here as your suggestions.
[~jlowe], I think the version here stands for the layout of NMState and Proto object get stored
within NMState. The scenario is still rolling upgrade - we should make sure if upgraded NM
can work with old NM store. If any incompatible changes happen (not matter layout change,
Proto changes) in future, the version check here can detect and stop the further wrong behavior.
RMState already has the similar mechanism, I think we should keep consistent in NMstate as
well as other components, i.e. ShuffleHandler and TimelineStore, etc. If you think previous
string-based schema version means something else - like other fundamental changes such as
store type. We can keep that version there for use in future. Thoughts?
I agree that PBImpl here is not too much use, but just make interface looks more clear, i.e.
I can call some handy method like: isCompatibleTo() rather than manipulate proto object directly.
There are also some other similar practices, like RMStateVersion. Can we have it here also?

> Data persisted in NM should be versioned
> ----------------------------------------
>                 Key: YARN-2045
>                 URL: https://issues.apache.org/jira/browse/YARN-2045
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: nodemanager
>    Affects Versions: 2.4.1
>            Reporter: Junping Du
>            Assignee: Junping Du
>         Attachments: YARN-2045-v2.patch, YARN-2045.patch
> As a split task from YARN-667, we want to add version info to NM related data, include:
> - NodeManager local LevelDB state
> - NodeManager directory structure

This message was sent by Atlassian JIRA

View raw message