hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Naganarasimha G R (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-5547) NMLeveldbStateStore should be more tolerant of unknown keys
Date Mon, 31 Oct 2016 00:39:58 GMT

    [ https://issues.apache.org/jira/browse/YARN-5547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15620860#comment-15620860

Naganarasimha G R commented on YARN-5547:

Hi [~ajithshetty],
Patch does not seem to apply now, please rebase the patch and *submit the path* for jenkins
to kick in.
Overall approach seems to be fine, except for these few nits
#  ContainerManagerImpl, ln 377: ??"Container killed after recovery."??, may be better message
like {{"Due to invalid StateStore info container was killed during recovery""}}
# ContainerManagerImpl, ln 386 : new line not req.
# NMLeveldbStateStoreService, ln 268 : requires logging(preferably warn) that *the container
<containerId> will be killed because of the unknown key <key> during recovery*.
# NMLeveldbStateStoreService, ln 1231 : seems to be already handled by a other patch, while
re-base please ensure compilation is successful.
# TestNMLeveldbStateStoreService, ln 986 : has compilation error, requires re-base.
# TestNMLeveldbStateStoreService, ln 939 : Need to handle positive scenario too when the app
can be recovered successfully

> NMLeveldbStateStore should be more tolerant of unknown keys
> -----------------------------------------------------------
>                 Key: YARN-5547
>                 URL: https://issues.apache.org/jira/browse/YARN-5547
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: nodemanager
>    Affects Versions: 2.6.0
>            Reporter: Jason Lowe
>            Assignee: Ajith S
>         Attachments: YARN-5547.01.patch, YARN-5547.02.patch
> Whenever new keys are added to the NM state store it will break rolling downgrades because
the code will throw if it encounters an unrecognized key.  If instead it skipped unrecognized
keys it could be simpler to continue supporting rolling downgrades.  We need to define the
semantics of unrecognized keys when containers and apps are cleaned up, e.g.: we may want
to delete all keys underneath an app or container directory when it is being removed from
the state store to prevent leaking unrecognized keys.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message