hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Li Lu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-3292) [Umbrella] Tests/documentation and/or tools for YARN rolling upgrades backwards/forward compatibility verification
Date Wed, 04 Mar 2015 22:39:38 GMT

    [ https://issues.apache.org/jira/browse/YARN-3292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14347710#comment-14347710

Li Lu commented on YARN-3292:

So far as we can see, YARN requires the following components to be compatible in a rolling
upgrade (please feel free to add more in the discussion):

- Protocols: both public protocols and private wiring protocols
- RM/NM/ATS state stores: RM/NM/ATS data version numbers and the data store/read schema for
each state store.
- APIs 
- Security tokens
- Configurations

We may want to provide a suite of tools and/or unit tests that can verify if an incoming YARN
patch will break the compatibility to the previous version. In the very first stage, we may
want to finish the following tasks:

# Implement a protobuf compatibility checker to check if a patch breaks the compatibility
with existing client and internal protocols
# Extend the protobuf compatibility checker in step 1 to check RM state store
# Look into the possibility to further extend the protobuf checker to NM/ATS(v1) state store
(I’m not very sure now, we can merge this with step 2 if a simple extension is possible).

# Implement a diff-based java API compatibility checker
# Wire up the implemented tools to Jenkins test runs
# Finish formal write ups for the YARN rolling upgrade standard

Please feel free to discuss more about our first step goal. Thanks! 

> [Umbrella] Tests/documentation and/or tools for YARN rolling upgrades backwards/forward
compatibility verification
> ------------------------------------------------------------------------------------------------------------------
>                 Key: YARN-3292
>                 URL: https://issues.apache.org/jira/browse/YARN-3292
>             Project: Hadoop YARN
>          Issue Type: Improvement
>            Reporter: Li Lu
>            Assignee: Li Lu
>              Labels: compatibility, rolling_upgrade, test, tools
> YARN-666 added the support to YARN rolling upgrade. In order to support this feature,
we made changes from many perspectives. There were many assumptions made together with these
existing changes. Future code changes may break these assumptions by accident, and hence break
the YARN rolling upgrades feature. 
> To simplify YARN RU regression tests, maybe we would like to create a set of tools/tests
that can verify YARN RU backward compatibility. 
> On the very first step, we may want to have a compatibility checker for important protocols
and APIs. We may also want to incorporate these tools into our test Jenkins runs, if necessary.

This message was sent by Atlassian JIRA

View raw message