Mailing-List: contact yarn-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Date: Wed, 14 Sep 2016 07:54:20 +0000 (UTC)
From: "Jian He (JIRA)" <jira@apache.org>
To: yarn-issues@hadoop.apache.org
Message-ID: <JIRA.13004343.1473695947000.568985.1473839660942@Atlassian.JIRA>
In-Reply-To: <JIRA.13004343.1473695947000@Atlassian.JIRA>
References: <JIRA.13004343.1473695947000@Atlassian.JIRA> <JIRA.13004343.1473695947859@arcas>
Subject: [jira] [Commented] (YARN-5637) Changes in NodeManager to support
 Container upgrade and rollback/commit
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
archived-at: Wed, 14 Sep 2016 07:54:23 -0000


    [ https://issues.apache.org/jira/browse/YARN-5637?page=3Dcom.atlassian.=
jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=3D15489=
728#comment-15489728 ]=20

Jian He commented on YARN-5637:
-------------------------------

Thanks Arun, some more comments:
- Here, we could make reInitEvent.getResourceSet() be merged with existing =
resourceSet.localizedResource upfront, so that both oldResourceSet and newR=
esourceSet contain full copy of resources, rather than delta. Doing this, t=
he logic of {{container.resourceSet =3D container.reInitContext.mergedResou=
rceSet();}} will not needed. We can simply set it with {{container.resource=
Set =3D reInitContext.newResoureSet}}, similar to what=E2=80=99s being done=
 for {{container.launchContext =3D reInitContext.newLaunchContext}}
{code}
return new ReInitializationContext(reInitEvent.getReInitLaunchContext(),
=C2=A0=C2=A0=C2=A0 reInitEvent.getResourceSet(), container.getLaunchContext=
(),
=C2=A0=C2=A0=C2=A0 container.resourceSet, reInitEvent.getRetryFailureContex=
t(),=20
=C2=A0=C2=A0=C2=A0 reInitEvent.isAutoCommit());

{code}
- nit:  the container.reInitContext!=3D null check is not needed.
{code}
if (container.reInitContext !=3D null=20
=C2=A0=C2=A0=C2=A0 && container.reInitContext.autoCommit) {
{code}

- I found the resourceSet is also not updated when rollback in RetryFailure=
Transition, I also tried some refactoring, may be something like below:
{code}
      ContainerRetryContext retryContext =3D container.containerRetryContex=
t;
      int remainingAttempts =3D container.remainingRetryAttempts;
      if (container.reInitContext !=3D null) {
        retryContext =3D container.reInitContext.retryOnFailueContext;
        remainingAttempts =3D container.reInitContext.retryAttemptsRemainin=
g;
      }

      if (shouldRetry(container.exitCode, retryContext,remainingAttempts)) =
{
        // TODO state-store operation
        doRelaunch(container, container.remainingRetryAttempts,
            container.containerRetryContext.getRetryInterval());
      } else if (container.canRollback()) {
        // rollback
        container.reInitContext =3D new ReInitializationContext(
            container.reInitContext.oldLaunchContext,
            container.reInitContext.oldResourceSet, null, null,
            container.containerRetryContext, true);
        new KilledExternallyForReInitTransition().transition(container, eve=
nt);
      } else {
        // fail
        new ExitedWithFailureTransition(true).transition(container, event);
        return ContainerState.EXITED_WITH_FAILURE;
      }
    }

  public static boolean shouldRetry(int errorCode,
      ContainerRetryContext retryContext, int remainingRetryAttempts) {
    if (retryContext =3D=3D null) {
      return false;
    }
  .....
{code}

- testContainerUpgradeRollbackDueToFailure: comment does not match code
{code}
    // Wait for new processStartfile to be created
    while (!oldStartFile.exists() && timeoutSecs++ < 20) {
{code}

> Changes in NodeManager to support Container upgrade and rollback/commit
> -----------------------------------------------------------------------
>
>                 Key: YARN-5637
>                 URL: https://issues.apache.org/jira/browse/YARN-5637
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Arun Suresh
>            Assignee: Arun Suresh
>         Attachments: YARN-5637.001.patch, YARN-5637.002.patch
>
>
> YARN-5620 added support for re-initialization of Containers using a new l=
aunch Context.
> This JIRA proposes to use the above feature to support upgrade and subseq=
uent rollback or commit of the upgrade.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org