hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kihwal Lee (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-9426) Rollingupgrade finalization is not backward compatible
Date Fri, 13 Nov 2015 16:28:11 GMT

    [ https://issues.apache.org/jira/browse/HDFS-9426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15004212#comment-15004212
] 

Kihwal Lee commented on HDFS-9426:
----------------------------------

Unfortunately the new datanodes will not interpret null {{rollingUpgradeStatus}} as finalized.

Prior to HDFS-7645,
{code:java}
  private void handleRollingUpgradeStatus(HeartbeatResponse resp) throws IOException {
    RollingUpgradeStatus rollingUpgradeStatus = resp.getRollingUpdateStatus();
    if (rollingUpgradeStatus != null &&
        rollingUpgradeStatus.getBlockPoolId().compareTo(bpos.getBlockPoolId()) != 0) {
      // Can this ever occur?
      LOG.error("Invalid BlockPoolId " +
          rollingUpgradeStatus.getBlockPoolId() +
          " in HeartbeatResponse. Expected " +
          bpos.getBlockPoolId());
    } else {
      bpos.signalRollingUpgrade(rollingUpgradeStatus != null);
    }
  } 
// ==============
  void signalRollingUpgrade(boolean inProgress) throws IOException {
    String bpid = getBlockPoolId();
    if (inProgress) {
      dn.getFSDataset().enableTrash(bpid);
      dn.getFSDataset().setRollingUpgradeMarker(bpid);
    } else {
      dn.getFSDataset().clearTrash(bpid);
      dn.getFSDataset().clearRollingUpgradeMarker(bpid);
    }
  } 
{code}

After HDFS-7645,
{code:java}
  private void handleRollingUpgradeStatus(HeartbeatResponse resp) throws IOException {
    RollingUpgradeStatus rollingUpgradeStatus = resp.getRollingUpdateStatus();
    if (rollingUpgradeStatus != null &&
        rollingUpgradeStatus.getBlockPoolId().compareTo(bpos.getBlockPoolId()) != 0) {
      // Can this ever occur?
      LOG.error("Invalid BlockPoolId " +
          rollingUpgradeStatus.getBlockPoolId() +
          " in HeartbeatResponse. Expected " +
          bpos.getBlockPoolId());
    } else {
      bpos.signalRollingUpgrade(rollingUpgradeStatus);
    }
  }
// ==============
  void signalRollingUpgrade(RollingUpgradeStatus rollingUpgradeStatus)
      throws IOException {
    if (rollingUpgradeStatus == null) {
      return;
    }
    String bpid = getBlockPoolId();
    if (!rollingUpgradeStatus.isFinalized()) {
      dn.getFSDataset().enableTrash(bpid);
      dn.getFSDataset().setRollingUpgradeMarker(bpid);
    } else {
      dn.getFSDataset().clearTrash(bpid);
      dn.getFSDataset().clearRollingUpgradeMarker(bpid);
    }
  }
{code}

> Rollingupgrade finalization is not backward compatible
> ------------------------------------------------------
>
>                 Key: HDFS-9426
>                 URL: https://issues.apache.org/jira/browse/HDFS-9426
>             Project: Hadoop HDFS
>          Issue Type: Bug
>            Reporter: Kihwal Lee
>            Priority: Blocker
>
> After HDFS-7645, the namenode can return non-null {{rollingUpgradeInfo}} in heatbeat
reponses. 2.7.1 or 2.6.x datanodes won't finalize the upgrade because it's not null.
> 2.7.2rc0 datanodes will finalize the upgrade if the namenode is running 2.7.1 or 2.6,
because it also checks for null {{rollingUpgradeInfo}}.
> I believe the quickest fix will be to return null in hearbeat response if rollingupgrade
is not in progress.
> HDFS-8656 recognized the compatibility issue of the changed semantics, but unfortunately
did not address the semantics of the heartbeat response.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message