hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "James Thomas (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-6800) Determine how Datanode layout changes should interact with rolling upgrade
Date Fri, 22 Aug 2014 17:30:13 GMT

    [ https://issues.apache.org/jira/browse/HDFS-6800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14107133#comment-14107133

James Thomas commented on HDFS-6800:

[~cmccabe], thanks for checking this out. Why couldn't we simply do the delete of the trash
dir before the doRollback() call? If the trash removal succeeds and the doRollback() fails,
it's not a problem, because we don't need the trash at all in the case where a previous directory
is created. After this happens, we may decide to 1) finalize the upgrade instead of rolling
it back, in which case we'll simply delete the previous directory and move forward or 2) make
a second attempt at the rollback, in which case we'll attempt to delete the nonexistent trash
directory (not a problem) and then restore the previous directory in doRollback(). So we're
fine in either case.

[~arpitagarwal], sorry to keep bothering you, but I was hoping to get the work committed by
next week (when my summer internship ends), so it'd be great if you could take a look or direct
me to someone else who might be qualified.

> Determine how Datanode layout changes should interact with rolling upgrade
> --------------------------------------------------------------------------
>                 Key: HDFS-6800
>                 URL: https://issues.apache.org/jira/browse/HDFS-6800
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: datanode
>    Affects Versions: 2.6.0
>            Reporter: Colin Patrick McCabe
>            Assignee: James Thomas
>         Attachments: HDFS-6800.2.patch, HDFS-6800.3.patch, HDFS-6800.4.patch, HDFS-6800.patch
> We need to handle attempts to rolling-upgrade the DataNode to a new storage directory
> One approach is to disallow such upgrades.  If we choose this approach, we should make
sure that the system administrator gets a helpful error message and a clean failure when trying
to use rolling upgrade to a version that doesn't support it.  Based on the compatibility guarantees
described in HDFS-5535, this would mean that *any* future DataNode layout changes would require
a major version upgrade.
> Another approach would be to support rolling upgrade from an old DN storage layout to
a new layout.  This approach requires us to change our documentation to explain to users that
they should supply the {{\-rollback}} command on the command-line when re-starting the DataNodes
during rolling rollback.  Currently the documentation just says to restart the DataNode normally.
> Another issue here is that the DataNode's usage message describes rollback options that
no longer exist.  The help text says that the DN supports {{\-rollingupgrade rollback}}, but
this option was removed by HDFS-6005.

This message was sent by Atlassian JIRA

View raw message