hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Nauroth (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-4428) FsDatasetImpl should disclose what the error is when a rename fails
Date Sat, 26 Jan 2013 08:19:25 GMT

    [ https://issues.apache.org/jira/browse/HDFS-4428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13563401#comment-13563401

Chris Nauroth commented on HDFS-4428:

Hi Colin,

This looks like it will be very helpful.  I applied the patch locally on an Ubuntu VM and
ran {{TestNativeIO}} successfully.  Here are a few questions:

1. It appears that the rename function can also fail with ENOTEMPTY.  Can we add that to the
errno mappings?

2. In {{NativeIO.c}}:

if (rename(src, dst) < 0) {
  throw_ioe(env, errno);

I've seen references for rename that say "On failure, a nonzero value is returned" rather
than specifically guaranteeing a negative number gets returned.  Maybe it's safest to change
this to check for != 0?

3. In {{TestNativeIO}}:

// Test attempting to rename a nonexistent file.
try {
  NativeIO.renameTo(nonExistentFile, targetFile);
} catch (NativeIOException e) {
  Assume.assumeThat(e.getErrno(), equalTo(Errno.ENOENT));

I think all of the assume calls were supposed to be assert calls.  With assume, if the condition
is not true, then JUnit marks the test skipped instead of failed, and it does not cause a
build failure.  For example, in the above test, I hacked my local copy to look for the wrong
errno: EIO instead of ENOENT.  JUnit reported a skipped test, but not a failure.

Also, the test would pass even if the {{NativeIOException}} was not thrown.  We'll need to
add fail calls in the try blocks after the rename calls.

4. Also in {{TestNativeIO}}:

File sourceFile = new File(TEST_DIR, "source");
FileOutputStream fos = new FileOutputStream(sourceFile);

It appears that the contents of the files aren't important for the test, so this could simplify
to {{File#createNewFile}} instead of a {{FileOutputStream}}.

5. In {{FsDatasetImpl}}:

try {
  NativeIO.renameTo(blkfile, newBlkFile);
} catch (IOException e) {
  if (!newmeta.renameTo(oldmeta)) {  // restore the meta file
    LOG.warn("Cannot move meta file " + newmeta + 
        "back to the finalized directory " + oldmeta);
  throw new IOException("Block " + replicaInfo + " reopen failed. " +
                          " Unable to move block file " + blkfile +
                          " to rbw dir " + newBlkFile);
                          " to rbw dir " + newBlkFile, e);

Do you want to use the native call for restoring the meta file too?  Then, we could log the
exception and see the errno from that call.

> FsDatasetImpl should disclose what the error is when a rename fails
> -------------------------------------------------------------------
>                 Key: HDFS-4428
>                 URL: https://issues.apache.org/jira/browse/HDFS-4428
>             Project: Hadoop HDFS
>          Issue Type: Bug
>    Affects Versions: 2.0.3-alpha
>            Reporter: Colin Patrick McCabe
>            Assignee: Colin Patrick McCabe
>            Priority: Minor
>         Attachments: HDFS-4428.001.patch
> It would be nice if {{FsDatasetImpl}} would print out an error message when a rename
fails, describing what went wrong.  This would make it a lot easier to investigate and resolve
test failures like HDFS-4051. 

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message