hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Abhishek Rai (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDFS-6431) File in snapshot tree is not readable due to null blocks
Date Mon, 19 May 2014 16:25:39 GMT
Abhishek Rai created HDFS-6431:

             Summary: File in snapshot tree is not readable due to null blocks
                 Key: HDFS-6431
                 URL: https://issues.apache.org/jira/browse/HDFS-6431
             Project: Hadoop HDFS
          Issue Type: Bug
          Components: namenode, snapshots
    Affects Versions: 2.2.0
         Environment: 2-node HDFS setup with hadoop-2.2.0.
Namenode on node A and datanode on both nodes, A and B.
            Reporter: Abhishek Rai

1. Took a snapshot of HDFS directory:

2. Snapshot was successful.

3. Initiated copy of the snapshotted directory
hadoop-2.2.0/bin/hdfs dfs -fs -get /orion/dogfood/.snapshot/20140517 ~/tmp/copy

4. Normally, this succeeds.  We have copied numerous (other) snapshots of this directory in
the past.  But copying this particular snapshot gives the following error:

get: Cannot open filename /orion/dogfood/.snapshot/20140517/cache/log-dogfood-20140513.111409-466b34cd-a488-4bd9-b1f2-a9f280e647c2

Above "hdfs -get" command fails with exit status 1.  Interestingly, this file was skipped
and all other good files were copied.

5. The file has zero length --
$ hadoop-2.2.0/bin/hdfs -ls /orion/dogfood/.snapshot/20140517/cache/log-dogfood-20140514.133924-132c3165-55c3-4908-ac19-31c61922205f

-rw-r--r--   3 admin admin          0 2014-05-14 13:39 /orion/dogfood/.snapshot/20140517/cache/log-dogfood-20140514.133924-132c3165-55c3-4908-ac19-31c61922205f

6. Trying to copy this file individually also gives same error --
$ hadoop-2.2.0/bin/hdfs -get /orion/dogfood/.snapshot/20140517/cache/log-dogfood-20140514.133924-132c3165-55c3-4908-ac19-31c61922205f

get: Cannot open filename /orion/dogfood/.snapshot/20140517/cache/log-dogfood-20140514.133924-132c3165-55c3-4908-ac19-31c61922205f

This seems like a bug to me.  From looking at HDFS code, I made the following observations:

o DFSInputStream is receiving null "LocatedBlocks" for the source file:
final LocatedBlocks newInfo = dfsClient.getLocatedBlocks(src, 0);

o This would happen when server returns GetLocatedBlocksResponse with "locations" not set,
which can happen only if INodeFile.blocks is null for the file being opened.

So, it appears that for this file, blocks is null in this snapshot.

This message was sent by Atlassian JIRA

View raw message