hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Brandon Li (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-5944) LeaseManager:findLeaseWithPrefixPath didn't handle path like /a/b/ right cause SecondaryNameNode failed do checkpoint
Date Thu, 13 Feb 2014 21:16:21 GMT

    [ https://issues.apache.org/jira/browse/HDFS-5944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13900739#comment-13900739

Brandon Li commented on HDFS-5944:

In branch-1, if the directory is deleted by a FileSystem object (e.g., fs.delete(...)), the
ending "/" will be trimmed before the delete request gets to NN. However, as in the uploaded
unit test, using DFSClient can send ending "/" to NN to trigger the error.

That is, as long as the HDFS users(e.g., Mapreduce) use FileSystem object, they should not
encounter this problem. Comments?

> LeaseManager:findLeaseWithPrefixPath didn't handle path like /a/b/ right cause SecondaryNameNode
failed do checkpoint
> ---------------------------------------------------------------------------------------------------------------------
>                 Key: HDFS-5944
>                 URL: https://issues.apache.org/jira/browse/HDFS-5944
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: namenode
>    Affects Versions: 1.2.0, 2.2.0
>            Reporter: zhaoyunjiong
>            Assignee: zhaoyunjiong
>         Attachments: HDFS-5944-branch-1.2.patch, HDFS-5944.patch, HDFS-5944.test.txt
> In our cluster, we encountered error like this:
> java.io.IOException: saveLeases found path /XXX/20140206/04_30/_SUCCESS.slc.log but is
not under construction.
> 	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.saveFilesUnderConstruction(FSNamesystem.java:6217)
> 	at org.apache.hadoop.hdfs.server.namenode.FSImageFormat$Saver.save(FSImageFormat.java:607)
> 	at org.apache.hadoop.hdfs.server.namenode.FSImage.saveCurrent(FSImage.java:1004)
> 	at org.apache.hadoop.hdfs.server.namenode.FSImage.saveNamespace(FSImage.java:949)
> What happened:
> Client A open file /XXX/20140206/04_30/_SUCCESS.slc.log for write.
> And Client A continue refresh it's lease.
> Client B deleted /XXX/20140206/04_30/
> Client C open file /XXX/20140206/04_30/_SUCCESS.slc.log for write
> Client C closed the file /XXX/20140206/04_30/_SUCCESS.slc.log
> Then secondaryNameNode try to do checkpoint and failed due to failed to delete lease
hold by Client A when Client B deleted /XXX/20140206/04_30/.
> The reason is a bug in findLeaseWithPrefixPath:
>      int srclen = prefix.length();
>      if (p.length() == srclen || p.charAt(srclen) == Path.SEPARATOR_CHAR) {
>         entries.put(entry.getKey(), entry.getValue());
>       }
> Here when prefix is /XXX/20140206/04_30/, and p is /XXX/20140206/04_30/_SUCCESS.slc.log,
p.charAt(srcllen) is '_'.
> The fix is simple, I'll upload patch later.

This message was sent by Atlassian JIRA

View raw message