hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eli Collins (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-3626) Creating file with invalid path can corrupt edit log
Date Wed, 18 Jul 2012 22:03:34 GMT

    [ https://issues.apache.org/jira/browse/HDFS-3626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13417739#comment-13417739
] 

Eli Collins commented on HDFS-3626:
-----------------------------------

bq. The change here is that Path(URI) did not normalize prior to this patch, whereas Path(String)
does normalize as you've demonstrated.

Right, that's the bug here. The other Path constructors, not just Path(String) but Path(Path,
Path) and friends too, normalize the path on creation.  Ie I don't think the intent was for
eg this usage in the code
{code}  
new Path(new Path(uri), new Path("/"));
{code}
to normalize the path but new Path(uri) not to.
                
> Creating file with invalid path can corrupt edit log
> ----------------------------------------------------
>
>                 Key: HDFS-3626
>                 URL: https://issues.apache.org/jira/browse/HDFS-3626
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: name-node
>    Affects Versions: 2.0.0-alpha
>            Reporter: Todd Lipcon
>            Assignee: Todd Lipcon
>            Priority: Blocker
>         Attachments: hdfs-3626.txt, hdfs-3626.txt
>
>
> Joris Bontje reports the following:
> The following command results in a corrupt NN editlog (note the double slash and reading
from stdin):
> $ cat /usr/share/dict/words | hadoop fs -put - hdfs://localhost:8020//path/file
> After this, restarting the namenode will result into the following fatal exception:
> {code}
> 2012-07-10 06:29:19,910 INFO org.apache.hadoop.hdfs.server.namenode.FSImage: Reading
/var/lib/hadoop-hdfs/cache/hdfs/dfs/name/current/edits_0000000000000000173-0000000000000000188
expecting start txid #173
> 2012-07-10 06:29:19,912 ERROR org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader:
Encountered exception on operation MkdirOp [length=0, path=/, timestamp=1341915658216, permissions=cloudera:supergroup:rwxr-xr-x,
opCode=OP_MKDIR, txid=182]
> java.lang.ArrayIndexOutOfBoundsException: -1
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message