hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tsz Wo (Nicholas), SZE (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-3179) failed to append data, DataStreamer throw an exception, "nodes.length != original.length + 1" on single datanode cluster
Date Tue, 03 Apr 2012 19:02:25 GMT

    [ https://issues.apache.org/jira/browse/HDFS-3179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13245601#comment-13245601
] 

Tsz Wo (Nicholas), SZE commented on HDFS-3179:
----------------------------------------------

I think the problem is "one datanode with replication 3".  What should be the user expectation?
 It seems that users won't be happy if we do not allow append.  However, if we allow appending
to a single replica and the replica become corrupted, then it is possible to have data loss
- I can imagine in some extreme cases that a user is appending to a single replica slowly,
admin add more datanodes later on but the block won't be replicated since the file is not
closed, and then the datanode with the single replica fails.  Is this case acceptable to you?

> So from the view of user, the first append succeed while the second fail, is that a good
idea?

The distinction is whether there is pre-append data.  There are pre-append data in the replica
in the second append.  The pre-append data was in a closed file and if the datanode fails
during append, it could have data loss.  However, in the first append, there is no pre-append
data.  If the append fails and the new replica is lost, it is a sort of okay since only the
new data is lost.

The add-datanode feature of is to prevent data loss on pre-append data.  Users (or admin)
could turn it off as mentioned in HDFS-3091.  I think we may improve the error message.  Is
it good enough?  Or any suggestion?
                
> failed to append data, DataStreamer throw an exception, "nodes.length != original.length
+ 1" on single datanode cluster
> ------------------------------------------------------------------------------------------------------------------------
>
>                 Key: HDFS-3179
>                 URL: https://issues.apache.org/jira/browse/HDFS-3179
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: data-node
>    Affects Versions: 0.23.2
>            Reporter: Zhanwei.Wang
>            Priority: Critical
>
> Create a single datanode cluster
> disable permissions
> enable webhfds
> start hdfs
> run the test script
> expected result:
> a file named "test" is created and the content is "testtest"
> the result I got:
> hdfs throw an exception on the second append operation.
> {code}
> ./test.sh 
> {"RemoteException":{"exception":"IOException","javaClassName":"java.io.IOException","message":"Failed
to add a datanode: nodes.length != original.length + 1, nodes=[127.0.0.1:50010], original=[127.0.0.1:50010]"}}
> {code}
> Log in datanode:
> {code}
> 2012-04-02 14:34:21,058 WARN org.apache.hadoop.hdfs.DFSClient: DataStreamer Exception
> java.io.IOException: Failed to add a datanode: nodes.length != original.length + 1, nodes=[127.0.0.1:50010],
original=[127.0.0.1:50010]
> 	at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.findNewDatanode(DFSOutputStream.java:778)
> 	at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.addDatanode2ExistingPipeline(DFSOutputStream.java:834)
> 	at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.setupPipelineForAppendOrRecovery(DFSOutputStream.java:930)
> 	at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:461)
> 2012-04-02 14:34:21,059 ERROR org.apache.hadoop.hdfs.DFSClient: Failed to close file
/test
> java.io.IOException: Failed to add a datanode: nodes.length != original.length + 1, nodes=[127.0.0.1:50010],
original=[127.0.0.1:50010]
> 	at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.findNewDatanode(DFSOutputStream.java:778)
> 	at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.addDatanode2ExistingPipeline(DFSOutputStream.java:834)
> 	at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.setupPipelineForAppendOrRecovery(DFSOutputStream.java:930)
> 	at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:461)
> {code}
> test.sh
> {code}
> #!/bin/sh
> echo "test" > test.txt
> curl -L -X PUT "http://localhost:50070/webhdfs/v1/test?op=CREATE"
> curl -L -X POST -T test.txt "http://localhost:50070/webhdfs/v1/test?op=APPEND"
> curl -L -X POST -T test.txt "http://localhost:50070/webhdfs/v1/test?op=APPEND"
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message