hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "amith (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-3384) DataStreamer thread should be closed immediatly when failed to setup a PipelineForAppendOrRecovery
Date Wed, 09 May 2012 16:15:50 GMT

     [ https://issues.apache.org/jira/browse/HDFS-3384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

amith updated HDFS-3384:
------------------------

    Attachment: HDFS-3384.patch

When a pipeline setup for append/recovery may fail. If the setup fails we are setting streamerClosed
but there is no condition which cause the streamer thread to stop.

currently in the given stack trace
{code}
java.lang.NullPointerException
	at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:510)
{code}

caused has a side effect of not closing the streamer thread even after setting the streamerClosed.

Manually verified the scenario 
write a file
corrupt block manually
call append




                
> DataStreamer thread should be closed immediatly when failed to setup a PipelineForAppendOrRecovery
> --------------------------------------------------------------------------------------------------
>
>                 Key: HDFS-3384
>                 URL: https://issues.apache.org/jira/browse/HDFS-3384
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: hdfs client
>    Affects Versions: 2.0.0
>            Reporter: Brahma Reddy Battula
>            Assignee: amith
>         Attachments: HDFS-3384.patch
>
>
> Scenraio:
> =========
> write a file
> corrupt block manually
> call append..
> {noformat}
> 2012-04-19 09:33:10,776 INFO  hdfs.DFSClient (DFSOutputStream.java:createBlockOutputStream(1059))
- Exception in createBlockOutputStream
> java.io.EOFException: Premature EOF: no length prefix available
> 	at org.apache.hadoop.hdfs.protocol.HdfsProtoUtil.vintPrefixed(HdfsProtoUtil.java:162)
> 	at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.createBlockOutputStream(DFSOutputStream.java:1039)
> 	at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.setupPipelineForAppendOrRecovery(DFSOutputStream.java:939)
> 	at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:461)
> 2012-04-19 09:33:10,807 WARN  hdfs.DFSClient (DFSOutputStream.java:run(549)) - DataStreamer
Exception
> java.lang.NullPointerException
> 	at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:510)
> 2012-04-19 09:33:10,807 WARN  hdfs.DFSClient (DFSOutputStream.java:hflush(1511)) - Error
while syncing
> java.io.IOException: All datanodes 10.18.40.20:50010 are bad. Aborting...
> 	at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.setupPipelineForAppendOrRecovery(DFSOutputStream.java:908)
> 	at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:461)
> java.io.IOException: All datanodes 10.18.40.20:50010 are bad. Aborting...
> 	at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.setupPipelineForAppendOrRecovery(DFSOutputStream.java:908)
> 	at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:461)
> {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message