hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rakesh R (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-8220) Erasure Coding: StripedDataStreamer fails to handle the blocklocations which doesn't satisfy BlockGroupSize
Date Tue, 12 May 2015 13:10:00 GMT

    [ https://issues.apache.org/jira/browse/HDFS-8220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14539806#comment-14539806
] 

Rakesh R commented on HDFS-8220:
--------------------------------

Thanks [~walter.k.su] for your time and try out the unit test. The way I shutdown the datanodes
was wrong previously. Now I've reworked the patch and uploaded the same. Also, for the safer
side I've included triggering dn heartbeats. Could you please look at the patch again.

> Erasure Coding: StripedDataStreamer fails to handle the blocklocations which doesn't
satisfy BlockGroupSize
> -----------------------------------------------------------------------------------------------------------
>
>                 Key: HDFS-8220
>                 URL: https://issues.apache.org/jira/browse/HDFS-8220
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Rakesh R
>            Assignee: Rakesh R
>         Attachments: HDFS-8220-001.patch, HDFS-8220-002.patch, HDFS-8220-003.patch, HDFS-8220-004.patch,
HDFS-8220-HDFS-7285.005.patch, HDFS-8220-HDFS-7285.006.patch, HDFS-8220-HDFS-7285.007.patch,
HDFS-8220-HDFS-7285.007.patch, HDFS-8220-HDFS-7285.008.patch
>
>
> During write operations {{StripedDataStreamer#locateFollowingBlock}} fails to validate
the available datanodes against the {{BlockGroupSize}}. Please see the exception to understand
more:
> {code}
> 2015-04-22 14:56:11,313 WARN  hdfs.DFSClient (DataStreamer.java:run(538)) - DataStreamer
Exception
> java.lang.NullPointerException
> 	at java.util.concurrent.LinkedBlockingQueue.offer(LinkedBlockingQueue.java:374)
> 	at org.apache.hadoop.hdfs.StripedDataStreamer.locateFollowingBlock(StripedDataStreamer.java:157)
> 	at org.apache.hadoop.hdfs.DataStreamer.nextBlockOutputStream(DataStreamer.java:1332)
> 	at org.apache.hadoop.hdfs.DataStreamer.run(DataStreamer.java:424)
> 	at org.apache.hadoop.hdfs.StripedDataStreamer.run(StripedDataStreamer.java:1)
> 2015-04-22 14:56:11,313 INFO  hdfs.MiniDFSCluster (MiniDFSCluster.java:shutdown(1718))
- Shutting down the Mini HDFS Cluster
> 2015-04-22 14:56:11,313 ERROR hdfs.DFSClient (DFSClient.java:closeAllFilesBeingWritten(608))
- Failed to close inode 16387
> java.io.IOException: DataStreamer Exception: 
> 	at org.apache.hadoop.hdfs.DataStreamer.run(DataStreamer.java:544)
> 	at org.apache.hadoop.hdfs.StripedDataStreamer.run(StripedDataStreamer.java:1)
> Caused by: java.lang.NullPointerException
> 	at java.util.concurrent.LinkedBlockingQueue.offer(LinkedBlockingQueue.java:374)
> 	at org.apache.hadoop.hdfs.StripedDataStreamer.locateFollowingBlock(StripedDataStreamer.java:157)
> 	at org.apache.hadoop.hdfs.DataStreamer.nextBlockOutputStream(DataStreamer.java:1332)
> 	at org.apache.hadoop.hdfs.DataStreamer.run(DataStreamer.java:424)
> 	... 1 more
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message