Mailing-List: contact hadoop-dev-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hadoop-dev@lucene.apache.org
Message-ID: <26122586.1153952954813.JavaMail.jira@brutus>
Date: Wed, 26 Jul 2006 15:29:14 -0700 (PDT)
From: "Wendy Chien (JIRA)" <jira@apache.org>
To: hadoop-dev@lucene.apache.org
Subject: [jira] Commented: (HADOOP-324) "IOException: No space left on
 device" is handled incorrectly
In-Reply-To: <18409093.1151369969863.JavaMail.jira@brutus>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit

    [ http://issues.apache.org/jira/browse/HADOOP-324?page=comments#action_12423728 ] 
            
Wendy Chien commented on HADOOP-324:
------------------------------------

I'm fixing this issue by creating a new exception (DiskOutOfSpaceException) that extends IOException.  In DataNode:writeBlock, a DiskOutOFSpaceException is thrown if while opening local disk out, data.writeToBlock fails due to insufficient space and if while processing the incoming data, the amount we want to write (bytesRead) is greater than the remaining space in the data set.

This comment serves to claim this bug as mine and as an invitation for comments on this approach.   


> "IOException: No space left on device" is handled incorrectly
> -------------------------------------------------------------
>
>                 Key: HADOOP-324
>                 URL: http://issues.apache.org/jira/browse/HADOOP-324
>             Project: Hadoop
>          Issue Type: Bug
>    Affects Versions: 0.3.2
>            Reporter: Konstantin Shvachko
>
> When a data node disk is almost full the name node still assigns blocks to the data node.
> By the time the data node actually tries to write that data to disk the disk may become full.
> Current implementation forces the data node to shutdown after that.
> The expected behavior is to report the block write failure and continue.
> The Exception looks as follows:
> java.io.IOException: No space left on device
> at java.io.FileOutputStream.writeBytes(Native Method)
> at java.io.FileOutputStream.write(FileOutputStream.java:260)
> at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
> at java.io.BufferedOutputStream.write(BufferedOutputStream.java:109)
> at java.io.DataOutputStream.write(DataOutputStream.java:90)
> at org.apache.hadoop.dfs.DataNode$DataXceiver.writeBlock(DataNode.java:623)
> at org.apache.hadoop.dfs.DataNode$DataXceiver.run(DataNode.java:410)
> at java.lang.Thread.run(Thread.java:595)
> 2006-06-26 08:26:04,751 INFO org.apache.hadoop.dfs.DataNode: Finishing DataNode in: /tmp/hadoop/dfs/data/data

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira