hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jason (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-2121) Unexpected IOException in DFSOutputStream.close()
Date Thu, 08 May 2008 23:22:55 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-2121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12595423#action_12595423
] 

Jason commented on HADOOP-2121:
-------------------------------

Cluster running 0.15.3, linux, jdk 1.6

I just had this occur on a machine that had a lot of free disk space, in a cluster where no
datanode was more than 40% full.

I was adding a new machine into a cluster that had a running job. Every job that was pushed
on to the machine failed after a bit, with this error.


> Unexpected IOException in DFSOutputStream.close()
> -------------------------------------------------
>
>                 Key: HADOOP-2121
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2121
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: dfs
>    Affects Versions: 0.14.3
>            Reporter: Raghu Angadi
>            Assignee: Raghu Angadi
>             Fix For: 0.16.0
>
>         Attachments: HADOOP-2121.patch, HADOOP-2121.patch, HADOOP-2121.patch
>
>
> While running a test with datanodes with disk space limitations, Hairong noticed many
IOExceptions like this :
> {noformat}
> java.io.IOException: Mismatch in writeChunk() args
>  at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.writeChunk(DFSClient.java:1575)
>  at org.apache.hadoop.fs.FSOutputSummer.writeChecksumChunk(FSOutputSummer.java:140)
>  at org.apache.hadoop.fs.FSOutputSummer.flushBuffer(FSOutputSummer.java:122)
>  at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.close(DFSClient.java:1715)
>  at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FSDataOutputStream.java:49)
>  at org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:64)
>  at org.apache.hadoop.io.SequenceFile$Writer.close(SequenceFile.java:918)
>  at org.apache.hadoop.mapred.SequenceFileOutputFormat$1.close(SequenceFileOutputFormat.java:72)
>  at org.apache.hadoop.mapred.MapTask$DirectMapOutputCollector.close(MapTask.java:232)
>  at org.apache.hadoop.mapred.MapTask.run(MapTask.java:197)
>  at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1935)
> {noformat}
> I will submit a patch. With the patch, we will still see an IOException, but an expected
one.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message