hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Raghu Angadi (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-758) FileNotFound on DFS block file
Date Wed, 10 Jan 2007 20:44:27 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12463708
] 

Raghu Angadi commented on HADOOP-758:
-------------------------------------

> Which raises the related question: why did the cleanup fail? Closing an open file shouldn't
throw an exception. That looks like a bug in DFSClient, no?

Yes. FSDataOutputStream does not store its failure condition. so initial flush() fails and
subsequent close() also result in flush() calls, which fail again. Do we want to fix that?
 I am not sure if this will require trapping exceptions in multiple places. will look into.

Since there are multiple java filters in the stream, close() actually results in many 
FSDataOutputStream.flush() calls.  Java FilterOutputStream ignores exceptions from flush()
during close.


> FileNotFound on DFS block file
> ------------------------------
>
>                 Key: HADOOP-758
>                 URL: https://issues.apache.org/jira/browse/HADOOP-758
>             Project: Hadoop
>          Issue Type: Bug
>          Components: dfs
>    Affects Versions: 0.8.0
>            Reporter: Owen O'Malley
>         Assigned To: Raghu Angadi
>         Attachments: HADOOP-758.patch
>
>
> While run the sort benchmark a reduce failed with:
> java.io.FileNotFoundException: /tmp/hadoop-oom/dfs/tmp/tmp/client-4362164194084664090
(No such file or directory)
> 	at java.io.FileInputStream.open(Native Method)
> 	at java.io.FileInputStream.(FileInputStream.java:106)
> 	at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.endBlock(DFSClient.java:1156)
> 	at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.close(DFSClient.java:1244)
> 	at java.io.FilterOutputStream.close(FilterOutputStream.java:143)
> 	at java.io.FilterOutputStream.close(FilterOutputStream.java:143)
> 	at java.io.FilterOutputStream.close(FilterOutputStream.java:143)
> 	at org.apache.hadoop.fs.FSDataOutputStream$Summer.close(FSDataOutputStream.java:98)
> 	at java.io.FilterOutputStream.close(FilterOutputStream.java:143)
> 	at java.io.FilterOutputStream.close(FilterOutputStream.java:143)
> 	at java.io.FilterOutputStream.close(FilterOutputStream.java:143)
> 	at org.apache.hadoop.io.SequenceFile$Writer.close(SequenceFile.java:515)
> 	at org.apache.hadoop.mapred.SequenceFileOutputFormat$1.close(SequenceFileOutputFormat.java:71)
> 	at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:310)
> 	at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1271)

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message