hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Raghu Angadi (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-757) "Bad File Descriptor" in closing DFS file
Date Thu, 11 Jan 2007 18:51:27 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12463989
] 

Raghu Angadi commented on HADOOP-757:
-------------------------------------

HADOOP-758 has a patch that handles exceptions better and ignores the exception show in the
description above. It is not enough since we are actually trying to write a fd that we don't
own, which could corrupt data if someone else gets that fd. Not sure why FileOutputStream
does not detect that it is already closed. Fix is to set backupStream to null when it is closed
and handle. 

> "Bad File Descriptor" in closing DFS file
> -----------------------------------------
>
>                 Key: HADOOP-757
>                 URL: https://issues.apache.org/jira/browse/HADOOP-757
>             Project: Hadoop
>          Issue Type: Bug
>          Components: dfs
>    Affects Versions: 0.8.0
>            Reporter: Owen O'Malley
>         Assigned To: Raghu Angadi
>
> Running the sort benchmark, I had a reduce fail with a DFS error:
> java.io.IOException: Bad file descriptor
> 	at java.io.FileOutputStream.writeBytes(Native Method)
> 	at java.io.FileOutputStream.write(FileOutputStream.java:260)
> 	at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.flushData(DFSClient.java:1128)
> 	at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.flush(DFSClient.java:1114)
> 	at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.close(DFSClient.java:1241)
> 	at java.io.FilterOutputStream.close(FilterOutputStream.java:143)
> 	at org.apache.hadoop.fs.FSDataOutputStream$Summer.close(FSDataOutputStream.java:99)
> 	at java.io.FilterOutputStream.close(FilterOutputStream.java:143)
> 	at java.io.FilterOutputStream.close(FilterOutputStream.java:143)
> 	at java.io.FilterOutputStream.close(FilterOutputStream.java:143)
> 	at org.apache.hadoop.io.SequenceFile$Writer.close(SequenceFile.java:515)
> 	at org.apache.hadoop.mapred.SequenceFileOutputFormat$1.close(SequenceFileOutputFormat.java:71)
> 	at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:310)
> 	at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1271)

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message