Mailing-List: contact hadoop-dev-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hadoop-dev@lucene.apache.org
Message-ID: <1674845.1168389507541.JavaMail.jira@brutus>
Date: Tue, 9 Jan 2007 16:38:27 -0800 (PST)
From: "Raghu Angadi (JIRA)" <jira@apache.org>
To: hadoop-dev@lucene.apache.org
Subject: [jira] Commented: (HADOOP-758) FileNotFound on DFS block file
In-Reply-To: <2482003.1164778941128.JavaMail.jira@brutus>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HADOOP-758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12463436 ] 

Raghu Angadi commented on HADOOP-758:
-------------------------------------


The exception in the bug is that last exception that that occurred. It masks the first exception that would be a better indicator of the problem.

ReduceTask.java (around line 313)Looks like 

try { /* run reducer */ } finally { /* close some streams */  }

The above trace and the one in HADOOP-757 both are in finally {} and mask the exception in try {}. I will submit a patch that prints the exception  thrown in try {} if finally block throws one. 

While trying reproduce the above trace I managed to produce "Bad File Descriptor" exception in HADOOP-757.

In summary, it looks like these failures are possible with low tmp spaces but we don't log the exceptions that were triggered initially. 

> FileNotFound on DFS block file
> ------------------------------
>
>                 Key: HADOOP-758
>                 URL: https://issues.apache.org/jira/browse/HADOOP-758
>             Project: Hadoop
>          Issue Type: Bug
>          Components: dfs
>    Affects Versions: 0.8.0
>            Reporter: Owen O'Malley
>         Assigned To: Raghu Angadi
>
> While run the sort benchmark a reduce failed with:
> java.io.FileNotFoundException: /tmp/hadoop-oom/dfs/tmp/tmp/client-4362164194084664090 (No such file or directory)
> 	at java.io.FileInputStream.open(Native Method)
> 	at java.io.FileInputStream.(FileInputStream.java:106)
> 	at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.endBlock(DFSClient.java:1156)
> 	at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.close(DFSClient.java:1244)
> 	at java.io.FilterOutputStream.close(FilterOutputStream.java:143)
> 	at java.io.FilterOutputStream.close(FilterOutputStream.java:143)
> 	at java.io.FilterOutputStream.close(FilterOutputStream.java:143)
> 	at org.apache.hadoop.fs.FSDataOutputStream$Summer.close(FSDataOutputStream.java:98)
> 	at java.io.FilterOutputStream.close(FilterOutputStream.java:143)
> 	at java.io.FilterOutputStream.close(FilterOutputStream.java:143)
> 	at java.io.FilterOutputStream.close(FilterOutputStream.java:143)
> 	at org.apache.hadoop.io.SequenceFile$Writer.close(SequenceFile.java:515)
> 	at org.apache.hadoop.mapred.SequenceFileOutputFormat$1.close(SequenceFileOutputFormat.java:71)
> 	at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:310)
> 	at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1271)

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira