hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Owen O'Malley (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-2243) Close all the file streams propely in a finally block to avoid their leakage.
Date Mon, 10 Jan 2011 17:04:47 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-2243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12979644#action_12979644
] 

Owen O'Malley commented on MAPREDUCE-2243:
------------------------------------------

In most cases, the exceptions outside of IOException don't matter much because they will bring
down 
the service. In cases where the system will stay up, I'd suggest using:

{code}
try {
  f = fs.open(...);
  f.write(...);
  f.close(...);
} catch (IOException ie) {
  IOUtils.cleanup(LOG, f);
  throw ie;
} catch (RuntimeException re) {
  IOUtils.cleanup(LOG, f);
  throw re;
} catch (Error err) {
  IOUtils.cleanup(LOG, f);
  throw err;
}
{code}

this leaves the nominal case simple. Note that this is the worst case, if we get an Error
every system in Hadoop should shutdown.
There is no point in continuing and worrying about lost file handles at that point is too
extreme. Also note that with Java's
garbage collector, this is far less critical, even for a server, than in C.

> Close all the file streams propely in a finally block to avoid their leakage.
> -----------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-2243
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2243
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: jobtracker, tasktracker
>    Affects Versions: 0.20.1, 0.22.0
>         Environment: NA
>            Reporter: Bhallamudi Venkata Siva Kamesh
>            Priority: Minor
>             Fix For: 0.22.0
>
>   Original Estimate: 72h
>  Remaining Estimate: 72h
>
> In the following classes streams should be closed in finally block to avoid their leakage
in the exceptional cases.
> CompletedJobStatusStore.java
> ------------------------------------------
>        dataOut.writeInt(events.length);
>         for (TaskCompletionEvent event : events) {
>           event.write(dataOut);
>         }
>        dataOut.close() ;
> EventWriter.java
> ----------------------
>    encoder.flush();
>    out.close();
> MapTask.java
> -------------------
>     splitMetaInfo.write(out);
>      out.close();
> TaskLog
> ------------
>  1) str = fis.readLine();
>       fis.close();
> 2) dos.writeBytes(Long.toString(new File(logLocation, LogName.SYSLOG
>       .toString()).length() - prevLogLength) + "\n");
>     dos.close();
> TotalOrderPartitioner.java
> -----------------------------------
>  while (reader.next(key, value)) {
> 	      parts.add(key);
> 	      key = ReflectionUtils.newInstance(keyClass, conf);
> 	    }
> reader.close();

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message