hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Todd Lipcon (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-4485) container-executor should deal with file descriptors better
Date Thu, 26 Jul 2012 01:12:34 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-4485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13422797#comment-13422797

Todd Lipcon commented on MAPREDUCE-4485:

When you fork, Java already takes care of closing all the open file descriptors. See UNIXProcess_md.c
> container-executor should deal with file descriptors better
> -----------------------------------------------------------
>                 Key: MAPREDUCE-4485
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4485
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: nodemanager
>            Reporter: Colin Patrick McCabe
>            Priority: Minor
> container-executor.c contains the following code:
> {code}
>   fclose(stdin);
>   fflush(LOGFILE);
>   if (LOGFILE != stdout) {
>     fclose(stdout);
>   }
>   if (ERRORFILE != stderr) {
>     fclose(stderr);
>   }
>   if (chdir(primary_app_dir) != 0) {
>     fprintf(LOGFILE, "Failed to chdir to app dir - %s\n", strerror(errno));
>     return -1;
>   }
>   execvp(args[0], args);
> {code}
> Whenever you open a new file descriptor, its number is the lowest available number. 
So if {{stdout}} (fd number 1) has been closed, and you do open("/my/important/file"), you'll
get assigned file descriptor 1.  This means that any printf statements in the program will
be now printing to /my/important/file.  Oops!
> The correct way to get rid of stdin, stdout, or stderr is not to close them, but to make
them point to /dev/null.  {{dup2}} can be used for this purpose.
> Another thing we should be doing in container-executor.c is closing any file descriptors
we don't need.  Because container-executor was forked off of the JVM, any file that was open
at the time the JVM called fork() will also be open for us.  These FDs will continue to be
open even after the {{execve}}, unless we close them manually.  This could be both a resource
leak and a security breach.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message