hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ravi Prakash (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-8325) Add a ShutdownHookManager to be used by different components instead of the JVM shutdownhook
Date Sun, 29 Apr 2012 21:55:48 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-8325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13264618#comment-13264618
] 

Ravi Prakash commented on HADOOP-8325:
--------------------------------------

Thanks Tucu! You are right. I had not thought of that.

I'm on the fence because we are serializing a process which should happen fast (so parallelizing
is good). Like you already mentioned 
bq. 1* Add a method in the FileSystem to disable its shutdown hook. MRAppMaster would disable
it and call closeAll() explicitly (it already does).
This is already present (by setting conf.setBoolean("fs.automatic.close", false) ) So I'm
not sure what problem this JIRA is fixing. Or are we planning for the future? I'm fine with
the latter answer being yes. I'm just curious.

The patch looks good for what it intends to do, although the latest one doesn't include the
fixes you made in response to Nicholas' comments.
                
> Add a ShutdownHookManager to be used by different components instead of the JVM shutdownhook
> --------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-8325
>                 URL: https://issues.apache.org/jira/browse/HADOOP-8325
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: fs
>    Affects Versions: 2.0.0
>            Reporter: Alejandro Abdelnur
>            Assignee: Alejandro Abdelnur
>            Priority: Critical
>             Fix For: 2.0.0
>
>         Attachments: HADOOP-8325.patch, HADOOP-8325.patch, HADOOP-8325.patch, HADOOP-8325.patch,
HADOOP-8325.patch, HADOOP-8325.patch, HADOOP-8325.patch
>
>
> FileSystem adds a JVM shutdown hook when a filesystem instance is cached.
> MRAppMaster also uses a JVM shutdown hook, among other things, the MRAppMaster JVM shutdown
hook is used to ensure state are written to HDFS.
> This creates a race condition because each JVM shutdown hook is a separate thread and
if there are multiple JVM shutdown hooks there is not assurance of order of execution, they
could even run in parallel.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message