Mailing-List: contact core-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: core-dev@hadoop.apache.org
Message-ID: <564302880.1204524650478.JavaMail.jira@brutus>
Date: Sun, 2 Mar 2008 22:10:50 -0800 (PST)
From: "Hemanth Yamijala (JIRA)" <jira@apache.org>
To: core-dev@hadoop.apache.org
Subject: [jira] Commented: (HADOOP-2815) Allowing processes to cleanup dfs
 on shutdown
In-Reply-To: <19116229.1202953628163.JavaMail.jira@brutus>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HADOOP-2815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12574351#action_12574351 ] 

Hemanth Yamijala commented on HADOOP-2815:
------------------------------------------

While Devaraj and I were trying to fix HADOOP-2899, we came up with a solution that exactly matches Dhruba's comment above. In HADOOP-2899, JobTracker needed to cleanup the directory pointed to by mapred.system.dir when it is shutting down. We experimented with some code in the JobTracker that adds a shutdown hook and deletes the directory at cleanup. The requirement for deleting the directory works, but we faced the following problems:
- the addition / removal of additional shutdown hooks sometimes causes IllegalStateExceptions in the FileSystem code
- the non-deterministic order of running shutdown hooks in the JVM actually makes the whole thing complicated, and seemed unsafe.

Dhruba's solution would work perfectly to handle this scenario, and seems a very clean approach. The JobTracker would just register a shutdown hook with the FileSystem, and would cleanup the system directory then. So, +1 for the suggestion.

Is there any chance this code can come up in Hadoop 0.16.1 because HADOOP-2899 should ideally be fixed in 0.16.1 ?


> Allowing processes to cleanup dfs on shutdown
> ---------------------------------------------
>
>                 Key: HADOOP-2815
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2815
>             Project: Hadoop Core
>          Issue Type: New Feature
>          Components: dfs
>            Reporter: Olga Natkovich
>            Assignee: dhruba borthakur
>
> Pig creates temp files that it wants to be removed at the end of the processing. The code that removes the temp file is in the shutdown hook so that they get removed both under normal shutdown as well as when process gets killed.
> The problem that we are seeing is that by the time the code is called the DFS might already be closed and the delete fails leaving temp files behind. Since we have no control over the shutdown order, we have no way to make sure that the files get removed.
> One way to solve this issue is to be able to mark the files as temp files so that hadoop can remove them during its shutdown.
> The stack trace I am seeing is
> at org.apache.hadoop.dfs.DFSClient.checkOpen(DFSClient.java:158)
>         at org.apache.hadoop.dfs.DFSClient.delete(DFSClient.java:417)
>         at org.apache.hadoop.dfs.DistributedFileSystem.delete(DistributedFileSystem.java:144)
>         at org.apache.pig.backend.hadoop.datastorage.HPath.delete(HPath.java:96)
>         at org.apache.pig.impl.io.FileLocalizer$1.run(FileLocalizer.java:275)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.