hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alejandro Abdelnur (JIRA)" <j...@apache.org>
Subject [jira] Created: (HADOOP-1558) changes to OutputFormat to work on temporary directory to enable re-running crashed jobs (Issue: 1121)
Date Mon, 02 Jul 2007 13:05:04 GMT
changes to OutputFormat to work on temporary directory to enable re-running crashed jobs (Issue:
1121)
------------------------------------------------------------------------------------------------------

                 Key: HADOOP-1558
                 URL: https://issues.apache.org/jira/browse/HADOOP-1558
             Project: Hadoop
          Issue Type: Improvement
          Components: mapred
         Environment: all
            Reporter: Alejandro Abdelnur
             Fix For: 0.14.0


Add  OutputFormat methods like:

/** Called to initialize output for this job. */
void initialize(JobConf job) throws IOException;

/** Called to finalize output for this job. */
void commit(JobConf job) throws IOException;

In the base implemenation for FileSystem output, initialize() might then create a temporary
directory for the job, removing any that already exists, and commit could rename the temporary
output directory to the final name. 

The existing checkOutputSpecs() would continue to throw an exception if the final output already
exists.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message