hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Amareshwari Sriramadasu (JIRA)" <j...@apache.org>
Subject [jira] Resolved: (MAPREDUCE-450) OutputFormat should have a "close" method called after all the reducers have completed
Date Thu, 11 Feb 2010 06:18:27 GMT

     [ https://issues.apache.org/jira/browse/MAPREDUCE-450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Amareshwari Sriramadasu resolved MAPREDUCE-450.
-----------------------------------------------

    Resolution: Duplicate

This functionality is provided by OutputCommitter. See HADOOP-3150

> OutputFormat should have a "close" method called after all the reducers have completed
> --------------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-450
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-450
>             Project: Hadoop Map/Reduce
>          Issue Type: New Feature
>            Reporter: Nathan Marz
>            Priority: Minor
>
> It would be very useful for OutputFormat's to have a "close" method so that any global
logic for outputting can take place there. For example, to output a meta data file along with
all the reduce outputs. The close method should have the following signature and be run after
all the reducers have finished, but before the "work output path" is renamed to the final
output path:
> void close(FileSystem fs, JobConf job, Progressable progress) throws IOException

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message