hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Amareshwari Sriramadasu (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-3150) Move task file promotion into the task
Date Thu, 10 Jul 2008 06:39:32 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-3150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12612386#action_12612386
] 

Amareshwari Sriramadasu commented on HADOOP-3150:
-------------------------------------------------

Commit of side files can be supported by _static_ commit method in FileOutputFormat. Then
task commit can include OutputFormat's commit and side file's commit. Also, cleaning up _\_temporary_
directory at the end of the job needs to be static.

bq. My vote goes to the option of maintaining the current semantics of the the providing the
api to get the directory where the user can create the side-files. It creates less api churn,
also is more amenable to Hadoop Pipes and such others.
SideFile creation in Pipes is still supported by configuration property _mapred.work.output.dir_


> Move task file promotion into the task
> --------------------------------------
>
>                 Key: HADOOP-3150
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3150
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: mapred
>            Reporter: Owen O'Malley
>            Assignee: Amareshwari Sriramadasu
>             Fix For: 0.19.0
>
>         Attachments: 3150.patch
>
>
> We need to move the task file promotion from the JobTracker to the Task and move it down
into the output format.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message