hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Milind Bhandarkar (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-5850) map/reduce doesn't run jobs with 0 maps
Date Tue, 19 May 2009 22:43:45 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-5850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12710930#action_12710930
] 

Milind Bhandarkar commented on HADOOP-5850:
-------------------------------------------

Should the output directories be empty or should they produce zero-length part files ?

> map/reduce doesn't run jobs with 0 maps
> ---------------------------------------
>
>                 Key: HADOOP-5850
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5850
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: mapred
>    Affects Versions: 0.20.0
>            Reporter: Owen O'Malley
>            Assignee: Vinod K V
>            Priority: Critical
>             Fix For: 0.20.1
>
>         Attachments: HADOOP-5850-20090519.1.txt, HADOOP-5850-20090519.txt
>
>
> Currently, the framework ignores jobs that have 0 maps. This is incorrect. Many pipelines
need the job to run (if nothing else, to create the output directory!) so that subsequent
jobs don't fail. Effectively, there will be no map tasks and the reduce tasks should immediately
set up the Reducer and RecordWriter and then call close on both since there are no inputs
to the reduce. I believe it should just work if we remove the check...

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message