hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vinod K V (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-5850) map/reduce doesn't run jobs with 0 maps
Date Wed, 20 May 2009 20:15:45 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-5850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Vinod K V updated HADOOP-5850:
------------------------------

    Attachment: HADOOP-5850-20090520-svn.1.txt

Attaching final patch. The previous patch had some problems because of which TestSetupAndCleanupFailure
failed.

This patch fixes those problems. It passed ant test-patch, core and contrib tests except the
following which failed/timeout even without this patch and are unrelated to the changes in
this patch.

Failed:
 - org.apache.hadoop.streaming.TestMultipleCachefiles
 - org.apache.hadoop.streaming.TestStreamingBadRecords
 - org.apache.hadoop.streaming.TestSymLink

Timedout:
 - org.apache.hadoop.mapred.TestJobInProgressListener FAILED (timeout)
 - org.apache.hadoop.mapred.TestQueueCapacities FAILED (timeout)

> map/reduce doesn't run jobs with 0 maps
> ---------------------------------------
>
>                 Key: HADOOP-5850
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5850
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: mapred
>    Affects Versions: 0.20.0
>            Reporter: Owen O'Malley
>            Assignee: Vinod K V
>            Priority: Critical
>             Fix For: 0.20.1
>
>         Attachments: HADOOP-5850-20090519.1.txt, HADOOP-5850-20090519.txt, HADOOP-5850-20090520-svn.1.txt
>
>
> Currently, the framework ignores jobs that have 0 maps. This is incorrect. Many pipelines
need the job to run (if nothing else, to create the output directory!) so that subsequent
jobs don't fail. Effectively, there will be no map tasks and the reduce tasks should immediately
set up the Reducer and RecordWriter and then call close on both since there are no inputs
to the reduce. I believe it should just work if we remove the check...

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message