hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tom White (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-5006) streaming tests failing
Date Fri, 22 Feb 2013 10:54:13 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-5006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13584183#comment-13584183
] 

Tom White commented on MAPREDUCE-5006:
--------------------------------------

bq. MAPREDUCE-4994 made it so that LocalClientProtocolProvider doesn't always set the number
of map tasks to 1. 

Changing the number of map tasks for the local job runner doesn't sound right to me. It makes
the behaviour different to Hadoop 1, which is bad because users' streaming jobs may behave
differently. Also, setting the number of maps is just a hint. Even with mapreduce.job.maps
set to 1, then there will still be one map per block (see FileInputFormat#getSplits), which
is why applications don't normally set mapreduce.job.maps. So I think we should revert the
change that MAPREDUCE-4994 made here.
                
> streaming tests failing
> -----------------------
>
>                 Key: MAPREDUCE-5006
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5006
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: contrib/streaming
>    Affects Versions: 2.0.4-beta
>            Reporter: Alejandro Abdelnur
>            Assignee: Sandy Ryza
>         Attachments: MAPREDUCE-5006.patch
>
>
> The following 2 tests are failing in trunk
> * org.apache.hadoop.streaming.TestStreamReduceNone
> * org.apache.hadoop.streaming.TestStreamXmlRecordReader

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message