hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ahmed Radwan (JIRA)" <j...@apache.org>
Subject [jira] [Created] (MAPREDUCE-2658) Problem running full map & reduce jobs on mrv2
Date Thu, 07 Jul 2011 20:53:16 GMT
Problem running full map & reduce jobs on mrv2

                 Key: MAPREDUCE-2658
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2658
             Project: Hadoop Map/Reduce
          Issue Type: Bug
          Components: mrv2
            Reporter: Ahmed Radwan

Following the installation instructions at: https://svn.apache.org/repos/asf/hadoop/common/branches/MR-279/mapreduce/INSTALL
the randomwriter example runs successfully. However, other full map & reduce jobs (e.g.
wordcount) fail with the error:

java.lang.UnsupportedOperationException: Incompatible with LocalRunner
	at org.apache.hadoop.mapred.YarnOutputFiles.getInputFile(YarnOutputFiles.java:200)
	at org.apache.hadoop.mapred.ReduceTask.getMapFiles(ReduceTask.java:223)
	at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:412)
	at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:148)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:396)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1094)
	at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:143)

The ReduceTask evaluates the isLocal flag based on the property "mapreduce.jobtracker.address",
the default value for this property in mapred-default.xml is 'local' and this is the cause
of the problem.

Setting "mapreduce.jobtracker.address" in the mapred-site.xml to something other than "local"
seems to solve the problem. 

This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message