hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Owen O'Malley (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-23) single node cluster gets one reducer
Date Mon, 06 Feb 2006 22:51:59 GMT
    [ http://issues.apache.org/jira/browse/HADOOP-23?page=comments#action_12365359 ] 

Owen O'Malley commented on HADOOP-23:
-------------------------------------

Ok, I found the problem and it was a config file ordering problem.

The problem is that the job's config file is loaded as a default resource and the site config
file is a final resource and I had set the number of reduces to one in the site file, because
on a single node cluster that seemed like a reasonable default.

Obviously, removing the default in the site file is necessary.

I realize now that is why you created the separate mapred-default.xml. It doesn't feel right
to have a config file that is specific for map/reduce, but I think there should be some kind
of default config file that the user can override in the job but that is editted by the site.

Things that probably belong there:
  fs.default.name
  mapred.job.tracker
  mapred.map.tasks
  mapred.reduce.tasks

> single node cluster gets one reducer
> ------------------------------------
>
>          Key: HADOOP-23
>          URL: http://issues.apache.org/jira/browse/HADOOP-23
>      Project: Hadoop
>         Type: Bug
>   Components: mapred
>     Reporter: Owen O'Malley
>     Priority: Minor

>
> Running on a single node cluster (it runs a job tracker and a single task tracker), even
though my application asks for 7 reduces, it only gets one. I haven't tracked down what is
happening yet.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira


Mime
View raw message