hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Douglas (JIRA)" <j...@apache.org>
Subject [jira] Updated: (MAPREDUCE-1726) fs.inmemory.size.mb not listed in conf. Cluster setup page gives wrong advice.
Date Sat, 24 Apr 2010 22:41:50 GMT

     [ https://issues.apache.org/jira/browse/MAPREDUCE-1726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Chris Douglas updated MAPREDUCE-1726:
-------------------------------------

    Attachment: M1726-0.patch

Moving issue to MAPREDUCE, as that is the current home of the cluster setup docs.

bq. If I understandard correctly the docs for current are based on current stable 0.20.2.
Current stable does not use fs.inmemory.size.mb. 

OK, I understand. HADOOP-3446 updated the mapred tutorial, but failed to update the cluster
setup docs.

bq. Think about the end user. An end user might read a blog that states, "io.sort.factor is
a magic tune set this to XXXX for awesome performance". Which file should end user put this
variable in?

It is inconsistent, but as long as the user adds the preferred value to one of the {{\-site.xml}}
files it should make no functional difference in MapReduce. You're right, though: the properties
you cite are usually added to {{mapred-site.xml}} anyway, because the SequenceFile sort is
rarely used. The situation is worse in trunk, where the properties are mapreduce specific,
but users are still directed to {{core-site.xml}}.

Thanks for clarifying.

> fs.inmemory.size.mb not listed in conf. Cluster setup page gives wrong advice.
> ------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-1726
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1726
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: documentation
>    Affects Versions: 0.20.2, 0.21.0, 0.22.0
>            Reporter: Edward Capriolo
>         Attachments: M1726-0.patch
>
>
> http://hadoop.apache.org/common/docs/current/cluster_setup.html
> fs.inmemory.size.mb does not appear in any xml file
> {noformat}
> grep "fs.inmemory.size.mb" ./mapred/mapred-default.xml 
> [edward@ec src]$ grep "fs.inmemory.size.mb" ./hdfs/hdfs-default.xml 
> [edward@ec src]$ grep "fs.inmemory.size.mb" ./core/core-default.xml 
> {noformat}
> http://hadoop.apache.org/common/docs/current/cluster_setup.html
> Documentation error:
> Real-World Cluster Configurations
> {noformat}
> conf/core-site.xml  	io.sort.factor  	100  	More streams merged at once while sorting
files.
> conf/core-site.xml 	io.sort.mb 	200 	Higher memory-limit while sorting data.
> {noformat}
> core --- io.sort.factor					-- should be mapred
> core --- io.sort.mb					-- should be mapred

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message