hadoop-common-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Wiki <wikidi...@apache.org>
Subject [Lucene-hadoop Wiki] Trivial Update of "HowToConfigure" by NigelDaley
Date Wed, 11 Oct 2006 20:45:45 GMT
Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Lucene-hadoop Wiki" for change notification.

The following page has been changed by NigelDaley:
http://wiki.apache.org/lucene-hadoop/HowToConfigure

------------------------------------------------------------------------------
  
  == Primary XML Files ==
  
- Hadoop is configured with a set of files. The files are loaded in order list in the table
below, with the lower files in the table overriding the higher ones:
+ Hadoop is configured with a set of files. The files are loaded in the order listed in the
table below, with the lower files in the table overriding the higher ones:
  
  || '''Filename''' || '''Description''' ||
  || hadoop-default.xml || Generic default values ||
@@ -16, +16 @@

  
  Configuration files are found via Java's Classpath. Only the first instance of each file
is used. The $HADOOP_CONF_DIR is added by the bin/hadoop script to the front of the path.
When installing Hadoop on a cluster, it is best to use a conf directory outside of the distribution.
That allows you to easily update the release on the cluster without changing your configuration
by mistake.
  
- === Hadoop-default.xml ===
+ === hadoop-default.xml ===
  
  This file has the default values for many of the configuration variables that are used by
Hadoop. This file should never be in $HADOOP_CONF_DIR so that the version in the hadoop-*.jar
is used. (Otherwise, if a variable is added to this file in a new release, you won't have
it defined.)
  
  === mapred-default.xml ===
  
- This file should contain the majority of your site's customization of Hadoop. Although this
file is named mapred, the default settings for the user maps and reduces are controlled by
it.
+ This file should contain the majority of your site's customization of Hadoop. Although this
file name is prefixed with mapred, the default settings for the user maps and reduces are
controlled by it.
  
  Some useful variables are:
  
  || '''Name''' || '''Meaning''' ||
  || dfs.block.size || size in bytes of each data block in DFS ||
- || io.sort.factor || number of input files to each level in the merge sort ||
+ || io.sort.factor || number of files input to each level in the merge sort ||
  || io.sort.mb || size of buffer to sort the reduce inputs in ||
  || io.file.buffer.size || number of bytes used for buffering io files ||
  || mapred.reduce.parallel.copies || number of threads fetching map outputs for each reduce
||
  || dfs.replication || number of replicas for each DFS block ||
- || mapred.child.java.opts || options passed to children task jvms ||
+ || mapred.child.java.opts || options passed to child task jvms ||
  || mapred.min.split.size || minimum number of bytes in a map input split ||
  || mapred.output.compress || Should the reduce outputs be compressed? ||
  
@@ -45, +45 @@

  
  === hadoop-site.xml ===
  
- This file overrides any settings in the job.xml and therefore should be very minimal. Usually
it just contains, the addresses of the NameNode and JobTracker, the port and working directories
for the various servers.
+ This file overrides any settings in the job.xml and therefore should be very minimal. Usually
it just contains the address of the NameNode, the address of the JobTracker, and the port
and working directories for the various servers.
  
  == Environment Variables ==
  

Mime
View raw message