hadoop-common-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Wiki <wikidi...@apache.org>
Subject [Lucene-hadoop Wiki] Update of "FAQ" by DevarajDas
Date Thu, 06 Sep 2007 12:45:23 GMT
Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Lucene-hadoop Wiki" for change notification.

The following page has been changed by DevarajDas:
http://wiki.apache.org/lucene-hadoop/FAQ

------------------------------------------------------------------------------
  
    * `dfs.block.size = 134217728`
    * `dfs.namenode.handler.count = 40`
-   * `mapred.speculative.execution = true`
    * `mapred.reduce.parallel.copies = 20`
    * `mapred.child.java.opts = -Xmx512m`
    * `fs.inmemory.size.mb = 200`
@@ -45, +44 @@

    * `io.sort.mb = 200`
    * `io.file.buffer.size = 131072`
  
- Sort performances on 1400 nodes and 2000 nodes are pretty good too - sorting 14TB of data
on a 1400-node cluster takes 2.2 hours; sorting 20TB on a 2000-node cluster takes 2.5 hours.
+ Sort performances on 1400 nodes and 2000 nodes are pretty good too - sorting 14TB of data
on a 1400-node cluster takes 2.2 hours; sorting 20TB on a 2000-node cluster takes 2.5 hours.
The updates to the above configuration being: 
+   * `mapred.job.tracker.handler.count = 60`
+   * `mapred.reduce.parallel.copies = 50`
+   * `tasktracker.http.threads = 50`
  
  == 4. Do I have to write my application in Java? ==
  

Mime
View raw message