hadoop-common-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Wiki <wikidi...@apache.org>
Subject [Hadoop Wiki] Update of "PerformanceTuning" by JeanDanielCryans
Date Tue, 15 Dec 2009 23:17:49 GMT
Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Hadoop Wiki" for change notification.

The "PerformanceTuning" page has been changed by JeanDanielCryans.
The comment on this change is: Deferred log flush is now default..
http://wiki.apache.org/hadoop/PerformanceTuning?action=diff&rev1=7&rev2=8

--------------------------------------------------

   * Are you swapping?  JVMs hate swapping.  Adjusting [[http://www.sollers.ca/blog/2008/swappiness/|swappiness]]
may help.
   * By default, each regionserver puts up 10 listeners only.  Up it if you have measurable
traffic (See hbase.regionserver.handler.count in hbase-default.xml).
   * To speed up the inserts in a non critical job (like an import job), you can use Put.writeToWAL(false)
to bypass writing to the write ahead log.
-  * New in HBase 0.21 (not released yet), you can set a table-scope attribute to defer the
write ahead log's flushes in order to improve write performance by exposing your table to
less durability. Use HTableDescriptor.setDeferredLogFlush(true) when creating your table.
Canonical use of this feature would be a table where you only keep counters and losing a few
seconds or milliseconds worth of edits isn't critical (when a region server dies). By setting
hbase.regionserver.optionallogflushinterval to a value like 1000 would mean that in the worst
case you lose 1 second of edits. Since the WAL is shared on a region server, any other table
not using this feature will flush your other table's edit.
+  * New in HBase 0.21 (not released yet), you can set a table-scope attribute to defer the
write ahead log's flushes in order to improve write performance, this is now the default behavior.
Use HTableDescriptor.setDeferredLogFlush(false) when creating your table to instead make sure
every WAL edit is flushed to the Datanodes. The default setting of hbase.regionserver.optionallogflushinterval
is 1000 which means that in the worst case you lose 1 second of edits. Since the WAL is shared
on a region server for all regions, any other table not using this feature will flush your
other table's edits. 
  
  == HBase JVM and GC ==
  

Mime
View raw message