Mailing-List: contact common-commits-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: common-dev@hadoop.apache.org
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: quoted-printable
From: Apache Wiki <wikidiffs@apache.org>
To: Apache Wiki <wikidiffs@apache.org>
Date: Thu, 24 Feb 2011 19:05:40 -0000
Message-ID: <20110224190540.72968.82586@eosnew.apache.org>
Subject: 
 =?utf-8?q?=5BHadoop_Wiki=5D_Update_of_=22PerformanceTuning=22_by_stack?=

Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Hadoop Wiki" for ch=
ange notification.

The "PerformanceTuning" page has been changed by stack.
http://wiki.apache.org/hadoop/PerformanceTuning?action=3Ddiff&rev1=3D9&rev2=
=3D10

--------------------------------------------------

   * Ram, ram, ram.  Don't starve HBase.
   * More CPUs is important, as you will see in the next section
   * Use a 64-bit platform, and a 64-bit JVM.
-  * Your clients might need tuning: [[http://ryantwopointoh.blogspot.com/2=
009/01/performance-of-hbase-importing.html]]
+  * Your clients might need tuning: [[http://ryantwopointoh.blogspot.com/2=
009/01/performance-of-hbase-importing.html|Performance of hbase importing]]
   * Make sure that the command {{{java}}} implies {{{-server}}} on your ma=
chines, or else you will have to explicitly enable it.
   * Are you swapping?  JVMs hate swapping.  Consider removing swap space.
   * By default, each regionserver puts up 10 listeners only.  Up it if you=
 have measurable traffic (See hbase.regionserver.handler.count in hbase-def=
ault.xml).
   * To speed up the inserts in a non critical job (like an import job), yo=
u can use Put.writeToWAL(false) to bypass writing to the write ahead log.
-  * New in HBase 0.21 (not released yet), you can set a table-scope attrib=
ute to defer the write ahead log's flushes in order to improve write perfor=
mance, this is now the default behavior. Use HTableDescriptor.setDeferredLo=
gFlush(false) when creating your table to instead make sure every WAL edit =
is flushed to the Datanodes. The default setting of hbase.regionserver.opti=
onallogflushinterval is 1000 which means that in the worst case you lose 1 =
second of edits. Since the WAL is shared on a region server for all regions=
, any other table not using this feature will flush your other table's edit=
s. =

+  * You can set a table-scope attribute to defer the write ahead log's flu=
shes in order to improve write performance, this is now the default behavio=
r. Use HTableDescriptor.setDeferredLogFlush(false) when creating your table=
 to instead make sure every WAL edit is flushed to the Datanodes. The defau=
lt setting of hbase.regionserver.optionallogflushinterval is 1000 which mea=
ns that in the worst case you lose 1 second of edits. Since the WAL is shar=
ed on a region server for all regions, any other table not using this featu=
re will flush your other table's edits. =

  =

  =3D=3D HBase JVM and GC =3D=3D
  =

  HBase is memory intensive, and using the default GC you can see long paus=
es in all threads.  With the addition of ZooKeeper this can cause false err=
ors as ZooKeeper and the HBase master thinks a regionserver has died.  =

  =

- To avoid this, you must use Java6 CMS: [[http://java.sun.com/javase/techn=
ologies/hotspot/gc/gc_tuning_6.html]]
+ To avoid this, you must use Java6 CMS: [[http://java.sun.com/javase/techn=
ologies/hotspot/gc/gc_tuning_6.html|GC Tuning]]
  =

  To enable, in hbase-env.sh add: {{{
  export HBASE_OPTS=3D"-XX:+UseConcMarkSweepGC -verbose:gc -XX:+PrintGCDeta=
ils -XX:+PrintGCTimeStamps -Xloggc:/home/hadoop/hbase/logs/gc-hbase.log"
@@ -39, +39 @@

  =

  Adjust the log directory to wherever you log.
  =

- the CMS collector will use many threads to do the concurrent sweeping of =
your heap, if you are running on a 2 cpu system, you should probably also e=
nable Incremental mode [[http://java.sun.com/javase/technologies/hotspot/gc=
/gc_tuning_6.html#icms]]: {{{
+ the CMS collector will use many threads to do the concurrent sweeping of =
your heap, if you are running on a 2 cpu system, you should probably also e=
nable [[http://java.sun.com/javase/technologies/hotspot/gc/gc_tuning_6.html=
#icms|Incremental CMS mode]]: {{{
  export HBASE_OPTS=3D"-XX:+UseConcMarkSweepGC -XX:+CMSIncrementalMode <oth=
er options>"
  }}}
  =

@@ -90, +90 @@

  =

  The key points here is to keep all these pauses low.  CMS pauses are alwa=
ys low, but if your ParNew starts growing, you can see minor GC pauses appr=
oach 100ms, exceed 100ms and hit as high at 400ms.
  =

- This can be due to the size of the ParNew, which should be relatively sma=
ll.  If your ParNew is very large after running HBase for a while, in one e=
xample a ParNew was about 150MB, then you might have to constrain the size =
of ParNew.
+ This can be due to the size of the ParNew, which should be relatively sma=
ll.  If your ParNew is very large after running HBase for a while, in one e=
xample a ParNew was about 150MB, then you might have to constrain the size =
of ParNew (The larger it is, the longer the collections take but if its too=
 small, objects are promoted to old gen too quickly).  In the below we cons=
train new gen size to 64m.
  =

  Add this to HBASE_OPTS: {{{
- export HBASE_OPTS=3D"-XX:NewSize=3D6m -XX:MaxNewSize=3D6m <cms options fr=
om above> <gc logging options from above>"
+ export HBASE_OPTS=3D"-XX:NewSize=3D64m -XX:MaxNewSize=3D64m <cms options =
from above> <gc logging options from above>"
  }}}
  =

+ See also the [[http://hbase.apache.org/book.html|HBase book]] for more on=
 GC tuning, in particular for suggestions dealing with long-running stop-th=
e-world GCs.
+=20