hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From tarang dawer <tarang.da...@gmail.com>
Subject Re: Compaction problem
Date Fri, 22 Mar 2013 14:44:17 GMT
Hi
As per my use case , I have to write around 100gb data , with a ingestion
speed of around 200 mbps. While writing , i am getting a performance hit by
compaction , which adds to the delay.
I am using a 8 core machine with 16 gb RAM available., 2 Tb hdd 7200RPM.
Got some idea from the archives and  tried pre splitting the regions ,
configured HBase with following parameters(configured the parameters in a
haste , so please guide me if anything's out of order) :-


        <property>
                <name>hbase.hregion.memstore.block.multiplier</name>
                <value>4</value>
        </property>
        <property>
                 <name>hbase.hregion.memstore.flush.size</name>
                 <value>1073741824</value>
        </property>

        <property>
                <name>hbase.hregion.max.filesize</name>
                <value>1073741824</value>
        </property>
        <property>
                <name>hbase.hstore.compactionThreshold</name>
                <value>5</value>
        </property>
        <property>
              <name>hbase.hregion.majorcompaction</name>
                  <value>0</value>
        </property>
        <property>
                <name>hbase.hstore.blockingWaitTime</name>
                <value>30000</value>
        </property>
         <property>
                 <name>hbase.hstore.blockingStoreFiles</name>
                 <value>200</value>
         </property>

  <property>
        <name>hbase.regionserver.lease.period</name>
        <value>3000000</value>
  </property>


but still m not able to achieve the optimal rate , getting around 110 mbps.
Need some optimizations ,so please could you help out ?

Thanks
Tarang Dawer





On Fri, Mar 22, 2013 at 6:05 PM, Jean-Marc Spaggiari <
jean-marc@spaggiari.org> wrote:

> Hi Tarang,
>
> I will recommand you to take a look at the list archives first to see
> all the discussions related to compaction. You will found many
> interesting hints and tips.
>
>
> http://search-hadoop.com/?q=compactions&fc_project=HBase&fc_type=mail+_hash_+user
>
> After that, you will need to provide more details regarding how you
> are using HBase and how the compaction is impacting you.
>
> JM
>
> 2013/3/22 tarang dawer <tarang.dawer@gmail.com>:
> > Hi
> > I am using HBase 0.94.2 currently. While using it  , its write
> performance,
> > due to compaction is being affeced by compaction.
> > Please could you suggest some quick tips in relation to how to deal with
> it
> > ?
> >
> > Thanks
> > Tarang Dawer
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message