hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jean-Marc Spaggiari <jean-m...@spaggiari.org>
Subject Re: Major compaction issues in hbase
Date Sun, 02 Feb 2014 21:45:37 GMT
Hi Yaniv,

When there is more than 3 files for a region (default setting), HBase will
trigger a minor compaction for this region. However, if all the regions
need to be compacted, then it will promote it as a major compaction. So
even by disabling major compaction so can still see some of it in the logs.

If you can run major compaction one a day it's better than once a week.
Also, this might be related to your design (too many column families?),
your usage and your configuration (compaction triggers, memory
configuration, etc.).

JM


2014-02-02 yanivG <yaniv.yancovich@gmail.com>:

> Hi,
> I am using hbase 0.94.6 version. We are developing a java application and
> using it as our storage for a high load web application.
> The average ration is about 500 requests per second.
> From time to time I see high latency (more then 300 ms!!!) when trying to
> store data object within hbase (using Put).
>
> We disabled major compaction and set it to run manually once a week.
> By examining the logs during those high latency , I see that major
> compaction was executed.
> I saw a post
> (https://groups.google.com/forum/#!topic/nosql-databases/YwE894gq1qM)
> which
> explains it.
> The question is how can I avoid this? Should I ran the major compaction
> once
> a day in low load hours? will it eliminate the sporadic major compaction?
> Thanks,
> Y
>
>
>
>
> --
> View this message in context:
> http://apache-hbase.679495.n3.nabble.com/Major-compaction-issues-in-hbase-tp4055592.html
> Sent from the HBase Developer mailing list archive at Nabble.com.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message