hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From mukund murrali <mukundmurra...@gmail.com>
Subject Re: Major compaction skipping for older regions
Date Mon, 10 Aug 2015 14:02:28 GMT
Any one help us in this :(  Are we missing somewhere in the use case? None
of the deleted cells are undergoing major compaction.

Thanks

On Wed, Aug 5, 2015 at 12:04 PM, mukund murrali <mukundmurrali9@gmail.com>
wrote:

> Hi,
>
> We wanted to have deleted data for a week. So we configured
>
> MIN_VERSIONS => 1
> KEEP_DELETED_CELLS => TTL
> TTL => 1 week.
>
> As per our understanding, after 1 week the deleted data becomes available
> for major compaction and should be purged (correct if wrong). Since we have
> time series data, we don't have any write operations in those regions after
> a week . But major compaction never took place for any regions and our
> overall size grew drastically though we have deletes happening. After
> analyzing, we found that major compaction takes place if any one of the 2
> condition is satisfied.
>
> 1. If the time interval between major compaction is greater than a week
> (default config).
> 2. if the block locality index falls below a threshold.
>
> In our case, since we have min_versions to be 1, the first case condition
> fails. Time to verify is set to Long.Max value, if min versions is not 0.
>
> Second is block locality.  To check the block locality index we enabled
> fine logs. And we found the  block locality is always 1, and we got logs
> stating "Skipping major compaction......".
>
> So, in this case is manually triggering major compaction the only choice?
>
> Thanks
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message