hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Monish r <monishs...@gmail.com>
Subject Re: Major Compaction in 0.90.6
Date Tue, 20 Aug 2013 06:52:36 GMT
Hi Jean,
Thanks for the explanation.

Just a clarification on the third answer,

In our current cluster ( 0.90.6 ) , i find that irrespective of whether TTL
is set or not , Major compaction compaction rewrites hfile for the region (
there is only one hfile for that region )  on every manual major compaction
trigger.

log :

2013-08-19 14:15:29,926 INFO org.apache.hadoop.hbase.regionserver.Store:
Completed major compaction of 1 file(s), new
file=hdfs://x.x.x.x:9000/hbase/NOTIFICATION_HISTORY/b00086bca62ee55796a960002291aca4/n/4754838096619480671

i find a new file is created for every major compaction triggger.

Regards,
R.Monish


On Mon, Aug 19, 2013 at 11:52 PM, Jean-Daniel Cryans <jdcryans@apache.org>wrote:

> Inline.
>
> J-D
>
>
> On Mon, Aug 19, 2013 at 2:48 AM, Monish r <monishsvce@gmail.com> wrote:
>
> > Hi guys,
> > I have the following questions in HBASE 0.90.6
> >
> > 1. Does hbase use only one compaction thread to handle both major and
> minor
> > compaction?
> >
>
> Yes, look at CompactSplitThread
>
>
> >
> > 2. If hbase uses multiple compaction threads, which configuration
> parameter
> > defines the number of compaction threads?
> >
>
> It doesn't in 0.90.6 but CompactSplitThread lists those for 0.92+
>
> hbase.regionserver.thread.compaction.large
> hbase.regionserver.thread.compaction.small
>
>
> >
> > 3. After hbase.majorcompaction.interval from last major compaction ,if
> > major compaction is executed on a table already major compacted Does
> hbase
> > skip all the table regions from major compaction?
> >
>
> Determining if something is major-compacted is definitely not at the
> table-level.
>
> In 0.90.6, MajorCompactionChecker will ask HRegion.isMajorCompaction() to
> check if it needs to major compact again, which in turns checks every
> Store. FWW if you have TTL turned on it will still major compact a major
> compacted file, HFiles don't have an index of what's deleted or TTL'd and
> it doesn't do a full read of each files to check.
>
>
> >
> > Regards,
> > R.Monish
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message