hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jean-Daniel Cryans <jdcry...@apache.org>
Subject Re: Major Compaction in 0.90.6
Date Tue, 20 Aug 2013 17:36:34 GMT
On Mon, Aug 19, 2013 at 11:52 PM, Monish r <monishsvce@gmail.com> wrote:

> Hi Jean,
>

s/Jean/Jean-Daniel ;)


> Thanks for the explanation.
>
> Just a clarification on the third answer,
>
> In our current cluster ( 0.90.6 ) , i find that irrespective of whether TTL
> is set or not , Major compaction compaction rewrites hfile for the region (
> there is only one hfile for that region )  on every manual major compaction
> trigger.
>


Can you enable DEBUG logs? You'd see why the major compaction is triggered.



>
> log :
>
> 2013-08-19 14:15:29,926 INFO org.apache.hadoop.hbase.regionserver.Store:
> Completed major compaction of 1 file(s), new
>
> file=hdfs://x.x.x.x:9000/hbase/NOTIFICATION_HISTORY/b00086bca62ee55796a960002291aca4/n/4754838096619480671
>
> i find a new file is created for every major compaction triggger.
>
> Regards,
> R.Monish
>
>
> On Mon, Aug 19, 2013 at 11:52 PM, Jean-Daniel Cryans <jdcryans@apache.org
> >wrote:
>
> > Inline.
> >
> > J-D
> >
> >
> > On Mon, Aug 19, 2013 at 2:48 AM, Monish r <monishsvce@gmail.com> wrote:
> >
> > > Hi guys,
> > > I have the following questions in HBASE 0.90.6
> > >
> > > 1. Does hbase use only one compaction thread to handle both major and
> > minor
> > > compaction?
> > >
> >
> > Yes, look at CompactSplitThread
> >
> >
> > >
> > > 2. If hbase uses multiple compaction threads, which configuration
> > parameter
> > > defines the number of compaction threads?
> > >
> >
> > It doesn't in 0.90.6 but CompactSplitThread lists those for 0.92+
> >
> > hbase.regionserver.thread.compaction.large
> > hbase.regionserver.thread.compaction.small
> >
> >
> > >
> > > 3. After hbase.majorcompaction.interval from last major compaction ,if
> > > major compaction is executed on a table already major compacted Does
> > hbase
> > > skip all the table regions from major compaction?
> > >
> >
> > Determining if something is major-compacted is definitely not at the
> > table-level.
> >
> > In 0.90.6, MajorCompactionChecker will ask HRegion.isMajorCompaction() to
> > check if it needs to major compact again, which in turns checks every
> > Store. FWW if you have TTL turned on it will still major compact a major
> > compacted file, HFiles don't have an index of what's deleted or TTL'd and
> > it doesn't do a full read of each files to check.
> >
> >
> > >
> > > Regards,
> > > R.Monish
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message