hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From lars hofhansl <la...@apache.org>
Subject Re: Is it feasible to delete qualified tombstones during minor compaction?
Date Thu, 14 Mar 2013 17:00:34 GMT
Unless you can guarantee that all KVs affected by a delete marker have been deleted you cannot
remove the delete marker. That's a fact :)
That means in order to remove a delete marker you have to look everywhere where a potentially
affected KV could reside. Currently that is everywhere and hence only a major compaction can
do that.

Sergey is working on striped compactions where will be able (hopefully) to limit where we
have to look for affected to something smaller than "everything".

-- Lars



________________________________
 From: Ted Yu <yuzhihong@gmail.com>
To: dev@hbase.apache.org 
Sent: Thursday, March 14, 2013 9:41 AM
Subject: Re: Is it feasible to delete qualified tombstones during minor compaction?
 
That is correct.

On Thu, Mar 14, 2013 at 9:28 AM, ramkrishna vasudevan <
ramkrishna.s.vasudevan@gmail.com> wrote:

> So seeing the final patch it means that whatever Liang has asked for is not
> available right?
>
> Regards
> Ram
>
> On Thu, Mar 14, 2013 at 9:55 PM, Ted Yu <yuzhihong@gmail.com> wrote:
>
> > The issue was HBASE-7902
> >
> > The final patch was just mostly renaming:
> >
> > -          Collections.singletonList(memstoreScanner),
> > ScanType.MINOR_COMPACT,
> > +          Collections.singletonList(memstoreScanner),
> > ScanType.COMPACT_RETAIN_DELETES,
> >
> > Cheers
> >
> > On Thu, Mar 14, 2013 at 9:22 AM, Anoop John <anoop.hbase@gmail.com>
> wrote:
> >
> > > Hi
> > >          I have seen some issue raised by Sergey regarding removing
> > Delete
> > > markers during minor compaction. Not remembering the issue id and
> details
> > > :(
> > >
> > > -Anoop-
> > >
> > >
> > > On Thu, Mar 14, 2013 at 3:11 PM, 谢良 <xieliang@xiaomi.com> wrote:
> > >
> > > > Currently, all tombstones only get cleaned up with major
> compactions. I
> > > > guess the original
> > > > design probably considered that we need to ensure whether there are
> > other
> > > > versions of
> > > > the deleted key in other storefiles which are not included in minor
> > > > compaction? If that's
> > > > true, i have a thought, maybe we could delete a bunch of tombstones
> per
> > > > bloom filter, in
> > > > other words, we can try to query the deleted key in other
> storefiles's
> > > > bloom filter info to determine
> > > > whether we can remove it safely or not.
> > > > I am not familiar with compaction&bloomfilter internal, it's just
a
> raw
> > > > thought right now:)
> > > >
> > > > Regards,
> > > > Liang
> > >
> >
>
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message