hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sergey Shelukhin <ser...@hortonworks.com>
Subject Re: Is it feasible to delete qualified tombstones during minor compaction?
Date Thu, 14 Mar 2013 17:46:02 GMT
Actually, major compaction still doesn't look at memstore so the window for
reappearing puts is still there, it's just small. I am +1 on the OP idea :)

On Thu, Mar 14, 2013 at 10:00 AM, lars hofhansl <larsh@apache.org> wrote:

> Unless you can guarantee that all KVs affected by a delete marker have
> been deleted you cannot remove the delete marker. That's a fact :)
> That means in order to remove a delete marker you have to look everywhere
> where a potentially affected KV could reside. Currently that is everywhere
> and hence only a major compaction can do that.
>
> Sergey is working on striped compactions where will be able (hopefully) to
> limit where we have to look for affected to something smaller than
> "everything".
>
> -- Lars
>
>
>
> ________________________________
>  From: Ted Yu <yuzhihong@gmail.com>
> To: dev@hbase.apache.org
> Sent: Thursday, March 14, 2013 9:41 AM
> Subject: Re: Is it feasible to delete qualified tombstones during minor
> compaction?
>
> That is correct.
>
> On Thu, Mar 14, 2013 at 9:28 AM, ramkrishna vasudevan <
> ramkrishna.s.vasudevan@gmail.com> wrote:
>
> > So seeing the final patch it means that whatever Liang has asked for is
> not
> > available right?
> >
> > Regards
> > Ram
> >
> > On Thu, Mar 14, 2013 at 9:55 PM, Ted Yu <yuzhihong@gmail.com> wrote:
> >
> > > The issue was HBASE-7902
> > >
> > > The final patch was just mostly renaming:
> > >
> > > -          Collections.singletonList(memstoreScanner),
> > > ScanType.MINOR_COMPACT,
> > > +          Collections.singletonList(memstoreScanner),
> > > ScanType.COMPACT_RETAIN_DELETES,
> > >
> > > Cheers
> > >
> > > On Thu, Mar 14, 2013 at 9:22 AM, Anoop John <anoop.hbase@gmail.com>
> > wrote:
> > >
> > > > Hi
> > > >          I have seen some issue raised by Sergey regarding removing
> > > Delete
> > > > markers during minor compaction. Not remembering the issue id and
> > details
> > > > :(
> > > >
> > > > -Anoop-
> > > >
> > > >
> > > > On Thu, Mar 14, 2013 at 3:11 PM, 谢良 <xieliang@xiaomi.com> wrote:
> > > >
> > > > > Currently, all tombstones only get cleaned up with major
> > compactions. I
> > > > > guess the original
> > > > > design probably considered that we need to ensure whether there are
> > > other
> > > > > versions of
> > > > > the deleted key in other storefiles which are not included in minor
> > > > > compaction? If that's
> > > > > true, i have a thought, maybe we could delete a bunch of tombstones
> > per
> > > > > bloom filter, in
> > > > > other words, we can try to query the deleted key in other
> > storefiles's
> > > > > bloom filter info to determine
> > > > > whether we can remove it safely or not.
> > > > > I am not familiar with compaction&bloomfilter internal, it's
just a
> > raw
> > > > > thought right now:)
> > > > >
> > > > > Regards,
> > > > > Liang
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message