hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jean-Daniel Cryans <jdcry...@apache.org>
Subject Re: Consistency issue when a Put is in the memstore but a more recent Delete is cleaned in a major compaction
Date Wed, 20 Mar 2013 18:01:22 GMT
On Tue, Mar 19, 2013 at 6:50 PM, lars hofhansl <larsh@apache.org> wrote:
> As long as a client application can date a Delete into the future or a Put into the past
I do not see how we can eliminate these conditions completely and not at the same time also
give up idempotent updates (which is a great property of HBase).

The bulk load case doesn't have those properties. The Delete is done
in order, but it gets into an HFile before the Put is flushed.

J-D

>
>
> ________________________________
>  From: Sergey Shelukhin <sergey@hortonworks.com>
> To: dev@hbase.apache.org
> Sent: Tuesday, March 19, 2013 6:36 PM
> Subject: Re: Consistency issue when a Put is in the memstore but a more recent Delete
is cleaned in a major compaction
>
> Yes, it is the same window of opportunity that was mentioned in some other
> JIRAs and threads (see also
> HBASE-7902<https://issues.apache.org/jira/browse/HBASE-7902>)
> and would have been made wider by dropping deletes during minor compactions
> :)
> If flush is done before major compaction, the window will just be narrowed,
> not eliminated, because you could write data during major compaction...
>
> On Tue, Mar 19, 2013 at 2:32 PM, Enis Söztutar <enis.soz@gmail.com> wrote:
>
>> I think this came up in recent discussions about the whether we get get rid
>> of delete tombstones on non-major compactions. One of the subtasks for
>> stripe compactions deals with this case.
>>
>>
>> On Tue, Mar 19, 2013 at 2:27 PM, Ted Yu <yuzhihong@gmail.com> wrote:
>>
>> > This is also related:
>> >
>> > HBASE-8086 Major compact should flush memstore firstly
>> >
>> > On Tue, Mar 19, 2013 at 2:24 PM, Ted Yu <yuzhihong@gmail.com> wrote:
>> >
>> > > Here is one related thread (on minor compaction, though) :
>> > >
>> > > Is it feasible to delete qualified tombstones during minor compaction?
>> > >
>> > >
>> > > On Tue, Mar 19, 2013 at 1:42 PM, Jean-Daniel Cryans <
>> jdcryans@apache.org
>> > >wrote:
>> > >
>> > >> Hey guys,
>> > >>
>> > >> I looked around a bit and couldn't find a jira directly related to
>> > >> this. Here's an example of inconsistency in every HBase version
>> > >> (although the shell won't let you do it in trunk):
>> > >>
>> > >> create 't', 'f'
>> > >> delete 't', '1', 'f:1', 3
>> > >> flush 't'
>> > >> put 't', '1', 'f:1', '1', 2
>> > >> scan 't'
>> > >> major_compact 't'
>> > >> scan 't'
>> > >>
>> > >> The first scan returns nothing, the second one returns the row 1. This
>> > >> is the same when the delete is bulk loaded and then major compacted
>> > >> (which is a more legitimate use case).
>> > >>
>> > >> What's the common wisdom here? Does anyone remember if we had this
>> > >> discussion in the past?
>> > >>
>> > >> Thx,
>> > >>
>> > >> J-D
>> > >>
>> > >
>> > >
>> >
>>

Mime
View raw message