Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
Received-SPF: pass (nike.apache.org: domain of billgraham@gmail.com designates
 74.125.82.41 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:reply-to:in-reply-to:references:from:date:message-id
         :subject:to:cc:content-type:content-transfer-encoding;
        b=IdICOKulYCiHyeN0u2S8/OY8A6VY9s/j14ysrmdM89MRv5HMX5aYJfZ7++7vSTgLaX
         QKiU7IDKtjki72eTw+igi9g7rZYeIJTsCWpNj+a/u2yF/eW8lkIY1CNqmXoT/SwQG4rx
         23aHodtim7oYcaJ3kWCQK/Q8VzMMSe98ohTqc=
MIME-Version: 1.0
Reply-To: billgraham@gmail.com
In-Reply-To: <AANLkTimZk32q0wcqZs2pRuOydEPuKJs2=t07VqSidRuf@mail.gmail.com>
References: <AANLkTikq1=r5A-FV2+7GVEGE7eTJ18=-WEk5jRNZU_h7@mail.gmail.com>
 <AANLkTikioEwZ1qGymh7iw92UszB66sUDJFUyn82cz7Gk@mail.gmail.com>
 <AANLkTi=ZDE7kuMFC5oXPmWuE3PgequqAAKv_hU8OOhig@mail.gmail.com>
 <AANLkTikd9KRtbFCeh8n6ekg-SGcDswTwaUzOnSvK1-jY@mail.gmail.com>
 <AANLkTinHguBB5ncFHS0pDVaZaZ44V5fbHqc6=UfT-H-q@mail.gmail.com>
 <AANLkTine-e+J1EhWM5qgrV_MfwRN55j5sj+EsXJMO2XR@mail.gmail.com>
 <AANLkTimZk32q0wcqZs2pRuOydEPuKJs2=t07VqSidRuf@mail.gmail.com>
From: Bill Graham <billgraham@gmail.com>
Date: Fri, 21 Jan 2011 15:41:55 -0800
Message-ID: <AANLkTinDhWjpgNgL14ZNDEQwcABtYJJm1XKG9fY1scBw@mail.gmail.com>
Subject: Re: delete using server's timestamp
To: Ryan Rawson <ryanobjc@gmail.com>
Cc: user@hbase.apache.org
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Thanks Ryan, that clears it up.


On Fri, Jan 21, 2011 at 3:29 PM, Ryan Rawson <ryanobjc@gmail.com> wrote:
> No, the storage model does not work like that. =A0The storage model
> revolves around the KeyValue, which is roughly:
>
> rowid/family/qualifier/timestamp/data
>
> and we store sequences of these in sorted order in HFiles.
>
> Note, we store the row with every single version of every column/cell.
>
> Therefore there is no such thing as "removing the bytes that represent
> the actual row key", they are part of every cell, and once those cells
> go away, then so does the row key.
>
> I hope this helps,
> -ryan
>
> On Fri, Jan 21, 2011 at 3:26 PM, Bill Graham <billgraham@gmail.com> wrote=
:
>> I follow the tombstone/compact/delete cycle of the column values, but
>> I'm still unclear of the row key life cycle.
>>
>> Is it that the bytes that represent the actual row key are associated
>> with and removed with each column value? Or are they removed upon
>> compaction when no column values exist for a given row key?
>>
>>
>>
>> On Fri, Jan 21, 2011 at 2:26 PM, Ryan Rawson <ryanobjc@gmail.com> wrote:
>>> Any of the deletes merely insert a 'tombstone' which doesnt delete the
>>> data immediately but does mark it so queries no longer return it.
>>>
>>> During the compactions we prune these delete values and they disappear
>>> for good. =A0(Barring other backups of course)
>>>
>>> Because of our variable length storage model, we dont store rows in
>>> particular blocks and rewrite said blocks, so notions of rows
>>> 'existing' or not, don't event apply to HBase as they do to RDBMS
>>> systems.
>>>
>>> -ryan
>>>
>>> On Fri, Jan 21, 2011 at 2:21 PM, Bill Graham <billgraham@gmail.com> wro=
te:
>>>> If you use some combination of delete requests and leave a row without
>>>> any column data will the row/rowkey still exist? I'm thinking of the
>>>> use case where you want to prune all old data, including row keys,
>>>> from a table.
>>>>
>>>>
>>>> On Fri, Jan 21, 2011 at 2:04 PM, Ryan Rawson <ryanobjc@gmail.com> wrot=
e:
>>>>> There are 3 kinds of deletes (with a 4th for win):
>>>>>
>>>>> - Delete.deleteFamily(byte [] family, [long])
>>>>> -- This removes all data from the given family before the given
>>>>> timestamp, or if none is given, System.currentTimeMillis()
>>>>> - Delete.deleteColumns(byte[] family, byte[]qualifier, [long])
>>>>> -- This removes all data from the given qualifier, before the given
>>>>> timestamp, or if none is given, System.currentTimeMillis()
>>>>> - Delete.deleteColumn(byte[]family, byte[]qualifier, [long])
>>>>> -- This removes A SINGLE VERSION at the given time, or if none is
>>>>> given, the most recent version is Get'ed and deleted.
>>>>> - Delete()
>>>>> -- Calls deleteFamily() on server side on every family.
>>>>>
>>>>> Stack is talking about the LAST delete form.
>>>>>
>>>>> I think what you want is probably deleteColumns() (plural!), or
>>>>> perhaps deleteFamily().
>>>>>
>>>>> One rarely wants to call deleteColumn(), since it removes just a
>>>>> single version, thus exposing older versions, which MAY be what you
>>>>> want, but I'm guessing probably isn't.
>>>>>
>>>>> Only the last form (deleteColumn (singlar!)) calls Get, the rest do
>>>>> not call Get and are very fast.
>>>>>
>>>>> -ryan
>>>>>
>>>>> On Fri, Jan 21, 2011 at 1:51 PM, Stack <stack@duboce.net> wrote:
>>>>>> On Fri, Jan 21, 2011 at 12:30 PM, Matt Corgan <mcorgan@hotpads.com> =
wrote:
>>>>>>> Is there a way to issue a delete using the server's current timesta=
mp? =A0I
>>>>>>> see methods using HConstants.LATEST_TIMESTAMP which is extremely ex=
pensive
>>>>>>> since it triggers a Get call.
>>>>>>
>>>>>> Yes. =A0Deleting latest version involves a Get to figure the most
>>>>>> recents timestamp. =A0And yes, in src code it says this is 'expensiv=
e'.
>>>>>> Seems like it does this lookup anything LATEST_TIMESTAMP is passed
>>>>>> whether column, columns, or family only to ensure the delete goes in
>>>>>> ahead of whatever is currently in the Store.
>>>>>>
>>>>>> St.Ack
>>>>>>
>>>>>
>>>>
>>>
>>
>