asterixdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Young-Seok Kim <kiss...@gmail.com>
Subject Re: Upsert for metadata changes
Date Mon, 15 Feb 2016 05:26:31 GMT
Considering the comment in BTree.update() method, the caller should take
care of deleting the existing entry as well if it is required by the upper
level's semantic or need.
In your situation, the upper level's semantic or need is to capture the
update operation correctly from the transaction point of view such that the
update could be done "all or nothing" manner.
As you pointed out in the first email, this is essentially about how the
logging should be done correctly when you use BTree.update() method instead
of delete() followed by insert().
Considering this, what you want to do is actually implementing true update
or upsert instead of delete() followed by insert().
That being said, questions that you want to ask are
Q1) what is "a" logical undo operation of update or upsert operation such
that the logical undo operation correctly removes the effect of the update
or upsert operation when there is an abort request.
Q2) what is "a" logical redo operation of update or upsert operation such
that the logical redo operation correctly redo the effect of the update or
upsert operation when the operation was committed but the system failed
before the committed effect was durable.

Also, if the required logging informations for the logical undo and redo
operations are different, the existing log record format needs to be
modified to capture both informations in a single log record.
Since the current logical operations (in transaction perspective) that we
support are insert and delete (neither update nor upsert), their log
records only keep the inserted entry or deleted entry for undo/redo
operations.

Best,
Young-Seok


On Sun, Feb 14, 2016 at 3:17 PM, Mike Carey <dtabass@gmail.com> wrote:

> That's what I figured.  I think you're right!  How hard to try it and see?
> Cheers,
> Mike
>
>
> On 2/14/16 3:06 PM, Ildar Absalyamov wrote:
>
>> I am not talking about user-facing insert operator, which Abdullah
>> implemented, but rather LSMBtree-level upset, which seems to be a separate
>> index-modification operation according to org.apache.hyracks.storage.am
>> .common.ophelpers.IndexOperation
>> According to
>> https://github.com/apache/incubator-asterixdb-hyracks/blob/master/hyracks/hyracks-storage-am-btree/src/main/java/org/apache/hyracks/storage/am/btree/impls/BTree.java#L329-L331
>> <
>> https://github.com/apache/incubator-asterixdb-hyracks/blob/master/hyracks/hyracks-storage-am-btree/src/main/java/org/apache/hyracks/storage/am/btree/impls/BTree.java#L329-L331>
>> it seems that it is safe to make changes to the fields as long as they are
>> not keys, which is indeed the case with metadata changes I was talking
>> about.
>>
>> On Feb 14, 2016, at 14:49, Mike Carey <dtabass@gmail.com> wrote:
>>>
>>> An upsert would be fine, I would think; perhaps we didn't have it back
>>> when that was done.
>>> (Note that depending on which layer you are talking about, upsert is
>>> synonymous with a
>>> delete and then a (re)insert - though you are probably many layers below
>>> my world at the
>>> user level where that's the case. :-))
>>>
>>> On 2/14/16 2:47 PM, Ildar Absalyamov wrote:
>>>
>>>> A number of metadata-related changes, like creating a new index
>>>> involves several stages:
>>>> 1) Create an index with PENDING_ADD_OP
>>>> 2) Bulkload the index
>>>> 3) Delete the index with PENDING_ADD_OP and reinsert it with
>>>> PENDING_NO_OP
>>>>
>>>> The last operation causes the issue with stats collection for
>>>> particular index: sometimes the stats are already persisted before 3)
>>>> starts executing, so they are become a subject to cascade delete, hence are
>>>> lost.
>>>> I was wondering why an upset is not an option for step 3 instead of
>>>> insert-delete? Are there any complications from transaction logging
>>>> perspective?
>>>>
>>>> Best regards,
>>>> Ildar
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> Best regards,
>> Ildar
>>
>>
>>
>>
>>
>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message