asterixdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ildar Absalyamov <ildar.absalya...@gmail.com>
Subject Re: Upsert for metadata changes
Date Mon, 15 Feb 2016 08:03:19 GMT
But the caller in this case (e.g. QueryTranslator.handleCreateIndexStatement()) does not need
to rollback to the version of the record (i.e. index) prior to update(). If anything goes
wrong during that process the whole index creation is aborted and whatever was written gets
cleaned up (undo). Index creation operation is aborted and there is no need to redo.
Does that make any difference?

> On Feb 14, 2016, at 21:26, Young-Seok Kim <kisskys@gmail.com> wrote:
> 
> Considering the comment in BTree.update() method, the caller should take
> care of deleting the existing entry as well if it is required by the upper
> level's semantic or need.
> In your situation, the upper level's semantic or need is to capture the
> update operation correctly from the transaction point of view such that the
> update could be done "all or nothing" manner.
> As you pointed out in the first email, this is essentially about how the
> logging should be done correctly when you use BTree.update() method instead
> of delete() followed by insert().
> Considering this, what you want to do is actually implementing true update
> or upsert instead of delete() followed by insert().
> That being said, questions that you want to ask are
> Q1) what is "a" logical undo operation of update or upsert operation such
> that the logical undo operation correctly removes the effect of the update
> or upsert operation when there is an abort request.
> Q2) what is "a" logical redo operation of update or upsert operation such
> that the logical redo operation correctly redo the effect of the update or
> upsert operation when the operation was committed but the system failed
> before the committed effect was durable.
> 
> Also, if the required logging informations for the logical undo and redo
> operations are different, the existing log record format needs to be
> modified to capture both informations in a single log record.
> Since the current logical operations (in transaction perspective) that we
> support are insert and delete (neither update nor upsert), their log
> records only keep the inserted entry or deleted entry for undo/redo
> operations.
> 
> Best,
> Young-Seok
> 
> 
> On Sun, Feb 14, 2016 at 3:17 PM, Mike Carey <dtabass@gmail.com> wrote:
> 
>> That's what I figured.  I think you're right!  How hard to try it and see?
>> Cheers,
>> Mike
>> 
>> 
>> On 2/14/16 3:06 PM, Ildar Absalyamov wrote:
>> 
>>> I am not talking about user-facing insert operator, which Abdullah
>>> implemented, but rather LSMBtree-level upset, which seems to be a separate
>>> index-modification operation according to org.apache.hyracks.storage.am
>>> .common.ophelpers.IndexOperation
>>> According to
>>> https://github.com/apache/incubator-asterixdb-hyracks/blob/master/hyracks/hyracks-storage-am-btree/src/main/java/org/apache/hyracks/storage/am/btree/impls/BTree.java#L329-L331
>>> <
>>> https://github.com/apache/incubator-asterixdb-hyracks/blob/master/hyracks/hyracks-storage-am-btree/src/main/java/org/apache/hyracks/storage/am/btree/impls/BTree.java#L329-L331>
>>> it seems that it is safe to make changes to the fields as long as they are
>>> not keys, which is indeed the case with metadata changes I was talking
>>> about.
>>> 
>>> On Feb 14, 2016, at 14:49, Mike Carey <dtabass@gmail.com> wrote:
>>>> 
>>>> An upsert would be fine, I would think; perhaps we didn't have it back
>>>> when that was done.
>>>> (Note that depending on which layer you are talking about, upsert is
>>>> synonymous with a
>>>> delete and then a (re)insert - though you are probably many layers below
>>>> my world at the
>>>> user level where that's the case. :-))
>>>> 
>>>> On 2/14/16 2:47 PM, Ildar Absalyamov wrote:
>>>> 
>>>>> A number of metadata-related changes, like creating a new index
>>>>> involves several stages:
>>>>> 1) Create an index with PENDING_ADD_OP
>>>>> 2) Bulkload the index
>>>>> 3) Delete the index with PENDING_ADD_OP and reinsert it with
>>>>> PENDING_NO_OP
>>>>> 
>>>>> The last operation causes the issue with stats collection for
>>>>> particular index: sometimes the stats are already persisted before 3)
>>>>> starts executing, so they are become a subject to cascade delete, hence
are
>>>>> lost.
>>>>> I was wondering why an upset is not an option for step 3 instead of
>>>>> insert-delete? Are there any complications from transaction logging
>>>>> perspective?
>>>>> 
>>>>> Best regards,
>>>>> Ildar
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> Best regards,
>>> Ildar
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>> 

Best regards,
Ildar





Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message