phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ohad Shacham (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PHOENIX-3860) Implement TAL functionality for Omid
Date Tue, 20 Feb 2018 09:26:01 GMT

    [ https://issues.apache.org/jira/browse/PHOENIX-3860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16369840#comment-16369840
] 

Ohad Shacham commented on PHOENIX-3860:
---------------------------------------

[~jamestaylor], another issue, in addition to the one above. Omid has an internal low water
mark that is used to bound the size of the transaction manager's data structure (for conflict
detection) and also to mark the point where the garbage collector can delete entries below
(while keeping below this point only one version per key, if exists).

When a transaction tries to commit with a begin id smaller than this low water mark the transaction
manager returns abort from two reasons: First, information in the conflict detection data
structure that might conflict with this transaction might be missing. And second, the garbage
collector might deleted information that was supposed to be read by the transaction (This
hurts Omid's Opacity).

 

If during the index creation the low water exceeds the fence id then there might be a case
that the scan of the data table will miss some of the information that needs to be written
to the index.

My question is how the index creation works. Do you create an index using a transaction
and commit when you finish populate all the data table info to the index? What happened if
the commit fails? You drop the index table, take a new fence and start the process again? 

If this is the case and the index is not valid until the transaction successfully commit (which
it should be invalid) then we can do it the same with Omid and this will overcome this issue.

The only difference from regular transaction is that we will directly write the fence id to
the shadow cell in order to remove the need of saving all the mutations in the client (for
future shadow cells update). This update is OK only assuming that the index is not valid
until all the data table info is populated and that if the commit fails then the table drops
and a new one is created.

 

> Implement TAL functionality for Omid
> ------------------------------------
>
>                 Key: PHOENIX-3860
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-3860
>             Project: Phoenix
>          Issue Type: Sub-task
>            Reporter: Ohad Shacham
>            Assignee: Ohad Shacham
>            Priority: Major
>
> Implement TAL functionality for Omid in order to be able to use Omid as Phoenix's transaction
processing engine. 
> To support the integration jira [OMID-82] was opened that encapsulates all Omid related
development for Phoenix.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message