phoenix-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ohad Shacham (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PHOENIX-5082) Observations about the Phoenix-Omid integration
Date Thu, 27 Dec 2018 18:39:00 GMT

    [ https://issues.apache.org/jira/browse/PHOENIX-5082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16729777#comment-16729777
] 

Ohad Shacham commented on PHOENIX-5082:
---------------------------------------

Hi [~lhofhansl],

Thanks for your tests and comments, we'll go over everything and fix. 

Regarding 8. As far as I see you upserted 400K rows without committing the transaction and
then ran count from a different client.

Omid does read the shadow cells together with the cells themselves. However, when the shadow
cell does not exists, Omid needs to check the commit table. In case the transaction commit
does not exists in there as well then another search for the shadow cell needs to be done. 

This is needed to avoid cases where the post commit  of client I updated the shadow cells
after client II read it at the first time and the same post commit removed the entry from
the commit table right before client II accessed the commit table.

In order to avoid this, we will need to leave the commit entries at the commit table and
GC them once in a while. This will also require some invariant guarantee though.

6. Omid can write, we just batch these from performance issues, this can be fixed though.
We just need to guarantee that all data exists in HBase before send commit request to the
TSO.

9. seems like a bug.

 

BTW, did you check the low latency version? It does not require batch write to commit table
from the TSO and can help with the degradation you experienced.

 

Sorry for the late response, we are on vacation and return to work on Sunday.

 

Thanks!

Ohad

> Observations about the Phoenix-Omid integration
> -----------------------------------------------
>
>                 Key: PHOENIX-5082
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-5082
>             Project: Phoenix
>          Issue Type: Task
>            Reporter: Lars Hofhansl
>            Assignee: Yonatan Gottesman
>            Priority: Major
>         Attachments: CommitTS-Sample.png, Omid-profile.png, Omid-sample.png
>
>
> Filing this issue to track my observations and perhaps some fixes.
> [~ohads]
> [~jamestaylor]
> [~yonigo]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message