omid-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ohad Shacham (JIRA)" <>
Subject [jira] [Closed] (OMID-102) Implement visibility filter as pure HBase Filter
Date Mon, 22 Oct 2018 13:05:01 GMT


Ohad Shacham closed OMID-102.

> Implement visibility filter as pure HBase Filter
> ------------------------------------------------
>                 Key: OMID-102
>                 URL:
>             Project: Apache Omid
>          Issue Type: Sub-task
>            Reporter: James Taylor
>            Assignee: Yonatan Gottesman
>            Priority: Major
>             Fix For:
>         Attachments: bug_fix.patch
> The way Omid currently filters through it's own RegionScanner won't work the way it's
implemented (i.e. the way the filtering is done *after* the next call). The reason is that
the state of HBase filters get messed up since these filters will start to see cells that
it shouldn't (i.e. cells that would be filtered based on snapshot isolation). It cannot be worked
around by manually running filters afterwards because filters may issue seek calls which
are handled during the running of scans by HBase.
> Instead, the filtering needs to be implemented as a pure HBase filter and that filter
needs to delegate to the other, delegate filter once it's determined that the cell is visible.
See Tephra's TransactionVisibilityFilter and they way it calls the delegate filter (cellFilters)
only after it's determined that the cell is visible. You may run into TEPHRA-169 without
including the CellSkipFilter too. 
> Because it'll be easier if you see shadow cells *before* their corresponding real cells you
can prefix instead of suffix the column qualifiers to guarantee that you'd see the shadow
cells prior to the actual cells. Or you could buffer cells in your filter prior to omitting
them. Another issue would be if the shadow cells aren't found and you need to consult the
commit table - I suppose if the shadow cells are first, this logic would be easier to know
when it needs to be called.
> To reproduce, see the Phoenix unit tests FlappingTransactionIT.testInflightUpdateNotSeen()
and testInflightDeleteNotSeen().

This message was sent by Atlassian JIRA

View raw message