phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Purtell (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (PHOENIX-3789) Execute cross region index maintenance calls in postBatchMutateIndispensably
Date Fri, 28 Apr 2017 18:12:04 GMT

    [ https://issues.apache.org/jira/browse/PHOENIX-3789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15989250#comment-15989250
] 

Andrew Purtell edited comment on PHOENIX-3789 at 4/28/17 6:11 PM:
------------------------------------------------------------------

[~jamestaylor] You had some concerns about this changing the ordering and replay of index
WALedits. 

I looked at HBase 1.3.1. We sync the log before calling either postBatchMutate or postBatchMutateIndespensibly.
Using one or the other won't change that. So whatever actions the Indexer takes as far as
creating and submitting its own WALedits will happen in the same order with respect to primary
table processing as before. It all happens after the primary table update is synced to the
log. (So, as before, use the preBatchMutate hook to create and add WALedits that should go
in at the same time.)

I don't believe advancing the MVCC read point (or not) affects log replay behavior. 

Advancing the read point makes the mutation in progress visible to scanners, and allows other
writers waiting on the MVCC queue to make progress, so that does mean Indexer work for one
primary table batch update will execute in parallel with Indexer work for another batch, where
before I think we were holding up all writers on the primary table while the Indexer completed
work for one update. This earlier behavior was not scalable.


was (Author: apurtell):
[~jamestaylor] You had some concerns about this changing the ordering and replay of index
WALedits. 

I looked at HBase 1.3.1. We sync the log before calling either postBatchMutate or postBatchMutateIndespensibly.
Using one or the other won't change that. So whatever actions the Indexer takes as far as
creating and submitting its own WALedits will happen in the same order with respect to primary
table processing as before. It all happens after the primary table update is synced to the
log.

I don't believe advancing the MVCC read point (or not) affects log replay behavior. 

Advancing the read point makes the mutation in progress visible to scanners, and allows other
writers waiting on the MVCC queue to make progress, so that does mean Indexer work for one
primary table batch update will execute in parallel with Indexer work for another batch, where
before I think we were holding up all writers on the primary table while the Indexer completed
work for one update. This earlier behavior was not scalable.

> Execute cross region index maintenance calls in postBatchMutateIndispensably
> ----------------------------------------------------------------------------
>
>                 Key: PHOENIX-3789
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-3789
>             Project: Phoenix
>          Issue Type: Bug
>            Reporter: James Taylor
>            Assignee: James Taylor
>             Fix For: 4.11.0
>
>         Attachments: PHOENIX-3789_addendum1.patch, PHOENIX-3789_addendum2.patch, PHOENIX-3789.patch,
PHOENIX-3789_v2.patch
>
>
> Making cross region server calls while the row is locked can lead to a greater chance
of resource starvation. We can use the postBatchMutateIndispensably hook instead of the postBatchMutate
call for our processing.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message