phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "James Taylor (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PHOENIX-4130) Avoid server retries for mutable indexes
Date Tue, 30 Jan 2018 00:17:00 GMT

    [ https://issues.apache.org/jira/browse/PHOENIX-4130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16344249#comment-16344249
] 

James Taylor commented on PHOENIX-4130:
---------------------------------------

For MetaDataRegionObserver, given that an index may enter the PENDING_DISABLE relatively frequently
(i.e. during normal HBase life cycle events like if a split is happening), I think it should
behave slightly differently:
- if an index is PENDING_DISABLE and elapsedSinceDisable <= pendingDisableThreshold, just
continue before even checking if all table regions are online (since that check is somewhat
expensive).
- otherwise, check the table regions are online as you're doing now.
- then, after that, if the table regions are online, transition directly into and INACTIVE
status and don't continue. Since index maintenance was never stopped when we go into PENDING_DISABLE,
we don't need to wait to partially rebuild the table (we have to wait when going from DISABLE
to INACTIVE as clients need to see this new state since they've stopped index maintenance).
Make sure this transition is allowed by MetaDataEndpointImpl.updateIndexState().
{code}
+                    } else if (indexState == PIndexState.PENDING_DISABLE) {
+                        if (elapsedSinceDisable > pendingDisableThreshold) {
+                            // too long in PENDING_DISABLE - client didn't disable the index,
so we do it here
+                            IndexUtil.updateIndexState(conn, indexTableFullName, PIndexState.DISABLE,
indexDisableTimestamp);
+                        }
+                        continue;
{code}

> Avoid server retries for mutable indexes
> ----------------------------------------
>
>                 Key: PHOENIX-4130
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-4130
>             Project: Phoenix
>          Issue Type: Improvement
>            Reporter: Lars Hofhansl
>            Assignee: Vincent Poon
>            Priority: Major
>             Fix For: 4.14.0
>
>         Attachments: PHOENIX-4130.v1.master.patch, PHOENIX-4130.v2.master.patch, PHOENIX-4130.v3.master.patch,
PHOENIX-4130.v4.master.patch, PHOENIX-4130.v5.master.patch
>
>
> Had some discussions with [~jamestaylor], [~samarthjain], and [~vincentpoon], during
which I suggested that we can possibly eliminate retry loops happening at the server that
cause the handler threads to be stuck potentially for quite a while (at least multiple seconds
to ride over common scenarios like splits).
> Instead we can do the retries at the Phoenix client that.
> So:
> # The index updates are not retried on the server. (retries = 0)
> # A failed index update would set the failed index timestamp but leave the index enabled.
> # Now the handler thread is done, it throws an appropriate exception back to the client.
> # The Phoenix client can now retry. When those retries fail the index is disabled (if
the policy dictates that) and throw the exception back to its caller.
> So no more waiting is needed on the server, handler threads are freed immediately.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message