phoenix-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Monani Mihir (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PHOENIX-5080) Index becomes Active during Partial Index Rebuilder if Index Failure happens
Date Thu, 27 Dec 2018 06:52:00 GMT

    [ https://issues.apache.org/jira/browse/PHOENIX-5080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16729385#comment-16729385
] 

Monani Mihir commented on PHOENIX-5080:
---------------------------------------

{quote} I think for rebuilds (code path following rebuildIndices()), maybe we can skip updating
the index state, and that way MetadataRegionObserver can handle the state transitions.
{quote}
With checks present in MetaDataEngPointImpl, like 1) if INDEX_STATE has been changed during
rebuild 2) INDEX_STATE is INACTIVE to successfully transition into ACTIVE etc, Once rebuild
is finished index will again transit into DISABLE/PENDING_DISABLE state via MetaDataRegionObserver
and we would again run partial rebuilder. This cycle could repeat so many times. 

I think the more optimal solution would be that for each mutation until all retires are exhausted
(considering Partial Rebuilder as client) , we should not attempt to update INDEX_STATE ,
from INACTIVE to PENDING_DISABLE. 
{code:java}
PhoenixIndexFailurePolicy#doBatchWithRetries :-
while (canRetryMore(numRetry++, maxTries, canRetryUntil)) {
try {
Thread.sleep(ConnectionUtils.getPauseTime(pause, numRetry)); // HBase's exponential backoff
mutateCommand.doMutation();
// success - change the index state from PENDING_DISABLE back to ACTIVE
handleIndexWriteSuccessFromClient(iwe, connection);
return;
...{code}
In above code block when we call mutateCommand.doMutation(), we should not attempt to update
INDEX_STATE when failure occurs. Instead , once all retries are exhausted in while loop, we
should DISABLE the index directly. 

What do you think [~vincentpoon] ? 

I don't have much idea about UpsertSelect logic and how it works on server side.

 

> Index becomes Active during Partial Index Rebuilder if Index Failure happens
> ----------------------------------------------------------------------------
>
>                 Key: PHOENIX-5080
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-5080
>             Project: Phoenix
>          Issue Type: Bug
>    Affects Versions: 4.14.1
>            Reporter: Monani Mihir
>            Priority: Blocker
>
> After PHOENIX-4130 and PHOENIX-4600 , If there is Index failure during Partial Index
Rebuild, Rebuilder will try again to write Index updates. If it succeeds then it will transition
Index from INACTIVE to ACTIVE, even before Rebuilder finishes.
> Here is where it goes wrong, I think :- 
> {code:java}
> PhoenixIndexFailurePolicy.java :- 
> public static void doBatchWithRetries(MutateCommand mutateCommand,
>             IndexWriteException iwe, PhoenixConnection connection, ReadOnlyProps
config) throws IOException {
> ....
>     while (canRetryMore(numRetry++, maxTries, canRetryUntil)) {
>     ...
>         handleIndexWriteSuccessFromClient(iwe, connection);
>     ...
>     }
> }
> ....
> private static void handleIndexWriteSuccessFromClient(IndexWriteException indexWriteException,
PhoenixConnection conn) {
>         handleExceptionFromClient(indexWriteException, conn, PIndexState.ACTIVE);
> }
> {code}
>  
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message