phoenix-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Monani Mihir (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (PHOENIX-5079) PhoenixIndexMetaData.isIndexRebuild gives wrong results in Partial Index Rebuilding
Date Mon, 31 Dec 2018 10:01:00 GMT

    [ https://issues.apache.org/jira/browse/PHOENIX-5079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16731250#comment-16731250
] 

Monani Mihir edited comment on PHOENIX-5079 at 12/31/18 10:00 AM:
------------------------------------------------------------------

Checking for REPLAY_ONLY_INDEX_WRITES won't work because we are setting REPLAY_WRITES in UngroupedAggregateRegionObserver#rebuildIndices
as part of put/delete mutations, current function implementations is correct.

In Indexer.java we set BatchMutateContext in preBatchMuate#preBatchMutateWithExceptions and
during postBatchMutateIndispensably , we use same index updates to do final index Mutations.

In Indexer#postBatchMutateIndispensably we are passing BatchMutateContext for doPost() for
Index Updates ant this list of muations doesn't have value set for REPLAY_WRITES. When postBatchMutateIndispensably
is called from UngroupedAggregateRegionObserver#commitBatch , we are passing list of mutations
(miniBatchOp) which has correct value set but indexUpdates from BatchMutateContext doesn't
have that value. 
{code:java}
public void postBatchMutateIndispensably(ObserverContext<RegionCoprocessorEnvironment>
c,
MiniBatchOperationInProgress<Mutation> miniBatchOp, final boolean success) throws IOException
{
...
    BatchMutateContext context = getBatchMutateContext(c);
...
    this.builder.batchCompleted(miniBatchOp);
    LOG.info("postBatchMutateIndispensably_miniBatchOp isIndexRebuild :- " + PhoenixIndexMetaData.isIndexRebuild(miniBatchOp.getOperation(0).getAttributesMap()));
    LOG.info("postBatchMutateIndispensably_context isIndexRebuild :- " + PhoenixIndexMetaData.isIndexRebuild(context.indexUpdates.iterator().next().getFirst().getAttributesMap()));
    if (success) { // if miniBatchOp was successfully written, write index updates
...

{code}
In above log output we would get true for miniBatchOp but false for BatchMuateContext

How do we set BatchMutateContext in case of Partial Index Rebuilder?  I am not able to find
it. Can you give some pointers? 
 [~vincentpoon] [~lhofhansl] [~jamestaylor] 


was (Author: mihir6692):
Checking for REPLAY_ONLY_INDEX_WRITES won't work because we are setting REPLAY_WRITES in UngroupedAggregateRegionObserver#rebuildIndices
as part of put/delete mutations, current function implementations is correct.

In Indexer.java we set BatchMutateContext in preBatchMuate#preBatchMutateWithExceptions and
during postBatchMutateIndispensably , we use same index updates to do final index Mutations.


In Indexer#postBatchMutateIndispensably we are passing BatchMutateContext for doPost() for
Index Updates ant this list of muations doesn't have value set for REPLAY_WRITES. When postBatchMutateIndispensably
is called from UngroupedAggregateRegionObserver#commitBatch , we are passing list of mutations
(miniBatchOp) which has correct value set but indexUpdates from BatchMutateContext doesn't
have that value. 
{code:java}
public void postBatchMutateIndispensably(ObserverContext<RegionCoprocessorEnvironment>
c,
MiniBatchOperationInProgress<Mutation> miniBatchOp, final boolean success) throws IOException
{
...
    this.builder.batchCompleted(miniBatchOp);
    LOG.info("postBatchMutateIndispensably_miniBatchOp isIndexRebuild :- " + PhoenixIndexMetaData.isIndexRebuild(miniBatchOp.getOperation(0).getAttributesMap()));
    LOG.info("postBatchMutateIndispensably_context isIndexRebuild :- " + PhoenixIndexMetaData.isIndexRebuild(context.indexUpdates.iterator().next().getFirst().getAttributesMap()));
    if (success) { // if miniBatchOp was successfully written, write index updates
...

{code}
In above log output we would get true for miniBatchOp but false for BatchMuateContext


How do we set BatchMutateContext in case of Partial Index Rebuilder?  I am not able to find
it. Can you give some pointers? 
[~vincentpoon] [~lhofhansl] [~jamestaylor] 

> PhoenixIndexMetaData.isIndexRebuild gives wrong results in Partial Index Rebuilding
> -----------------------------------------------------------------------------------
>
>                 Key: PHOENIX-5079
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-5079
>             Project: Phoenix
>          Issue Type: Bug
>    Affects Versions: 4.14.1
>            Reporter: Monani Mihir
>            Priority: Minor
>
> During Partial Rebuild, If rebuilder get index failure (because of region move/split
etc), it will call PhoenixIndexFailurePolicy.handleFailure . Here it checks if mutations are
part of Index Rebuild or not . 
> {code:java}
> if (!throwing) {
>                 SQLException sqlException =
>                         new SQLExceptionInfo.Builder(SQLExceptionCode.INDEX_WRITE_FAILURE)
>                                 .setRootCause(cause).setMessage(cause.getLocalizedMessage()).build()
>                                 .buildException();
>                 IOException ioException = ServerUtil.wrapInDoNotRetryIOException(null,
sqlException, timestamp);
>             Mutation m = attempted.entries().iterator().next().getValue();
>             boolean isIndexRebuild = PhoenixIndexMetaData.isIndexRebuild(m.getAttributesMap());
>             // Always throw if rebuilding index since the rebuilder needs to know
if it was successful
>             if (throwIndexWriteFailure || isIndexRebuild) {
>             throw ioException;
>             } else {
>                     LOG.warn("Swallowing index write failure", ioException);
>             }
>             }
> {code}
> Here isIndexRebuild becomes falls even though we have set attribute for REPLY_WRITE to 
> REPLAY_ONLY_INDEX_WRITES  in UngroupedAggregateRegionObserver.rebuildIndices 
> {code:java}
> ...
> put.setAttribute(REPLAY_WRITES, REPLAY_ONLY_INDEX_WRITES);
> ...{code}
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message