phoenix-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PHOENIX-5048) Index Rebuilder does not handle INDEX_STATE timestamp check for all index
Date Mon, 17 Dec 2018 12:47:00 GMT

    [ https://issues.apache.org/jira/browse/PHOENIX-5048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16722950#comment-16722950
] 

Hadoop QA commented on PHOENIX-5048:
------------------------------------

{color:red}-1 overall{color}.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12952014/PHOENIX-5048.v5.patch
  against master branch at commit 1c042c25ed833945abe07958b82d5e9fb701ac89.
  ATTACHMENT ID: 12952014

    {color:green}+1 @author{color}.  The patch does not contain any @author tags.

    {color:red}-1 tests included{color}.  The patch doesn't appear to include any new or modified
tests.
                        Please justify why no new tests are needed for this patch.
                        Also please list what manual steps were performed to verify this patch.

    {color:green}+1 javac{color}.  The applied patch does not increase the total number of
javac compiler warnings.

    {color:red}-1 release audit{color}.  The applied patch generated 1 release audit warnings
(more than the master's current 0 warnings).

    {color:green}+1 lineLengths{color}.  The patch does not introduce lines longer than 100

     {color:red}-1 core tests{color}.  The patch failed these unit tests:
     ./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.index.MutableIndexSplitReverseScanIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.ConcurrentMutationsIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.index.MutableIndexSplitForwardScanIT

Test results: https://builds.apache.org/job/PreCommit-PHOENIX-Build/2216//testReport/
Release audit warnings: https://builds.apache.org/job/PreCommit-PHOENIX-Build/2216//artifact/patchprocess/patchReleaseAuditWarnings.txt
Console output: https://builds.apache.org/job/PreCommit-PHOENIX-Build/2216//console

This message is automatically generated.

> Index Rebuilder does not handle INDEX_STATE timestamp check for all index
> -------------------------------------------------------------------------
>
>                 Key: PHOENIX-5048
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-5048
>             Project: Phoenix
>          Issue Type: Improvement
>    Affects Versions: 4.14.0, 5.0.0, 4.14.1
>            Reporter: Monani Mihir
>            Assignee: Monani Mihir
>            Priority: Major
>         Attachments: PHOENIX-5048.patch, PHOENIX-5048.v2.patch, PHOENIX-5048.v3.patch,
PHOENIX-5048.v4.patch, PHOENIX-5048.v5.patch
>
>
> After rebuilder is finished for Partial Index Rebuild, It will check if Index state has
been updated after Upper bound of the scan we use in partial index Rebuild. If that happens
then it will fail Index Rebuild as Index write failure occured while we were rebuilding Index.
> {code:java}
> MetaDataEndpointImpl.java#updateIndexState()
> public void updateIndexState(RpcController controller, UpdateIndexStateRequest request,
> RpcCallback<MetaDataResponse> done) {
> ...
> // If the index status has been updated after the upper bound of the scan we use
> // to partially rebuild the index, then we need to fail the rebuild because an
> // index write failed before the rebuild was complete.
> if (actualTimestamp > expectedTimestamp) {
> builder.setReturnCode(MetaDataProtos.MutationCode.UNALLOWED_TABLE_MUTATION);
> builder.setMutationTime(EnvironmentEdgeManager.currentTimeMillis());
> done.run(builder.build());
> return;
> }
> ...
> }{code}
> After Introduction of TrackingParallelWriterIndexCommitter [PHOENIX-3815|https://issues.apache.org/jira/browse/PHOENIX-3815],
we only disable Index which get failure . Before that , in ParallelWriterIndexCommitter we
were disabling all index even if Index failure happens for one Index only. 
> Suppose Data Table has 3 index and above condition becomes true for first index , then
we won't even check for remain two Index.
> {code:java}
> MetaDataRegionObserver.java#BuildIndexScheduleTask.java#run()
> for (PTable indexPTable : indexesToPartiallyRebuild) {
> String indexTableFullName = SchemaUtil.getTableName(
> indexPTable.getSchemaName().getString(),
> indexPTable.getTableName().getString());
> if (scanEndTime == latestUpperBoundTimestamp) {
> IndexUtil.updateIndexState(conn, indexTableFullName, PIndexState.ACTIVE, 0L, latestUpperBoundTimestamp);
> batchExecutedPerTableMap.remove(dataPTable.getName());
> LOG.info("Making Index:" + indexPTable.getTableName() + " active after rebuilding");
> } else {
> // Increment timestamp so that client sees updated disable timestamp
> IndexUtil.updateIndexState(conn, indexTableFullName, indexPTable.getIndexState(), scanEndTime
* signOfDisableTimeStamp, latestUpperBoundTimestamp);
> Long noOfBatches = batchExecutedPerTableMap.get(dataPTable.getName());
> if (noOfBatches == null) {
> noOfBatches = 0l;
> }
> batchExecutedPerTableMap.put(dataPTable.getName(), ++noOfBatches);
> LOG.info("During Round-robin build: Successfully updated index disabled timestamp for
"
> + indexTableFullName + " to " + scanEndTime);
> }
> }
> {code}
>  
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message