phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PHOENIX-3525) Cap automatic index rebuilding to inactive timestamp
Date Tue, 08 Aug 2017 03:52:00 GMT

    [ https://issues.apache.org/jira/browse/PHOENIX-3525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16117814#comment-16117814
] 

Hadoop QA commented on PHOENIX-3525:
------------------------------------

{color:red}-1 overall{color}.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12878294/PHOENIX-3525_wip2.patch
  against master branch at commit 27170bd78dc59c999d94259f14fffd93ae5e89e8.
  ATTACHMENT ID: 12878294

    {color:green}+1 @author{color}.  The patch does not contain any @author tags.

    {color:red}-1 tests included{color}.  The patch doesn't appear to include any new or modified
tests.
                        Please justify why no new tests are needed for this patch.
                        Also please list what manual steps were performed to verify this patch.

    {color:green}+1 javac{color}.  The applied patch does not increase the total number of
javac compiler warnings.

    {color:red}-1 javadoc{color}.  The javadoc tool appears to have generated 54 warning messages.

    {color:green}+1 release audit{color}.  The applied patch does not increase the total number
of release audit warnings.

    {color:red}-1 lineLengths{color}.  The patch introduces the following lines longer than
100:
    +                Cell currentRebuildUpperBoundTimestamp = currentResult.getColumnLatestCell(TABLE_FAMILY_BYTES,
REBUILD_INDEX_TIME_RANGE_UPPER_BOUND_BYTES);
+                    // We only set a new value for the upper bound when the index disable
timestamp is passed in
+                        newDisableTimeStampCell.getValueOffset(), newDisableTimeStampCell.getValueLength());
+                    if ((currentDisableTimeStamp != null && currentDisableTimeStamp.getValueLength()
> 0)) {
+                        long curDisableTimeStampVal = (Long) PLong.INSTANCE.toObject(currentDisableTimeStamp.getValueArray(),
+                            currentDisableTimeStamp.getValueOffset(), currentDisableTimeStamp.getValueLength());
+                        // We use the sign of the INDEX_DISABLE_TIMESTAMP to differentiate
the keep-index-active (negative)
+                        // from block-writes-to-data-table case. In either case, we want
to keep the oldest timestamp to
+                        // drive the partial index rebuild rather than update it with each
attempt to update the index
+                        if (curDisableTimeStampVal != 0 && Math.abs(curDisableTimeStampVal)
< Math.abs(newDisableTimeStamp)) {

     {color:red}-1 core tests{color}.  The patch failed these unit tests:
     ./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.MutableQueryIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.SortMergeJoinIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.GroupByIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.InListIT

Test results: https://builds.apache.org/job/PreCommit-PHOENIX-Build/1252//testReport/
Javadoc warnings: https://builds.apache.org/job/PreCommit-PHOENIX-Build/1252//artifact/patchprocess/patchJavadocWarnings.txt
Console output: https://builds.apache.org/job/PreCommit-PHOENIX-Build/1252//console

This message is automatically generated.

> Cap automatic index rebuilding to inactive timestamp
> ----------------------------------------------------
>
>                 Key: PHOENIX-3525
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-3525
>             Project: Phoenix
>          Issue Type: Improvement
>            Reporter: Ankit Singhal
>            Assignee: James Taylor
>             Fix For: 4.12.0, 4.11.1
>
>         Attachments: PHOENIX-3525_v1.patch, PHOENIX-3525_wip2.patch, PHOENIX-3525_wip.patch
>
>
> From [~chrajeshbabu32@gmail.com] review comment on 
> https://github.com/apache/phoenix/pull/210
> For automatic rebuilding ,DISABLED_TIMESTAMP is lower bound but there is no upper bound
so we are going rebuild all the new writes written after DISABLED_TIMESTAMP even though indexes
updated properly. So we can introduce an upper bound of time where we are going to start a
rebuild thread so we can limit the data to rebuild. In case If there are frequent writes then
we can increment the rebuild period exponentially



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message