phoenix-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PHOENIX-5137) Index Rebuilder scan increases data table region split time
Date Fri, 22 Feb 2019 07:52:03 GMT

    [ https://issues.apache.org/jira/browse/PHOENIX-5137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16774876#comment-16774876
] 

Hadoop QA commented on PHOENIX-5137:
------------------------------------

{color:red}-1 overall{color}.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12959716/PHOENIX-5137-4.x-HBase-1.3.01.patch
  against 4.x-HBase-1.3 branch at commit f9908c80ed8d986325867b44adf119f2d26ff783.
  ATTACHMENT ID: 12959716

    {color:green}+1 @author{color}.  The patch does not contain any @author tags.

    {color:red}-1 tests included{color}.  The patch doesn't appear to include any new or modified
tests.
                        Please justify why no new tests are needed for this patch.
                        Also please list what manual steps were performed to verify this patch.

    {color:green}+1 javac{color}.  The applied patch does not increase the total number of
javac compiler warnings.

    {color:red}-1 release audit{color}.  The applied patch generated 1 release audit warnings
(more than the master's current 0 warnings).

    {color:green}+1 lineLengths{color}.  The patch does not introduce lines longer than 100

     {color:red}-1 core tests{color}.  The patch failed these unit tests:
     ./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.trace.PhoenixTracingEndToEndIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.join.HashJoinMoreIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.CountDistinctCompressionIT

Test results: https://builds.apache.org/job/PreCommit-PHOENIX-Build/2352//testReport/
Release audit warnings: https://builds.apache.org/job/PreCommit-PHOENIX-Build/2352//artifact/patchprocess/patchReleaseAuditWarnings.txt
Console output: https://builds.apache.org/job/PreCommit-PHOENIX-Build/2352//console

This message is automatically generated.

> Index Rebuilder scan increases data table region split time
> -----------------------------------------------------------
>
>                 Key: PHOENIX-5137
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-5137
>             Project: Phoenix
>          Issue Type: Bug
>    Affects Versions: 4.14.1
>            Reporter: Kiran Kumar Maturi
>            Assignee: Kiran Kumar Maturi
>            Priority: Major
>         Attachments: PHOENIX-5137-4.14-HBase-1.3.02.patch, PHOENIX-5137-4.14-Hbase-1.3.01.patch,
PHOENIX-5137-4.14-Hbase-1.3.01.patch, PHOENIX-5137-4.x-HBase-1.3.01.patch
>
>
> [~lhofhansl] [~vincentpoon] [~tdsilva] please review
> In order to differentiate between the index rebuilder retries  (UngroupedAggregateRegionObserver.rebuildIndices())
and commits that happen in the loop of UngroupedAggregateRegionObserver.doPostScannerOpen()
as part of  PHOENIX-4600 blockingMemstoreSize was set to -1 for rebuildIndices;
> {code:java}
> commitBatchWithRetries(region, mutations, -1);{code}
> blocks the region split as the check for region closing does not happen  blockingMemstoreSize
> 0
> {code:java}
> for (int i = 0; blockingMemstoreSize > 0 && region.getMemstoreSize() >
blockingMemstoreSize && i < 30; i++) {
>   try{
>    checkForRegionClosing();
>    ....
> {code}
> Plan is to have the check for region closing at least once before committing the batch
> {code:java}
> checkForRegionClosing();
> for (int i = 0; blockingMemstoreSize > 0 && region.getMemstoreSize() >
blockingMemstoreSize && i < 30; i++) {
>   try{
>    checkForRegionClosing();
>    ....
> {code}
> Steps to reproduce 
> 1. Create a table with one index (startime) 
> 2. Add 1-2 million rows 
> 3. Wait till the index is active 
> 4. Disable the index with start time (noted in step 1) 
> 5. Once the rebuilder starts split data table region 
> Repeat the steps again after applying the patch to check the difference.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message