phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PHOENIX-4287) Incorrect aggregate query results when stats are disable for parallelization
Date Thu, 02 Nov 2017 05:27:02 GMT

    [ https://issues.apache.org/jira/browse/PHOENIX-4287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16235201#comment-16235201
] 

Hadoop QA commented on PHOENIX-4287:
------------------------------------

{color:red}-1 overall{color}.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12895332/PHOENIX-4287_addendum4.patch
  against master branch at commit dc9c2fa8f92548a67a58ea495eed1011b5294fa5.
  ATTACHMENT ID: 12895332

    {color:green}+1 @author{color}.  The patch does not contain any @author tags.

    {color:red}-1 tests included{color}.  The patch doesn't appear to include any new or modified
tests.
                        Please justify why no new tests are needed for this patch.
                        Also please list what manual steps were performed to verify this patch.

    {color:green}+1 javac{color}.  The applied patch does not increase the total number of
javac compiler warnings.

    {color:green}+1 release audit{color}.  The applied patch does not increase the total number
of release audit warnings.

    {color:red}-1 lineLengths{color}.  The patch introduces the following lines longer than
100:
    +                            + " (PK1 INTEGER NOT NULL PRIMARY KEY, KV1 VARCHAR) USE_STATS_FOR_PARALLELIZATION
= false";
+            conn.createStatement().execute("ALTER TABLE " + tableName + " SET USE_STATS_FOR_PARALLELIZATION
= " + false);
+                    .execute("CREATE VIEW " + viewName + " AS SELECT * FROM " + tableName
+ " USE_STATS_FOR_PARALLELIZATION = false");
+        Boolean useStatsForParallelization = useStatsForParallelizationKv == null ? null
: Boolean.TRUE.equals(PBoolean.INSTANCE.toObject(useStatsForParallelizationKv.getValueArray(),
useStatsForParallelizationKv.getValueOffset(), useStatsForParallelizationKv.getValueLength()));
+                        result.getViewIndexId(), indexType, rowKeyOrderOptimizable, transactional,
updateCacheFrequency, 0L, isNamespaceMapped, autoPartitionSeq, isAppendOnlySchema, immutableStorageScheme,
encodingScheme, cqCounterToBe, useStatsForParallelizationProp);
+    public PTableImpl(PName tenantId, String schemaName, String tableName, long timestamp,
List<PColumnFamily> families, boolean isNamespaceMapped, ImmutableStorageScheme storageScheme,
QualifierEncodingScheme encodingScheme, Boolean useStatsForParallelization) { // For base
table of mapped VIEW
+            EncodedCQCounter encodedCQCounter, Boolean useStatsForParallelization) throws
SQLException {
+            long indexDisableTimestamp, boolean isNamespaceMapped, String autoPartitionSeqName,
boolean isAppendOnlySchema, ImmutableStorageScheme storageScheme, QualifierEncodingScheme
qualifierEncodingScheme, EncodedCQCounter encodedCQCounter, Boolean useStatsForParallelization)
throws SQLException {
+            QualifierEncodingScheme qualifierEncodingScheme, EncodedCQCounter encodedCQCounter,
Boolean useStatsForParallelization)
+            QualifierEncodingScheme qualifierEncodingScheme, EncodedCQCounter encodedCQCounter,
Boolean useStatsForParallelization) throws SQLException {

    {color:green}+1 core tests{color}.  The patch passed unit tests in .

Test results: https://builds.apache.org/job/PreCommit-PHOENIX-Build/1602//testReport/
Console output: https://builds.apache.org/job/PreCommit-PHOENIX-Build/1602//console

This message is automatically generated.

> Incorrect aggregate query results when stats are disable for parallelization
> ----------------------------------------------------------------------------
>
>                 Key: PHOENIX-4287
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-4287
>             Project: Phoenix
>          Issue Type: Bug
>    Affects Versions: 4.12.0
>         Environment: HBase 1.3.1
>            Reporter: Mujtaba Chohan
>            Assignee: Samarth Jain
>            Priority: Major
>              Labels: localIndex
>             Fix For: 4.13.0, 4.12.1
>
>         Attachments: PHOENIX-4287.patch, PHOENIX-4287_addendum.patch, PHOENIX-4287_addendum2.patch,
PHOENIX-4287_addendum3.patch, PHOENIX-4287_addendum4.patch, PHOENIX-4287_v2.patch, PHOENIX-4287_v3.patch,
PHOENIX-4287_v3_wip.patch, PHOENIX-4287_v4.patch
>
>
> With {{phoenix.use.stats.parallelization}} set to {{false}}, aggregate query returns
incorrect results when stats are available.
> With local index and stats disabled for parallelization:
> {noformat}
> explain select count(*) from TABLE_T;
> +-------------------------------------------------------------------------------------------------------+-----------------+----------------+-----------+
> |                                                 PLAN                              
                   | EST_BYTES_READ  | EST_ROWS_READ  |  EST_INFO |
> +-------------------------------------------------------------------------------------------------------+-----------------+----------------+-----------+
> | CLIENT 0-CHUNK 332170 ROWS 625043899 BYTES PARALLEL 0-WAY RANGE SCAN OVER TABLE_T [1]
 | 625043899       | 332170         | 150792825 |
> |     SERVER FILTER BY FIRST KEY ONLY                                               
                   | 625043899       | 332170         | 150792825 |
> |     SERVER AGGREGATE INTO SINGLE ROW                                              
                   | 625043899       | 332170         | 150792825 |
> +-------------------------------------------------------------------------------------------------------+-----------------+----------------+-----------+
> select count(*) from TABLE_T;
> +-----------+
> | COUNT(1)  |
> +-----------+
> | 0         |
> +-----------+
> {noformat}
> Using data table
> {noformat}
> explain select /*+NO_INDEX*/ count(*) from TABLE_T;
> +--------------------------------------------------------------------------------------------------+-----------------+----------------+----------------+
> |                                               PLAN                                
              | EST_BYTES_READ  | EST_ROWS_READ  |  EST_INFO_TS   |
> +--------------------------------------------------------------------------------------------------+-----------------+----------------+----------------+
> | CLIENT 2-CHUNK 332151 ROWS 438492470 BYTES PARALLEL 1-WAY FULL SCAN OVER TABLE_T  |
438492470       | 332151         | 1507928257617  |
> |     SERVER FILTER BY FIRST KEY ONLY                                               
              | 438492470       | 332151         | 1507928257617  |
> |     SERVER AGGREGATE INTO SINGLE ROW                                              
              | 438492470       | 332151         | 1507928257617  |
> +--------------------------------------------------------------------------------------------------+-----------------+----------------+----------------+
> select /*+NO_INDEX*/ count(*) from TABLE_T;
> +-----------+
> | COUNT(1)  |
> +-----------+
> | 14        |
> +-----------+
> {noformat}
> Without stats available, results are correct:
> {noformat}
> explain select /*+NO_INDEX*/ count(*) from TABLE_T;
> +----------------------------------------------------------------------+-----------------+----------------+--------------+
> |                                 PLAN                                 | EST_BYTES_READ
 | EST_ROWS_READ  | EST_INFO_TS  |
> +----------------------------------------------------------------------+-----------------+----------------+--------------+
> | CLIENT 2-CHUNK PARALLEL 1-WAY FULL SCAN OVER TABLE_T  | null            | null    
      | null         |
> |     SERVER FILTER BY FIRST KEY ONLY                                  | null       
    | null           | null         |
> |     SERVER AGGREGATE INTO SINGLE ROW                                 | null       
    | null           | null         |
> +----------------------------------------------------------------------+-----------------+----------------+--------------+
> select /*+NO_INDEX*/ count(*) from TABLE_T;
> +-----------+
> | COUNT(1)  |
> +-----------+
> | 333327    |
> +-----------+
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message