phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PHOENIX-4089) Prevent index from getting out of sync with data table under high concurrency
Date Thu, 17 Aug 2017 04:24:00 GMT

    [ https://issues.apache.org/jira/browse/PHOENIX-4089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16129887#comment-16129887
] 

Hadoop QA commented on PHOENIX-4089:
------------------------------------

{color:red}-1 overall{color}.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12882250/PHOENIX-4089_v4.patch
  against master branch at commit b13413614fef3cdb87233fd1543081e7198d685f.
  ATTACHMENT ID: 12882250

    {color:green}+1 @author{color}.  The patch does not contain any @author tags.

    {color:green}+1 tests included{color}.  The patch appears to include 15 new or modified
tests.

    {color:green}+1 javac{color}.  The applied patch does not increase the total number of
javac compiler warnings.

    {color:red}-1 javadoc{color}.  The javadoc tool appears to have generated 56 warning messages.

    {color:red}-1 release audit{color}.  The applied patch generated 2 release audit warnings
(more than the master's current 0 warnings).

    {color:red}-1 lineLengths{color}.  The patch introduces the following lines longer than
100:
    +        conn.createStatement().execute("CREATE TABLE " + tableName + "(k1 INTEGER NOT
NULL, k2 INTEGER NOT NULL, v1 INTEGER, CONSTRAINT pk PRIMARY KEY (k1,k2)) STORE_NULLS=true,
VERSIONS=1");
+                           conn.createStatement().execute("UPSERT INTO " + tableName + "
VALUES (" + (i % nRows) + ", 0, " + (isNull ? null : randInt) + ")");
+            conn.createStatement().execute("CREATE TABLE " + fullTableName + "(k VARCHAR
PRIMARY KEY, v VARCHAR) COLUMN_ENCODED_BYTES = 0, STORE_NULLS=true");
+            conn.createStatement().execute("CREATE INDEX " + indexName + " ON " + fullTableName
+ " (v)");
+            int count = conn.createStatement().executeUpdate("DELETE FROM " + fullIndexName
+ " WHERE \":K\"='a' AND \"0:V\"='ccc'");
+                assertEquals(e.getMessage(),"Expected data table row count to match expected:<2>
but was:<1>");
+            conn.createStatement().execute("CREATE TABLE " + fullTableName + "(k VARCHAR
PRIMARY KEY, v VARCHAR, v2 VARCHAR) COLUMN_ENCODED_BYTES = 0, STORE_NULLS=true");
+            conn.createStatement().execute("CREATE INDEX " + indexName + " ON " + fullTableName
+ " (v) INCLUDE (v2)");
+            conn.createStatement().execute("UPSERT INTO " + fullTableName + " VALUES('b','bb','0')");
+            conn.createStatement().execute("UPSERT INTO " + fullTableName + " VALUES('a','ccc','1')");

     {color:red}-1 core tests{color}.  The patch failed these unit tests:
     ./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.IndexToolForPartialBuildWithNamespaceEnabledIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.ProductMetricsIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.IndexToolForPartialBuildIT

Test results: https://builds.apache.org/job/PreCommit-PHOENIX-Build/1266//testReport/
Release audit warnings: https://builds.apache.org/job/PreCommit-PHOENIX-Build/1266//artifact/patchprocess/patchReleaseAuditWarnings.txt
Javadoc warnings: https://builds.apache.org/job/PreCommit-PHOENIX-Build/1266//artifact/patchprocess/patchJavadocWarnings.txt
Console output: https://builds.apache.org/job/PreCommit-PHOENIX-Build/1266//console

This message is automatically generated.

> Prevent index from getting out of sync with data table under high concurrency
> -----------------------------------------------------------------------------
>
>                 Key: PHOENIX-4089
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-4089
>             Project: Phoenix
>          Issue Type: Bug
>            Reporter: James Taylor
>            Assignee: James Taylor
>             Fix For: 4.12.0
>
>         Attachments: PHOENIX-4089_4.x-HBase-0.98.patch, PHOENIX-4089_4.x-HBase-0.98_v2.patch,
PHOENIX-4089_v1.patch, PHOENIX_4089_v2.patch, PHOENIX_4089_v3.patch, PHOENIX-4089_v4.patch
>
>
> Under high concurrency, we're still seeing the index get out of sync with the data table.
It seems that the particular case is when the same Put occurs with the same time stamp from
different clients, based on the locking we do, Phoenix thinks a different Put was the last
one than HBase does, leading to inconsistencies.
> The solution is to timestamp the cells on the server-side after the lock has been taken.
The new concurrent unit test passes 50x with this in place, while it otherwise fails 1/10
of the time (or more on HBase 1.3).



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message