phoenix-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PHOENIX-4703) Provide an option to fully rebuild indexes asynchronously through SQL
Date Mon, 06 May 2019 20:54:00 GMT

    [ https://issues.apache.org/jira/browse/PHOENIX-4703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16834184#comment-16834184
] 

Hadoop QA commented on PHOENIX-4703:
------------------------------------

{color:red}-1 overall{color}.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12967958/PHOENIX-4703.patch
  against master branch at commit bb1327ef89fb0844094470ada74cbe5071b43a0d.
  ATTACHMENT ID: 12967958

    {color:green}+1 @author{color}.  The patch does not contain any @author tags.

    {color:red}-1 tests included{color}.  The patch doesn't appear to include any new or modified
tests.
                        Please justify why no new tests are needed for this patch.
                        Also please list what manual steps were performed to verify this patch.

    {color:green}+1 javac{color}.  The applied patch does not increase the total number of
javac compiler warnings.

    {color:red}-1 release audit{color}.  The applied patch generated 6 release audit warnings
(more than the master's current 0 warnings).

    {color:red}-1 lineLengths{color}.  The patch introduces the following lines longer than
100:
    +            DATA_TABLE_DDL = "CREATE TABLE %s (TENANT_ID VARCHAR(15) NOT NULL, ID INTEGER
NOT NULL, NAME VARCHAR"
+            + ", ZIP INTEGER, EMPLOYER VARCHAR , CONSTRAINT PK_1 PRIMARY KEY (TENANT_ID,
ID)) MULTI_TENANT=true";
+        serverProps.put(QueryServices.MAX_SERVER_METADATA_CACHE_TIME_TO_LIVE_MS_ATTRIB, Long.toString(5));
+        conn.createStatement().execute(String.format(INDEX_GLOBAL_DDL, globalIndexName, dataTableFullName));
+        ConnectionQueryServices queryServices = conn.unwrap(PhoenixConnection.class).getQueryServices();
+     * For local indexes the data is on the same row and all local indexes share the same
column family
+        conn.createStatement().execute(String.format(INDEX_LOCAL_DDL, localIndexName, dataTableFullName));
+    private static final Option DELETE_ALL_AND_REBUILD_OPTION = new Option("deleteall", "delete-all-and-rebuild",
false,
+		if (cmdLine.hasOption(PARTIAL_REBUILD_OPTION.getOpt()) && cmdLine.hasOption(DELETE_ALL_AND_REBUILD_OPTION.getOpt()))
{
+            throw new IllegalStateException(DELETE_ALL_AND_REBUILD_OPTION.getLongOpt() +
" is not compatible with "

     {color:red}-1 core tests{color}.  The patch failed these unit tests:
     ./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.join.HashJoinMoreIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.UpgradeIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.index.MutableIndexSplitForwardScanIT

Test results: https://builds.apache.org/job/PreCommit-PHOENIX-Build/2558//testReport/
Release audit warnings: https://builds.apache.org/job/PreCommit-PHOENIX-Build/2558//artifact/patchprocess/patchReleaseAuditWarnings.txt
Console output: https://builds.apache.org/job/PreCommit-PHOENIX-Build/2558//console

This message is automatically generated.

> Provide an option to fully rebuild indexes asynchronously through SQL
> ---------------------------------------------------------------------
>
>                 Key: PHOENIX-4703
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-4703
>             Project: Phoenix
>          Issue Type: Bug
>            Reporter: Vincent Poon
>            Assignee: Gokcen Iskender
>            Priority: Major
>         Attachments: PHOENIX-4703-4.x.patch, PHOENIX-4703.patch
>
>          Time Spent: 4h 50m
>  Remaining Estimate: 0h
>
> Currently if we run "ALTER INDEX ... REBUILD" , all the rows in the index are deleted
and the index is rebuilt synchronously.
> "ALTER INEX ... REBUILD ASYNC" seems to be used for the IndexTool's partial rebuild option,
rebuilding from ASYNC_REBUILD_TIMESTAMP (PHOENIX-2890)
> So it seems currently the only way to fully rebuild is the drop the index, and recreate
it.  This is burdensome as it requires have the schema DDL.
> We should have an option to fully rebuild asynchronously, that has the same semantics
as dropping and recreating the index.  A further advantage of this is we can maintain the
splits of the index table while dropping its data.  We are currently seeing issues where
rebuilding a large table via a MR job results in hotspotting due to all data regions writing
to the same index region at the start.
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message