Mailing-List: contact reviews-help@impala.incubator.apache.org; run by ezmlm
Precedence: bulk
Message-Id: <201701132203.v0DM3IVt001410@ip-10-146-233-104.ec2.internal>
Date: Fri, 13 Jan 2017 22:03:17 +0000
From: "David Knupp (Code Review)" <gerrit@cloudera.org>
To: Dimitris Tsirogiannis <dtsirogiannis@cloudera.com>, impala-cr@cloudera.com,
        reviews@impala.incubator.apache.org
CC: Jim Apple <jbapple-impala@apache.org>,
        Harrison Sheinblatt <hs7@hotmail.com>
Reply-To: dknupp@cloudera.com
Subject: =?UTF-8?Q?=5BImpala-ASF-CR=5D_IMPALA-4482=3A_Use_ALTER_TABLE_/_RECOVER_PARTITIONS_when_loading_tpcds=2Estore_sales=0A?=
In-Reply-To: <gerrit.1479839424000.Iaae97d1d44201aeeacacdd39adbae35753512950@gerrit.cloudera.org>
References: <gerrit.1479839424000.Iaae97d1d44201aeeacacdd39adbae35753512950@gerrit.cloudera.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Content-Disposition: inline
User-Agent: Gerrit/2.12.2
archived-at: Fri, 13 Jan 2017 22:03:29 -0000

Hello Internal Jenkins, Dimitris Tsirogiannis,

I'd like you to reexamine a change.  Please visit

    http://gerrit.cloudera.org:8080/5177

to look at the new patch set (#5).

Change subject: IMPALA-4482: Use ALTER TABLE / RECOVER PARTITIONS when loading tpcds.store_sales
......................................................................

IMPALA-4482: Use ALTER TABLE / RECOVER PARTITIONS when loading tpcds.store_sales

This patch changes the way we load tpcds.store_sales test data. Before
this, we were relying on a force_reload to build the table partitions
based upon the data that had been copied over to HDFS from the warehouse
snapshot. This worked on the local mini-cluster, but for some reason,
it was selectively duplicating data when run on a remote cluster.

This patch doesn't solve the mystery of why data duplication occurs on
remote clusters, but it does resolve the immediate concern of loading
test data by using Impala's recover partitions feature to automatically
recognize the partitions in the HDFS directories. We just needed to add
an ALTER TABLE store_sales RECOVER PARTITIONS to the tpcds schema
template file.

Tested by dropping the tpcds table on from a remote cluster setup,
reloading the table, and running the tests in test_tpcds_queries.py.
Tests that had been failng before are now passing.

Change-Id: Iaae97d1d44201aeeacacdd39adbae35753512950
---
M testdata/datasets/tpcds/tpcds_schema_template.sql
1 file changed, 6 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/77/5177/5
-- 
To view, visit http://gerrit.cloudera.org:8080/5177
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Iaae97d1d44201aeeacacdd39adbae35753512950
Gerrit-PatchSet: 5
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: David Knupp <dknupp@cloudera.com>
Gerrit-Reviewer: David Knupp <dknupp@cloudera.com>
Gerrit-Reviewer: Dimitris Tsirogiannis <dtsirogiannis@cloudera.com>
Gerrit-Reviewer: Harrison Sheinblatt <hs7@hotmail.com>
Gerrit-Reviewer: Internal Jenkins
Gerrit-Reviewer: Jim Apple <jbapple-impala@apache.org>