Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id ECFA1200B41 for ; Thu, 7 Jul 2016 22:36:53 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id EBC90160A72; Thu, 7 Jul 2016 20:36:53 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 6E6A2160A4F for ; Thu, 7 Jul 2016 22:36:52 +0200 (CEST) Received: (qmail 8627 invoked by uid 500); 7 Jul 2016 20:36:51 -0000 Mailing-List: contact commits-help@trafodion.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: commits@trafodion.apache.org Delivered-To: mailing list commits@trafodion.apache.org Received: (qmail 8618 invoked by uid 99); 7 Jul 2016 20:36:51 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 07 Jul 2016 20:36:51 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id 310EFC0447 for ; Thu, 7 Jul 2016 20:36:51 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -4.646 X-Spam-Level: X-Spam-Status: No, score=-4.646 tagged_above=-999 required=6.31 tests=[KAM_ASCII_DIVIDERS=0.8, KAM_LAZY_DOMAIN_SECURITY=1, RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RP_MATCHES_RCVD=-1.426] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id jZ-2M6G4YAYE for ; Thu, 7 Jul 2016 20:36:47 +0000 (UTC) Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with SMTP id 001CE60D25 for ; Thu, 7 Jul 2016 20:36:45 +0000 (UTC) Received: (qmail 8262 invoked by uid 99); 7 Jul 2016 20:36:45 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 07 Jul 2016 20:36:45 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id EED55E0844; Thu, 7 Jul 2016 20:36:44 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: sandhya@apache.org To: commits@trafodion.incubator.apache.org Date: Thu, 07 Jul 2016 20:36:49 -0000 Message-Id: In-Reply-To: References: X-Mailer: ASF-Git Admin Mailer Subject: [6/7] incubator-trafodion git commit: Re-enabling executor/TEST130. Changed hive/TEST018 to add a "sleep" after each Unload statement to avoid the EAGAIN error. The real rety fix will get addresssed in JIRA 2097 archived-at: Thu, 07 Jul 2016 20:36:54 -0000 Re-enabling executor/TEST130. Changed hive/TEST018 to add a "sleep" after each Unload statement to avoid the EAGAIN error. The real rety fix will get addresssed in JIRA 2097 Project: http://git-wip-us.apache.org/repos/asf/incubator-trafodion/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-trafodion/commit/9a851e25 Tree: http://git-wip-us.apache.org/repos/asf/incubator-trafodion/tree/9a851e25 Diff: http://git-wip-us.apache.org/repos/asf/incubator-trafodion/diff/9a851e25 Branch: refs/heads/master Commit: 9a851e2591636c7f301902a657ee02752b69143e Parents: 5f99c6b Author: Sandhya Sundaresan Authored: Wed Jul 6 21:12:26 2016 +0000 Committer: Sandhya Sundaresan Committed: Wed Jul 6 21:12:26 2016 +0000 ---------------------------------------------------------------------- core/sql/regress/hive/EXPECTED018 | 198 ++++++++++++----------- core/sql/regress/hive/TEST018 | 22 +-- core/sql/regress/tools/runregr_executor.ksh | 2 +- 3 files changed, 113 insertions(+), 109 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/incubator-trafodion/blob/9a851e25/core/sql/regress/hive/EXPECTED018 ---------------------------------------------------------------------- diff --git a/core/sql/regress/hive/EXPECTED018 b/core/sql/regress/hive/EXPECTED018 index cff41c7..4228108 100644 --- a/core/sql/regress/hive/EXPECTED018 +++ b/core/sql/regress/hive/EXPECTED018 @@ -150,9 +150,9 @@ Task: CLEANUP Status: Started Object: TRAFODION.HBASE.CUSTOMER_ADDRE Task: CLEANUP Status: Ended Object: TRAFODION.HBASE.CUSTOMER_ADDRESS Task: PREPARATION Status: Started Object: TRAFODION.HBASE.CUSTOMER_ADDRESS Rows Processed: 50000 -Task: PREPARATION Status: Ended ET: 00:00:08.543 +Task: PREPARATION Status: Ended ET: 00:00:10.156 Task: COMPLETION Status: Started Object: TRAFODION.HBASE.CUSTOMER_ADDRESS -Task: COMPLETION Status: Ended ET: 00:00:00.256 +Task: COMPLETION Status: Ended ET: 00:00:00.404 --- 50000 row(s) loaded. >>-- @@ -183,7 +183,7 @@ Task: PREPARATION Status: Started Object: TRAFODION.HBASE.CUSTOMER_DEMOG Rows Processed: 20000 Task: PREPARATION Status: Ended ET: 00:00:11.022 Task: COMPLETION Status: Started Object: TRAFODION.HBASE.CUSTOMER_DEMOGRAPHICS -Task: COMPLETION Status: Ended ET: 00:00:00.196 +Task: COMPLETION Status: Ended ET: 00:00:00.279 --- 20000 row(s) loaded. >>-- @@ -213,9 +213,9 @@ Task: CLEANUP Status: Started Object: TRAFODION.HBASE.CUSTOMER_DEMOG Task: CLEANUP Status: Ended Object: TRAFODION.HBASE.CUSTOMER_DEMOGRAPHICS_SALT Task: PREPARATION Status: Started Object: TRAFODION.HBASE.CUSTOMER_DEMOGRAPHICS_SALT Rows Processed: 20000 -Task: PREPARATION Status: Ended ET: 00:00:07.143 +Task: PREPARATION Status: Ended ET: 00:00:08.435 Task: COMPLETION Status: Started Object: TRAFODION.HBASE.CUSTOMER_DEMOGRAPHICS_SALT -Task: COMPLETION Status: Ended ET: 00:00:00.177 +Task: COMPLETION Status: Ended ET: 00:00:00.266 --- 20000 row(s) loaded. >>-- @@ -235,9 +235,9 @@ Task: CLEANUP Status: Started Object: TRAFODION.HBASE.CUSTOMER_SALT Task: CLEANUP Status: Ended Object: TRAFODION.HBASE.CUSTOMER_SALT Task: PREPARATION Status: Started Object: TRAFODION.HBASE.CUSTOMER_SALT Rows Processed: 100000 -Task: PREPARATION Status: Ended ET: 00:00:07.625 +Task: PREPARATION Status: Ended ET: 00:00:10.885 Task: COMPLETION Status: Started Object: TRAFODION.HBASE.CUSTOMER_SALT -Task: COMPLETION Status: Ended ET: 00:00:00.199 +Task: COMPLETION Status: Ended ET: 00:00:00.258 --- 100000 row(s) loaded. >>-- @@ -266,9 +266,9 @@ Task: CLEANUP Status: Started Object: TRAFODION.HBASE.STORE_SALES_SA Task: CLEANUP Status: Ended Object: TRAFODION.HBASE.STORE_SALES_SALT Task: PREPARATION Status: Started Object: TRAFODION.HBASE.STORE_SALES_SALT Rows Processed: 160756 -Task: PREPARATION Status: Ended ET: 00:00:10.636 +Task: PREPARATION Status: Ended ET: 00:00:13.092 Task: COMPLETION Status: Started Object: TRAFODION.HBASE.STORE_SALES_SALT -Task: COMPLETION Status: Ended ET: 00:00:00.208 +Task: COMPLETION Status: Ended ET: 00:00:00.843 --- 160756 row(s) loaded. >>-- @@ -356,10 +356,10 @@ a +> select * from null_format_src; Task: UNLOAD Status: Started Task: EMPTY TARGET Status: Started -Task: EMPTY TARGET Status: Ended ET: 00:00:00.002 +Task: EMPTY TARGET Status: Ended ET: 00:00:00.005 Task: EXTRACT Status: Started Rows Processed: 10 -Task: EXTRACT Status: Ended ET: 00:00:00.238 +Task: EXTRACT Status: Ended ET: 00:00:00.284 --- 10 row(s) unloaded. >>select * from hive.hive.null_format_default; @@ -385,10 +385,10 @@ a +> select * from null_format_src; Task: UNLOAD Status: Started Task: EMPTY TARGET Status: Started -Task: EMPTY TARGET Status: Ended ET: 00:00:00.001 +Task: EMPTY TARGET Status: Ended ET: 00:00:00.004 Task: EXTRACT Status: Started Rows Processed: 10 -Task: EXTRACT Status: Ended ET: 00:00:00.369 +Task: EXTRACT Status: Ended ET: 00:00:00.366 --- 10 row(s) unloaded. >>select * from hive.hive.null_format_empty; @@ -414,10 +414,10 @@ a ? +> select * from null_format_src; Task: UNLOAD Status: Started Task: EMPTY TARGET Status: Started -Task: EMPTY TARGET Status: Ended ET: 00:00:00.002 +Task: EMPTY TARGET Status: Ended ET: 00:00:00.005 Task: EXTRACT Status: Started Rows Processed: 10 -Task: EXTRACT Status: Ended ET: 00:00:00.225 +Task: EXTRACT Status: Ended ET: 00:00:00.293 --- 10 row(s) unloaded. >>select * from hive.hive.null_format_colon; @@ -471,12 +471,12 @@ LC RC OP OPERATOR OPT DESCRIPTION CARD +>; Task: UNLOAD Status: Started Task: EMPTY TARGET Status: Started -Task: EMPTY TARGET Status: Ended ET: 00:00:00.002 +Task: EMPTY TARGET Status: Ended ET: 00:00:00.011 Task: EXTRACT Status: Started Rows Processed: 50000 -Task: EXTRACT Status: Ended ET: 00:00:02.759 +Task: EXTRACT Status: Ended ET: 00:00:03.450 Task: MERGE FILES Status: Started -Task: MERGE FILES Status: Ended ET: 00:00:00.027 +Task: MERGE FILES Status: Ended ET: 00:00:00.042 --- 50000 row(s) unloaded. >>log; @@ -509,12 +509,12 @@ LC RC OP OPERATOR OPT DESCRIPTION CARD +><<+ cardinality 10e10 >>; Task: UNLOAD Status: Started Task: EMPTY TARGET Status: Started -Task: EMPTY TARGET Status: Ended ET: 00:00:00.002 +Task: EMPTY TARGET Status: Ended ET: 00:00:00.005 Task: EXTRACT Status: Started Rows Processed: 20000 -Task: EXTRACT Status: Ended ET: 00:00:01.075 +Task: EXTRACT Status: Ended ET: 00:00:00.976 Task: MERGE FILES Status: Started -Task: MERGE FILES Status: Ended ET: 00:00:00.028 +Task: MERGE FILES Status: Ended ET: 00:00:00.034 --- 20000 row(s) unloaded. >>log; @@ -536,9 +536,9 @@ Task: EMPTY TARGET Status: Started Task: EMPTY TARGET Status: Ended ET: 00:00:00.005 Task: EXTRACT Status: Started Rows Processed: 20000 -Task: EXTRACT Status: Ended ET: 00:00:01.282 +Task: EXTRACT Status: Ended ET: 00:00:01.204 Task: MERGE FILES Status: Started -Task: MERGE FILES Status: Ended ET: 00:00:00.026 +Task: MERGE FILES Status: Ended ET: 00:00:00.031 --- 20000 row(s) unloaded. >>log; @@ -570,12 +570,12 @@ LC RC OP OPERATOR OPT DESCRIPTION CARD +><<+ cardinality 10e10 >>; Task: UNLOAD Status: Started Task: EMPTY TARGET Status: Started -Task: EMPTY TARGET Status: Ended ET: 00:00:00.003 +Task: EMPTY TARGET Status: Ended ET: 00:00:00.005 Task: EXTRACT Status: Started Rows Processed: 20000 -Task: EXTRACT Status: Ended ET: 00:00:00.850 +Task: EXTRACT Status: Ended ET: 00:00:01.108 Task: MERGE FILES Status: Started -Task: MERGE FILES Status: Ended ET: 00:00:00.035 +Task: MERGE FILES Status: Ended ET: 00:00:00.040 --- 20000 row(s) unloaded. >> @@ -593,12 +593,12 @@ regrhadoop.ksh fs -du -s /bulkload/customer_demographics_salt/merged_customer_de +><<+ cardinality 10e10 >>; Task: UNLOAD Status: Started Task: EMPTY TARGET Status: Started -Task: EMPTY TARGET Status: Ended ET: 00:00:00.006 +Task: EMPTY TARGET Status: Ended ET: 00:00:00.004 Task: EXTRACT Status: Started Rows Processed: 20000 -Task: EXTRACT Status: Ended ET: 00:00:00.673 +Task: EXTRACT Status: Ended ET: 00:00:00.873 Task: MERGE FILES Status: Started -Task: MERGE FILES Status: Ended ET: 00:00:00.035 +Task: MERGE FILES Status: Ended ET: 00:00:00.039 --- 20000 row(s) unloaded. >> @@ -631,10 +631,10 @@ LC RC OP OPERATOR OPT DESCRIPTION CARD +><<+ cardinality 10e10 >>; Task: UNLOAD Status: Started Task: EMPTY TARGET Status: Started -Task: EMPTY TARGET Status: Ended ET: 00:00:00.004 +Task: EMPTY TARGET Status: Ended ET: 00:00:00.007 Task: EXTRACT Status: Started Rows Processed: 20000 -Task: EXTRACT Status: Ended ET: 00:00:00.671 +Task: EXTRACT Status: Ended ET: 00:00:00.803 --- 20000 row(s) unloaded. >> @@ -657,9 +657,9 @@ Task: EMPTY TARGET Status: Started Task: EMPTY TARGET Status: Ended ET: 00:00:00.011 Task: EXTRACT Status: Started Rows Processed: 20000 -Task: EXTRACT Status: Ended ET: 00:00:00.683 +Task: EXTRACT Status: Ended ET: 00:00:00.838 Task: MERGE FILES Status: Started -Task: MERGE FILES Status: Ended ET: 00:00:00.035 +Task: MERGE FILES Status: Ended ET: 00:00:00.046 --- 20000 row(s) unloaded. >> @@ -792,12 +792,12 @@ CD_DEMO_SK CD_GENDER +><<+ cardinality 10e10 >>; Task: UNLOAD Status: Started Task: EMPTY TARGET Status: Started -Task: EMPTY TARGET Status: Ended ET: 00:00:00.003 +Task: EMPTY TARGET Status: Ended ET: 00:00:00.004 Task: EXTRACT Status: Started Rows Processed: 20000 -Task: EXTRACT Status: Ended ET: 00:00:00.596 +Task: EXTRACT Status: Ended ET: 00:00:00.837 Task: MERGE FILES Status: Started -Task: MERGE FILES Status: Ended ET: 00:00:00.030 +Task: MERGE FILES Status: Ended ET: 00:00:00.042 --- 20000 row(s) unloaded. >>log; @@ -832,15 +832,15 @@ regrhadoop.ksh fs -ls /bulkload/customer_demographics_salt/merged* | grep merge +><<+ cardinality 10e10 >>; Task: UNLOAD Status: Started Task: EMPTY TARGET Status: Started -Task: EMPTY TARGET Status: Ended ET: 00:00:00.005 +Task: EMPTY TARGET Status: Ended ET: 00:00:00.004 Task: EXTRACT Status: Started Rows Processed: 20000 -Task: EXTRACT Status: Ended ET: 00:00:00.622 +Task: EXTRACT Status: Ended ET: 00:00:00.836 Task: MERGE FILES Status: Started -Task: MERGE FILES Status: Ended ET: 00:00:00.037 +Task: MERGE FILES Status: Ended ET: 00:00:00.040 --- 20000 row(s) unloaded. ->> +>>sh sleep 10; >>select count(*) from hive.hive.unload_customer_demographics; (EXPR) @@ -890,10 +890,10 @@ Task: EMPTY TARGET Status: Started Task: EMPTY TARGET Status: Ended ET: 00:00:00.005 Task: EXTRACT Status: Started Rows Processed: 20000 -Task: EXTRACT Status: Ended ET: 00:00:00.594 +Task: EXTRACT Status: Ended ET: 00:00:00.946 --- 20000 row(s) unloaded. ->> +>>sh sleep 10; >>select count(*) from hive.hive.unload_customer_demographics; *** WARNING[8597] Statement was automatically retried 1 time(s). Delay before each retry was 0 seconds. See next entry for the error that caused this retry. @@ -941,13 +941,13 @@ CD_DEMO_SK CD_GENDER +>select * from trafodion.hbase.customer_address ; Task: UNLOAD Status: Started Task: EMPTY TARGET Status: Started -Task: EMPTY TARGET Status: Ended ET: 00:00:00.003 +Task: EMPTY TARGET Status: Ended ET: 00:00:00.007 Task: EXTRACT Status: Started Rows Processed: 50000 -Task: EXTRACT Status: Ended ET: 00:00:01.887 +Task: EXTRACT Status: Ended ET: 00:00:02.325 --- 50000 row(s) unloaded. ->> +>>sh sleep 10; >>select count(*) from hive.hive.unload_customer_address; (EXPR) @@ -994,13 +994,13 @@ CA_ADDRESS_SK CA_ADDRESS_ID +>select * from trafodion.hbase.customer_address ; Task: UNLOAD Status: Started Task: EMPTY TARGET Status: Started -Task: EMPTY TARGET Status: Ended ET: 00:00:00.004 +Task: EMPTY TARGET Status: Ended ET: 00:00:00.005 Task: EXTRACT Status: Started Rows Processed: 50000 -Task: EXTRACT Status: Ended ET: 00:00:02.001 +Task: EXTRACT Status: Ended ET: 00:00:02.438 --- 50000 row(s) unloaded. ->> +>>sh sleep 10; >>select count(*) from hive.hive.unload_customer_address; *** WARNING[8597] Statement was automatically retried 1 time(s). Delay before each retry was 0 seconds. See next entry for the error that caused this retry. @@ -1060,12 +1060,13 @@ CA_ADDRESS_SK CA_ADDRESS_ID +>select * from trafodion.hbase.customer_salt; Task: UNLOAD Status: Started Task: EMPTY TARGET Status: Started -Task: EMPTY TARGET Status: Ended ET: 00:00:00.002 +Task: EMPTY TARGET Status: Ended ET: 00:00:00.006 Task: EXTRACT Status: Started Rows Processed: 100000 -Task: EXTRACT Status: Ended ET: 00:00:06.708 +Task: EXTRACT Status: Ended ET: 00:00:07.083 --- 100000 row(s) unloaded. +>>sh sleep 10; >>select count(*) from hive.hive.unload_customer; (EXPR) @@ -1113,13 +1114,13 @@ C_CUSTOMER_SK C_CUSTOMER_ID +>select * from trafodion.hbase.customer_demographics_salt; Task: UNLOAD Status: Started Task: EMPTY TARGET Status: Started -Task: EMPTY TARGET Status: Ended ET: 00:00:00.007 +Task: EMPTY TARGET Status: Ended ET: 00:00:00.010 Task: EXTRACT Status: Started Rows Processed: 20000 -Task: EXTRACT Status: Ended ET: 00:00:01.046 +Task: EXTRACT Status: Ended ET: 00:00:01.226 --- 20000 row(s) unloaded. ->> +>>sh sleep 10; >>select count(*) from hive.hive.unload_customer_demographics; (EXPR) @@ -1166,12 +1167,12 @@ CD_DEMO_SK CD_GENDER +>select * from trafodion.hbase.customer_address where ca_address_sk < 100; Task: UNLOAD Status: Started Task: EMPTY TARGET Status: Started -Task: EMPTY TARGET Status: Ended ET: 00:00:00.004 +Task: EMPTY TARGET Status: Ended ET: 00:00:00.006 Task: EXTRACT Status: Started Rows Processed: 99 -Task: EXTRACT Status: Ended ET: 00:00:00.195 +Task: EXTRACT Status: Ended ET: 00:00:00.244 Task: MERGE FILES Status: Started -Task: MERGE FILES Status: Ended ET: 00:00:00.019 +Task: MERGE FILES Status: Ended ET: 00:00:00.024 --- 99 row(s) unloaded. >> @@ -1207,13 +1208,13 @@ regrhadoop.ksh fs -rm /user/hive/exttables/unload_customer_demographics/* +>select ss_sold_date_sk,ss_store_sk, sum (ss_quantity) from store_sales_salt group by ss_sold_date_sk ,ss_store_sk; Task: UNLOAD Status: Started Task: EMPTY TARGET Status: Started -Task: EMPTY TARGET Status: Ended ET: 00:00:00.003 +Task: EMPTY TARGET Status: Ended ET: 00:00:00.004 Task: EXTRACT Status: Started Rows Processed: 12349 -Task: EXTRACT Status: Ended ET: 00:00:06.194 +Task: EXTRACT Status: Ended ET: 00:00:07.776 --- 12349 row(s) unloaded. ->> +>>sh sleep 10; >>select [first 100] * from hive.hive.unload_store_sales_summary order by ss_sold_date_sk,ss_store_sk; SS_SOLD_DATE_SK SS_STORE_SK SS_QUANTITY @@ -1329,13 +1330,13 @@ SS_SOLD_DATE_SK SS_STORE_SK SS_QUANTITY +>select * from trafodion.hbase.customer_salt c join trafodion.hbase.customer_address ca on c.c_current_addr_sk = ca.ca_address_sk ; Task: UNLOAD Status: Started Task: EMPTY TARGET Status: Started -Task: EMPTY TARGET Status: Ended ET: 00:00:00.002 +Task: EMPTY TARGET Status: Ended ET: 00:00:00.005 Task: EXTRACT Status: Started Rows Processed: 100000 -Task: EXTRACT Status: Ended ET: 00:00:08.079 +Task: EXTRACT Status: Ended ET: 00:00:08.464 --- 100000 row(s) unloaded. ->> +>>sh sleep 10; >>select count(*) from hive.hive.unload_customer_and_address; (EXPR) @@ -1380,13 +1381,13 @@ C_CUSTOMER_SK C_CUSTOMER_ID +>select * from customer_address where ca_address_sk < 1000 union select * from customer_address where ca_address_sk > 40000 and ca_address_sk < 41000; Task: UNLOAD Status: Started Task: EMPTY TARGET Status: Started -Task: EMPTY TARGET Status: Ended ET: 00:00:00.009 +Task: EMPTY TARGET Status: Ended ET: 00:00:00.008 Task: EXTRACT Status: Started Rows Processed: 1998 -Task: EXTRACT Status: Ended ET: 00:00:00.545 +Task: EXTRACT Status: Ended ET: 00:00:00.695 --- 1998 row(s) unloaded. ->> +>>sh sleep 10; >>select count(*) from hive.hive.unload_customer_address; (EXPR) @@ -1494,7 +1495,7 @@ ESP_EXCHANGE ============================== SEQ_NO 3 ONLY CHILD 2 use_snapshot_scan ...... TRUE full_table_name ........ TRAFODION.HBASE.CUSTOMER_DEMOGRAPHICS_SALT snapshot_name .......... TRAFODION.HBASE.CUSTOMER_DEMOGRAPHICS_SALT_SNAP111 - snapshot_temp_location /bulkload/20160602095555/ + snapshot_temp_location /bulkload/20160706204232/ grep -i -e 'explain reg' -e snapshot -e full_table_name -e esp_exchange LOG018_REGULAR_SCAN_PLAN.TXT | grep -v snapshot_scan_run_id >>--no snapshot >>explain reg; @@ -1574,7 +1575,7 @@ grep -i -e 'explain snp' -e snapshot -e full_table_name -e esp_exchange LOG018_S use_snapshot_scan ...... TRUE full_table_name ........ TRAFODION.HBASE.CUSTOMER_ADDRESS snapshot_name .......... TRAFODION.HBASE.CUSTOMER_ADDRESS_SNAP111 - snapshot_temp_location /bulkload/20160602095603/ + snapshot_temp_location /bulkload/20160706204243/ grep -i -e 'explain reg' -e snapshot -e full_table_name -e esp_exchange LOG018_REGULAR_SCAN_PLAN.TXT | grep -v snapshot_scan_run_id >>--no snapshot >>explain reg; @@ -1656,11 +1657,11 @@ grep -i -e 'explain snp' -e snapshot -e full_table_name -e esp_exchange LOG018_S use_snapshot_scan ...... TRUE full_table_name ........ TRAFODION.HBASE.CUSTOMER_SALT snapshot_name .......... TRAFODION.HBASE.CUSTOMER_SALT_SNAP111 - snapshot_temp_location /bulkload/20160602095624/ + snapshot_temp_location /bulkload/20160706204314/ use_snapshot_scan ...... TRUE full_table_name ........ TRAFODION.HBASE.CUSTOMER_ADDRESS snapshot_name .......... TRAFODION.HBASE.CUSTOMER_ADDRESS_SNAP111 - snapshot_temp_location /bulkload/20160602095624/ + snapshot_temp_location /bulkload/20160706204314/ grep -i -e 'explain reg' -e snapshot -e full_table_name -e esp_exchange LOG018_REGULAR_SCAN_PLAN.TXT | grep -v snapshot_scan_run_id >>--no snapshot >>explain reg; @@ -1773,13 +1774,13 @@ LC RC OP OPERATOR OPT DESCRIPTION CARD +><<+ cardinality 10e10 >>; Task: UNLOAD Status: Started Task: EMPTY TARGET Status: Started -Task: EMPTY TARGET Status: Ended ET: 00:00:00.003 +Task: EMPTY TARGET Status: Ended ET: 00:00:00.005 Task: VERIFY SNAPSHO Status: Started Snapshots verified: 1 -Task: VERIFY SNAPSHO Status: Ended ET: 00:00:00.294 +Task: VERIFY SNAPSHO Status: Ended ET: 00:00:00.425 Task: EXTRACT Status: Started Rows Processed: 50000 -Task: EXTRACT Status: Ended ET: 00:00:02.337 +Task: EXTRACT Status: Ended ET: 00:00:02.515 --- 50000 row(s) unloaded. >> @@ -1851,13 +1852,13 @@ LC RC OP OPERATOR OPT DESCRIPTION CARD +>select * from trafodion.hbase.customer_demographics_salt <<+ cardinality 10e10 >>; Task: UNLOAD Status: Started Task: EMPTY TARGET Status: Started -Task: EMPTY TARGET Status: Ended ET: 00:00:00.002 +Task: EMPTY TARGET Status: Ended ET: 00:00:00.003 Task: VERIFY SNAPSHO Status: Started Snapshots verified: 1 -Task: VERIFY SNAPSHO Status: Ended ET: 00:00:00.272 +Task: VERIFY SNAPSHO Status: Ended ET: 00:00:00.333 Task: EXTRACT Status: Started Rows Processed: 20000 -Task: EXTRACT Status: Ended ET: 00:00:02.527 +Task: EXTRACT Status: Ended ET: 00:00:02.008 --- 20000 row(s) unloaded. >> @@ -1905,16 +1906,16 @@ CD_DEMO_SK CD_GENDER +>select * from trafodion.hbase.customer_demographics_salt <<+ cardinality 10e10 >>; Task: UNLOAD Status: Started Task: EMPTY TARGET Status: Started -Task: EMPTY TARGET Status: Ended ET: 00:00:00.006 +Task: EMPTY TARGET Status: Ended ET: 00:00:00.016 Task: CREATE SNAPSHO Status: Started Snapshots created: 1 -Task: CREATE SNAPSHO Status: Ended ET: 00:00:02.385 +Task: CREATE SNAPSHO Status: Ended ET: 00:00:00.707 Task: EXTRACT Status: Started Rows Processed: 20000 -Task: EXTRACT Status: Ended ET: 00:00:01.247 +Task: EXTRACT Status: Ended ET: 00:00:01.166 Task: DELETE SNAPSHO Status: Started Snapshots deleted: 1 -Task: DELETE SNAPSHO Status: Ended ET: 00:00:00.009 +Task: DELETE SNAPSHO Status: Ended ET: 00:00:00.026 --- 20000 row(s) unloaded. >> @@ -1966,16 +1967,16 @@ CD_DEMO_SK CD_GENDER +>select * from trafodion.hbase.customer_demographics_salt <<+ cardinality 10e10 >>; Task: UNLOAD Status: Started Task: EMPTY TARGET Status: Started -Task: EMPTY TARGET Status: Ended ET: 00:00:00.006 +Task: EMPTY TARGET Status: Ended ET: 00:00:00.009 Task: CREATE SNAPSHO Status: Started Snapshots created: 1 -Task: CREATE SNAPSHO Status: Ended ET: 00:00:00.561 +Task: CREATE SNAPSHO Status: Ended ET: 00:00:01.515 Task: EXTRACT Status: Started Rows Processed: 20000 -Task: EXTRACT Status: Ended ET: 00:00:01.201 +Task: EXTRACT Status: Ended ET: 00:00:01.263 Task: DELETE SNAPSHO Status: Started Snapshots deleted: 1 -Task: DELETE SNAPSHO Status: Ended ET: 00:00:00.004 +Task: DELETE SNAPSHO Status: Ended ET: 00:00:00.019 --- 20000 row(s) unloaded. >> @@ -2028,16 +2029,16 @@ CD_DEMO_SK CD_GENDER +>select * from customer_address where ca_address_sk < 1000 union select * from customer_address where ca_address_sk > 40000 and ca_address_sk < 41000; Task: UNLOAD Status: Started Task: EMPTY TARGET Status: Started -Task: EMPTY TARGET Status: Ended ET: 00:00:00.004 +Task: EMPTY TARGET Status: Ended ET: 00:00:00.005 Task: CREATE SNAPSHO Status: Started Snapshots created: 1 -Task: CREATE SNAPSHO Status: Ended ET: 00:00:00.958 +Task: CREATE SNAPSHO Status: Ended ET: 00:00:01.233 Task: EXTRACT Status: Started Rows Processed: 1998 -Task: EXTRACT Status: Ended ET: 00:00:00.731 +Task: EXTRACT Status: Ended ET: 00:00:00.847 Task: DELETE SNAPSHO Status: Started Snapshots deleted: 1 -Task: DELETE SNAPSHO Status: Ended ET: 00:00:00.004 +Task: DELETE SNAPSHO Status: Ended ET: 00:00:00.006 --- 1998 row(s) unloaded. >> @@ -2112,19 +2113,19 @@ CA_ADDRESS_SK CA_ADDRESS_ID +>select * from trafodion.hbase.customer_salt c join trafodion.hbase.customer_address ca on c.c_current_addr_sk = ca.ca_address_sk ; Task: UNLOAD Status: Started Task: EMPTY TARGET Status: Started -Task: EMPTY TARGET Status: Ended ET: 00:00:00.004 +Task: EMPTY TARGET Status: Ended ET: 00:00:00.048 Task: CREATE SNAPSHO Status: Started Snapshots created: 2 -Task: CREATE SNAPSHO Status: Ended ET: 00:00:02.075 +Task: CREATE SNAPSHO Status: Ended ET: 00:00:02.709 Task: EXTRACT Status: Started Rows Processed: 100000 -Task: EXTRACT Status: Ended ET: 00:00:08.534 +Task: EXTRACT Status: Ended ET: 00:00:09.569 Task: DELETE SNAPSHO Status: Started Snapshots deleted: 2 -Task: DELETE SNAPSHO Status: Ended ET: 00:00:00.007 +Task: DELETE SNAPSHO Status: Ended ET: 00:00:00.010 --- 100000 row(s) unloaded. ->> +>>sh sleep 10; >>select count(*) from hive.hive.unload_customer_and_address; (EXPR) @@ -2192,16 +2193,16 @@ Task: EMPTY TARGET Status: Started Task: EMPTY TARGET Status: Ended ET: 00:00:00.004 Task: CREATE SNAPSHO Status: Started Snapshots created: 1 -Task: CREATE SNAPSHO Status: Ended ET: 00:00:00.415 +Task: CREATE SNAPSHO Status: Ended ET: 00:00:01.410 Task: EXTRACT Status: Started Rows Processed: 100000 -Task: EXTRACT Status: Ended ET: 00:00:01.417 +Task: EXTRACT Status: Ended ET: 00:00:01.589 Task: DELETE SNAPSHO Status: Started Snapshots deleted: 1 -Task: DELETE SNAPSHO Status: Ended ET: 00:00:00.003 +Task: DELETE SNAPSHO Status: Ended ET: 00:00:00.006 --- 100000 row(s) unloaded. ->> +>>sh sleep 10; >>select count(*) from hive.hive.unload_customer_name; (EXPR) @@ -2271,7 +2272,7 @@ unload with delimiter 0 into '/bulkload/test' select * from CUSTOMER_ADDRESS; Task: UNLOAD Status: Started Task: EXTRACT Status: Started Rows Processed: 50000 -Task: EXTRACT Status: Ended ET: 00:00:01.957 +Task: EXTRACT Status: Ended ET: 00:00:02.253 --- 50000 row(s) unloaded. >>--unload 24 -- should give an error @@ -2336,9 +2337,10 @@ regrhadoop.ksh fs -rm /user/hive/exttables/unload_customer_demographics/* Task: UNLOAD Status: Started Task: EXTRACT Status: Started Rows Processed but NOT Written to Disk: 20000 -Task: EXTRACT Status: Ended ET: 00:00:00.806 +Task: EXTRACT Status: Ended ET: 00:00:00.840 --- 20000 row(s) unloaded. +>>sh sleep 10; >>select count(*) from hive.hive.unload_customer_demographics; (EXPR) http://git-wip-us.apache.org/repos/asf/incubator-trafodion/blob/9a851e25/core/sql/regress/hive/TEST018 ---------------------------------------------------------------------- diff --git a/core/sql/regress/hive/TEST018 b/core/sql/regress/hive/TEST018 index 86758e6..0932da5 100644 --- a/core/sql/regress/hive/TEST018 +++ b/core/sql/regress/hive/TEST018 @@ -446,7 +446,7 @@ MERGE FILE 'merged_customer_demographics' OVERWRITE INTO '/user/hive/exttables/unload_customer_demographics' select * from trafodion.hbase.customer_demographics_salt <<+ cardinality 10e10 >>; - +sh sleep 10; select count(*) from hive.hive.unload_customer_demographics; select [first 20] * from hive.hive.unload_customer_demographics where cd_demo_sk < 100 order by cd_demo_sk; @@ -458,7 +458,7 @@ WITH PURGEDATA FROM TARGET INTO '/user/hive/exttables/unload_customer_demographics' select * from trafodion.hbase.customer_demographics_salt <<+ cardinality 10e10 >>; - +sh sleep 10; select count(*) from hive.hive.unload_customer_demographics; select [first 20] * from hive.hive.unload_customer_demographics where cd_demo_sk <100 order by cd_demo_sk; @@ -467,7 +467,7 @@ UNLOAD WITH PURGEDATA FROM TARGET INTO '/user/hive/exttables/unload_customer_address' select * from trafodion.hbase.customer_address ; - +sh sleep 10; select count(*) from hive.hive.unload_customer_address; select [first 20] * from hive.hive.unload_customer_address where ca_address_sk <100 order by ca_address_sk; @@ -477,7 +477,7 @@ UNLOAD WITH PURGEDATA FROM TARGET DELIMITER 124 RECORD_SEPARATOR 10 INTO '/user/hive/exttables/unload_customer_address' select * from trafodion.hbase.customer_address ; - +sh sleep 10; select count(*) from hive.hive.unload_customer_address; select [first 20] * from hive.hive.unload_customer_address where ca_address_sk < 100 order by ca_address_sk; @@ -492,6 +492,7 @@ WITH PURGEDATA FROM TARGET --COMPRESSION GZIP INTO '/user/hive/exttables/unload_customer' select * from trafodion.hbase.customer_salt; +sh sleep 10; select count(*) from hive.hive.unload_customer; select [first 20] * from hive.hive.unload_customer where c_customer_sk < 100 order by c_customer_sk; @@ -502,7 +503,7 @@ WITH PURGEDATA FROM TARGET --COMPRESSION GZIP INTO '/user/hive/exttables/unload_customer_demographics' select * from trafodion.hbase.customer_demographics_salt; - +sh sleep 10; select count(*) from hive.hive.unload_customer_demographics; select [first 20] * from hive.hive.unload_customer_demographics where cd_demo_sk < 100 order by cd_demo_sk; @@ -539,7 +540,7 @@ WITH PURGEDATA FROM TARGET DELIMITER '|' RECORD_SEPARATOR '\n' INTO '/user/hive/exttables/unload_store_sales_summary' select ss_sold_date_sk,ss_store_sk, sum (ss_quantity) from store_sales_salt group by ss_sold_date_sk ,ss_store_sk; - +sh sleep 10; select [first 100] * from hive.hive.unload_store_sales_summary order by ss_sold_date_sk,ss_store_sk; --unload 18 @@ -547,7 +548,7 @@ UNLOAD WITH PURGEDATA FROM TARGET INTO '/user/hive/exttables/unload_customer_and_address' select * from trafodion.hbase.customer_salt c join trafodion.hbase.customer_address ca on c.c_current_addr_sk = ca.ca_address_sk ; - +sh sleep 10; select count(*) from hive.hive.unload_customer_and_address; select [first 20] * from hive.hive.unload_customer_and_address order by ca_address_sk,c_customer_sk; @@ -557,7 +558,7 @@ WITH PURGEDATA FROM TARGET INTO '/user/hive/exttables/unload_customer_address' select * from customer_address where ca_address_sk < 1000 union select * from customer_address where ca_address_sk > 40000 and ca_address_sk < 41000; - +sh sleep 10; select count(*) from hive.hive.unload_customer_address; select [first 20] * from hive.hive.unload_customer_address order by ca_address_sk; select [first 20] * from hive.hive.unload_customer_address order by ca_address_sk desc; @@ -694,7 +695,7 @@ WITH PURGEDATA FROM TARGET NEW SNAPSHOT HAVING SUFFIX 'SNAP' INTO '/user/hive/exttables/unload_customer_and_address' select * from trafodion.hbase.customer_salt c join trafodion.hbase.customer_address ca on c.c_current_addr_sk = ca.ca_address_sk ; - +sh sleep 10; select count(*) from hive.hive.unload_customer_and_address; select [first 20] * from hive.hive.unload_customer_and_address order by ca_address_sk,c_customer_sk; @@ -710,7 +711,7 @@ WITH PURGEDATA FROM TARGET NEW SNAPSHOT HAVING SUFFIX 'SNAP111' INTO '/user/hive/exttables/unload_customer_name' select c_first_name,c_last_name from trafodion.hbase.customer_salt; - +sh sleep 10; select count(*) from hive.hive.unload_customer_name; select [first 20] * from hive.hive.unload_customer_name order by c_first_name,c_last_name; @@ -752,6 +753,7 @@ UNLOAD WITH PURGEDATA FROM TARGET INTO '/user/hive/exttables/unload_customer_demographics' (select * from trafodion.hbase.customer_demographics_salt) ; +sh sleep 10; select count(*) from hive.hive.unload_customer_demographics; CQD TRAF_UNLOAD_SKIP_WRITING_TO_FILES reset; http://git-wip-us.apache.org/repos/asf/incubator-trafodion/blob/9a851e25/core/sql/regress/tools/runregr_executor.ksh ---------------------------------------------------------------------- diff --git a/core/sql/regress/tools/runregr_executor.ksh b/core/sql/regress/tools/runregr_executor.ksh index 07840b2..3411009 100755 --- a/core/sql/regress/tools/runregr_executor.ksh +++ b/core/sql/regress/tools/runregr_executor.ksh @@ -237,7 +237,7 @@ skippedfiles= # sbtestfiles contains the list of tests to be run in seabase mode if [ "$seabase" -ne 0 ]; then - sbtestfiles="TEST001 TEST002 TEST012 TEST013 TEST014 TEST015 TEST016 TEST020 TEST022 TEST025 TEST050 TEST063 TEST088 TEST101 TEST106 TEST107 TEST122 TEST131 TEST140" + sbtestfiles="TEST001 TEST002 TEST012 TEST013 TEST014 TEST015 TEST016 TEST020 TEST022 TEST025 TEST050 TEST063 TEST088 TEST101 TEST106 TEST107 TEST122 TEST130 TEST131 TEST140" sbprettyfiles= for i in $prettyfiles; do for j in $sbtestfiles; do