Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id E3DDC200BBE for ; Thu, 27 Oct 2016 21:56:42 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id E2B8F160AE6; Thu, 27 Oct 2016 19:56:42 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 03CEA160B04 for ; Thu, 27 Oct 2016 21:56:41 +0200 (CEST) Received: (qmail 42808 invoked by uid 500); 27 Oct 2016 19:56:41 -0000 Mailing-List: contact commits-help@hawq.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@hawq.incubator.apache.org Delivered-To: mailing list commits@hawq.incubator.apache.org Received: (qmail 42784 invoked by uid 99); 27 Oct 2016 19:56:41 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 27 Oct 2016 19:56:41 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id A1EA0C0B41 for ; Thu, 27 Oct 2016 19:56:41 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -6.219 X-Spam-Level: X-Spam-Status: No, score=-6.219 tagged_above=-999 required=6.31 tests=[KAM_ASCII_DIVIDERS=0.8, KAM_LAZY_DOMAIN_SECURITY=1, RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RP_MATCHES_RCVD=-2.999] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id mZLzpU4QOWZ4 for ; Thu, 27 Oct 2016 19:56:39 +0000 (UTC) Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with SMTP id D70D55FC6F for ; Thu, 27 Oct 2016 19:56:37 +0000 (UTC) Received: (qmail 42542 invoked by uid 99); 27 Oct 2016 19:56:37 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 27 Oct 2016 19:56:37 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id CBA36F0BF2; Thu, 27 Oct 2016 19:56:36 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: yozie@apache.org To: commits@hawq.incubator.apache.org Date: Thu, 27 Oct 2016 19:56:37 -0000 Message-Id: In-Reply-To: <02994af8143343e19e5e613fb4974a9e@git.apache.org> References: <02994af8143343e19e5e613fb4974a9e@git.apache.org> X-Mailer: ASF-Git Admin Mailer Subject: [2/9] incubator-hawq-docs git commit: hawq register --repair update archived-at: Thu, 27 Oct 2016 19:56:43 -0000 hawq register --repair update Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/commit/3fde706c Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/tree/3fde706c Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/diff/3fde706c Branch: refs/heads/develop Commit: 3fde706c4cf33351096ad86681b02e03073bbacf Parents: baaf05f Author: Jane Beckman Authored: Thu Oct 6 16:38:23 2016 -0700 Committer: Jane Beckman Committed: Thu Oct 6 16:38:23 2016 -0700 ---------------------------------------------------------------------- datamgmt/load/g-register_files.html.md.erb | 2 +- reference/cli/admin_utilities/hawqregister.html.md.erb | 11 ++++++++--- 2 files changed, 9 insertions(+), 4 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/3fde706c/datamgmt/load/g-register_files.html.md.erb ---------------------------------------------------------------------- diff --git a/datamgmt/load/g-register_files.html.md.erb b/datamgmt/load/g-register_files.html.md.erb index f9c407d..1d95492 100644 --- a/datamgmt/load/g-register_files.html.md.erb +++ b/datamgmt/load/g-register_files.html.md.erb @@ -77,7 +77,7 @@ Data is registered according to the following conditions: - Existing tables have files appended to the existing HAWQ table. - If a table does not exist, it is created and registered into HAWQ. The catalog table will be updated with the file size specified by the YAML file. - If the -\\\-force option is used, the data in existing catalog tables is erased and re-registered. All HDFS-related catalog contents in `pg_aoseg.pg_paqseg_$relid ` are cleared. The original files on HDFS are retained. -- The -\\\-repair option rolls data back to a specified checkpoint. If the table already exists, both the file folder and `pg_aoseg.pg_paqseg_$relid` catalog entry are folled back to the checkpoint configuration in the YAML file. Files generated after the timestamp of the checkpoint will be deleted. Hash table redistribution, table truncate, and table drop are not supported. Using the -\\\- repair option with redistributed table data will result in an error. +- If the -\\\-repair option is used, data is rolled back to a specified checkpoint. If the table already exists, both the file folder and `pg_aoseg.pg_paqseg_$relid` catalog entry are rolled back to the checkpoint configuration in the YAML file. Files generated after the timestamp of the checkpoint will be deleted. you cannot redistribute, truncate, or drop hash tables. Using the -\\\- repair option with redistributed table data will result in an error. Tables using random distribution are preferred for registering into HAWQ. If hash tables are to be registered, the distribution policy in the YAML file must match that of the table being registered into. http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/3fde706c/reference/cli/admin_utilities/hawqregister.html.md.erb ---------------------------------------------------------------------- diff --git a/reference/cli/admin_utilities/hawqregister.html.md.erb b/reference/cli/admin_utilities/hawqregister.html.md.erb index bd2907e..2f1259b 100644 --- a/reference/cli/admin_utilities/hawqregister.html.md.erb +++ b/reference/cli/admin_utilities/hawqregister.html.md.erb @@ -22,7 +22,8 @@ Connection Options: Misc. Options: [-f ] [-e ] - [--force] + [--force] + [--repair] [-c ] hawq register help | -? hawq register --version @@ -55,8 +56,8 @@ Two usage models are available. Metadata for the Parquet file(s) and the destination table must be consistent. Different data types are used by HAWQ tables and Parquet files, so the data is mapped. Refer to the section [Data Type Mapping](hawqregister.html#topic1__section7) below. You must verify that the structure of the Parquet files and the HAWQ table are compatible before running `hawq register`. ####Limitations -Only HAWQ or Hive-generated Parquet tables are supported. -Hash tables and artitioned tables are not supported in this use model. +Only HAWQ or Hive-generated Parquet tables are supported. Partitioned tables are supported, but only single-level partitioned tables can be registered. +Hash tables are not supported in this use model. ###Usage Model 2: Use information from a YAML configuration file to register data @@ -69,6 +70,7 @@ The register process behaves differently, according to different conditions. - Existing tables have files appended to the existing HAWQ table. - If a table does not exist, it is created and registered into HAWQ. - If the -\\\-force option is used, the data in existing catalog tables is erased and re-registered. +- If the -\\\-repair option is used, data is rolled back to the checkpoint configuration in the YAML file. Files generated after the timestamp of the checkpoint will be deleted. Hash table redistribution, table truncate, and table drop are not supported. ###Limitations for Registering Hive Tables to HAWQ The currently-supported data types for generating Hive tables into HAWQ tables are: boolean, int, smallint, tinyint, bigint, float, double, string, binary, char, and varchar. @@ -208,6 +210,9 @@ The following options are used with specific use models.
-F , -\\\-force
Used for disaster recovery of a cluster. Clears all HDFS-related catalog contents in `pg_aoseg.pg_paqseg_$relid `and re-registers files to a specified table. The HDFS files are not removed or modified. To use this option for recovery, data is assumed to be periodically imported to the cluster to be recovered. Used with Usage Model 2.
+
-R , -\\\-repair \
+
Rolls data back to a specified checkpoint. If the table already exists, both the file folder and `pg_aoseg.pg_paqseg_$relid` catalog entry are rolled back to the checkpoint configuration in the YAML file. Deletes any files generated after the timestamp of the checkpoint. Used with Usage Model 2.
+
-c , -\\\-config \
Registers files specified by YAML-format configuration files into HAWQ. Used with Usage Model 2.