Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 3AC03200C23 for ; Wed, 22 Feb 2017 23:11:04 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id 397C4160B49; Wed, 22 Feb 2017 22:11:04 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 320A4160B62 for ; Wed, 22 Feb 2017 23:11:03 +0100 (CET) Received: (qmail 87172 invoked by uid 500); 22 Feb 2017 22:11:02 -0000 Mailing-List: contact commits-help@hawq.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@hawq.incubator.apache.org Delivered-To: mailing list commits@hawq.incubator.apache.org Received: (qmail 87163 invoked by uid 99); 22 Feb 2017 22:11:02 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 22 Feb 2017 22:11:02 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id ED655C1471 for ; Wed, 22 Feb 2017 22:11:01 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -6.218 X-Spam-Level: X-Spam-Status: No, score=-6.218 tagged_above=-999 required=6.31 tests=[KAM_ASCII_DIVIDERS=0.8, KAM_LAZY_DOMAIN_SECURITY=1, RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RP_MATCHES_RCVD=-2.999, URIBL_BLOCKED=0.001] autolearn=disabled Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id 3dGLaSN3pfhz for ; Wed, 22 Feb 2017 22:10:59 +0000 (UTC) Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with SMTP id 9BE7C61C5C for ; Wed, 22 Feb 2017 22:10:57 +0000 (UTC) Received: (qmail 86421 invoked by uid 99); 22 Feb 2017 22:10:56 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 22 Feb 2017 22:10:56 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id F3B2CDFF46; Wed, 22 Feb 2017 22:10:55 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: yozie@apache.org To: commits@hawq.incubator.apache.org Date: Wed, 22 Feb 2017 22:11:08 -0000 Message-Id: In-Reply-To: References: X-Mailer: ASF-Git Admin Mailer Subject: [14/18] incubator-hawq-docs git commit: HAWQ-1307 Configuring HAWQ Best Practices page (closes #93) archived-at: Wed, 22 Feb 2017 22:11:04 -0000 HAWQ-1307 Configuring HAWQ Best Practices page (closes #93) Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/commit/7a7a87a6 Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/tree/7a7a87a6 Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/diff/7a7a87a6 Branch: refs/heads/release/2.1.0.0-incubating Commit: 7a7a87a6584588a36955361bab6432f9901af762 Parents: 10938ae Author: Jane Beckman Authored: Thu Feb 9 08:53:27 2017 -0800 Committer: David Yozie Committed: Thu Feb 9 08:53:27 2017 -0800 ---------------------------------------------------------------------- .../source/subnavs/apache-hawq-nav-210.erb | 1 + .../HAWQBestPracticesOverview.html.md.erb | 3 ++ .../config_hawq_bestpractices.html.md.erb | 32 +++++++++++++++++ .../operating_hawq_bestpractices.html.md.erb | 36 +++++++++----------- 4 files changed, 52 insertions(+), 20 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/7a7a87a6/book/master_middleman/source/subnavs/apache-hawq-nav-210.erb ---------------------------------------------------------------------- diff --git a/book/master_middleman/source/subnavs/apache-hawq-nav-210.erb b/book/master_middleman/source/subnavs/apache-hawq-nav-210.erb index bede889..3316cba 100644 --- a/book/master_middleman/source/subnavs/apache-hawq-nav-210.erb +++ b/book/master_middleman/source/subnavs/apache-hawq-nav-210.erb @@ -396,6 +396,7 @@
  • Best Practices
      +
    • Configuring HAWQ
    • Operating HAWQ
    • Securing HAWQ
    • Managing Resources
      • http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/7a7a87a6/markdown/bestpractices/HAWQBestPracticesOverview.html.md.erb ---------------------------------------------------------------------- diff --git a/markdown/bestpractices/HAWQBestPracticesOverview.html.md.erb b/markdown/bestpractices/HAWQBestPracticesOverview.html.md.erb index 5fa7769..ddca235 100644 --- a/markdown/bestpractices/HAWQBestPracticesOverview.html.md.erb +++ b/markdown/bestpractices/HAWQBestPracticesOverview.html.md.erb @@ -23,7 +23,10 @@ under the License. This chapter provides best practices on using the components and features that are part of a HAWQ system. +- **[Best Practices for Configuring HAWQ Parameters](../bestpractices/config_hawq_bestpractices.html)** + This topic provides best practices for configuring the parameters in `hawq-site.xml`. + - **[Best Practices for Operating HAWQ](../bestpractices/operating_hawq_bestpractices.html)** This topic provides best practices for operating HAWQ, including recommendations for stopping, starting and monitoring HAWQ. http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/7a7a87a6/markdown/bestpractices/config_hawq_bestpractices.html.md.erb ---------------------------------------------------------------------- diff --git a/markdown/bestpractices/config_hawq_bestpractices.html.md.erb b/markdown/bestpractices/config_hawq_bestpractices.html.md.erb new file mode 100644 index 0000000..e4b27b0 --- /dev/null +++ b/markdown/bestpractices/config_hawq_bestpractices.html.md.erb @@ -0,0 +1,32 @@ +--- +title: Best Practices for Configuring HAWQ Parameters +--- + + + + +The configuration parameters for HAWQ are maintained in the `$GPHOME/etc/hawq-site.xml` file. This file resides on all HAWQ instances and can be modified either via Ambari or by using the HAWQ command line interface. Use a consistent strategy (Ambari or command line interface) to maintain the contents of `hawq-site.xml`. + +For Ambari-managed clusters, always use Ambari for configuring `hawq-site.xml` parameters; restarting or reconfiguring HAWQ via Ambari will overwrite any configuration changes to `hawq-site.xml` made outside the Ambari interface. + +If you manage your cluster using command line tools instead of Ambari, use a consistent `hawq-site.xml` file to configure your entire cluster. + +**Note:** Do not configure HAWQ parameters through the `postgresql.conf` file. Use only `hawq-site.xml` to configure your HAWQ cluster. Any parameters defined in `hawq-site.xml` will overwrite parameters in `postgresql.conf`. + http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/7a7a87a6/markdown/bestpractices/operating_hawq_bestpractices.html.md.erb ---------------------------------------------------------------------- diff --git a/markdown/bestpractices/operating_hawq_bestpractices.html.md.erb b/markdown/bestpractices/operating_hawq_bestpractices.html.md.erb index b19ba25..61687df 100644 --- a/markdown/bestpractices/operating_hawq_bestpractices.html.md.erb +++ b/markdown/bestpractices/operating_hawq_bestpractices.html.md.erb @@ -23,21 +23,14 @@ under the License. This topic provides best practices for operating HAWQ, including recommendations for stopping, starting and monitoring HAWQ. -## Best Practices for Configuring HAWQ Parameters +## Best Practices Using the Command Line to Start/Stop HAWQ Cluster Members -The HAWQ configuration guc/parameters are located in `$GPHOME/etc/hawq-site.xml`. This configuration file resides on all HAWQ instances and can be modified either by the Ambari interface or the command line. +The following best practices are recommended when using `hawq start` and `hawq stop` to manage your HAWQ cluster. -If you install and manage HAWQ using Ambari, use the Ambari interface for all configuration changes. Do not use command line utilities such as `hawq config` to set or change HAWQ configuration properties for Ambari-managed clusters. Configuration changes to `hawq-site.xml` made outside the Ambari interface will be overwritten when you restart or reconfigure HAWQ using Ambari. - -If you manage your cluster using command line tools instead of Ambari, use a consistent `hawq-site.xml` file to configure your entire cluster. - -**Note:** While `postgresql.conf` still exists in HAWQ, any parameters defined in `hawq-site.xml` will overwrite configurations in `postgresql.conf`. For this reason, we recommend that you only use `hawq-site.xml` to configure your HAWQ cluster. For Ambari clusters, always use Ambari for configuring `hawq-site.xml` parameters. - -## Best Practices to Start/Stop HAWQ Cluster Members - -For best results in using `hawq start` and `hawq stop` to manage your HAWQ system, the following best practices are recommended. - -- Issue the `CHECKPOINT` command to update and flush all data files to disk and update the log file before stopping the cluster. A checkpoint ensures that, in the event of a crash, files can be restored from the checkpoint snapshot. +- Always use HAWQ management commands to start and stop HAWQ, instead of Postgres equivalents. +- Use `hawq start cluster` or `hawq restart cluster` to start the entire cluster, rather than starting the master and individual segments. +- If you do decide to use `hawq start standby|master|segment` to start nodes individually, always start the standby before the active master. Otherwise, the standby can become unsynchronized with the active master. +- When stopping a cluster, issue the `CHECKPOINT` command to update and flush all data files to disk and update the log file before stopping the cluster. A checkpoint ensures that, in the event of a crash, files can be restored from the checkpoint snapshot. - Stop the entire HAWQ system by stopping the cluster on the master host: ```shell $ hawq stop cluster @@ -70,8 +63,6 @@ For best results in using `hawq start` and `hawq stop` to manage your HAWQ syste $ hawq stop allsegments ``` -- Typically you should always use `hawq start cluster` or `hawq restart cluster` to start the cluster. If you do end up using `hawq start standby|master|segment` to start nodes individually, make sure you always start the standby before the active master. Otherwise, the standby can become unsynchronized with the active master. - ## Guidelines for Cluster Expansion This topic provides some guidelines around expanding your HAWQ cluster. @@ -84,7 +75,12 @@ There are several recommendations to keep in mind when modifying the size of you - Note that for hash distributed tables, expanding the cluster will not immediately improve performance since hash distributed tables use a fixed number of virtual segments. In order to obtain better performance with hash distributed tables, you must redistribute the table to the updated cluster by either the [ALTER TABLE](../reference/sql/ALTER-TABLE.html) or [CREATE TABLE AS](../reference/sql/CREATE-TABLE-AS.html#topic1) command. - If you are using hash tables, consider updating the `default_hash_table_bucket_number` server configuration parameter to a larger value after expanding the cluster but before redistributing the hash tables. -## Database State Monitoring Activities +## Best Practices for Monitoring a HAWQ System +The sections below provide recommendations for common monitoring tasks. + +**Note:** If your HAWQ cluster is managed through the Ambari interface, use Ambari to perform the suggested monitoring tasks. + +### Database State Monitoring Activities @@ -120,7 +116,7 @@ WHERE status <> 'u'; -## Hardware and Operating System Monitoring +### Hardware and Operating System Monitoring @@ -195,7 +191,7 @@ If transfer rates are lower than expected, consult with your data architect rega -## Data Maintenance +### Data Maintenance @@ -224,7 +220,7 @@ If transfer rates are lower than expected, consult with your data architect rega -## Database Maintenance +### Database Maintenance @@ -277,7 +273,7 @@ If transfer rates are lower than expected, consult with your data architect rega -## Patching and Upgrading +### Patching and Upgrading