Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 428CE200BA4 for ; Sat, 1 Oct 2016 02:51:59 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 41259160AE7; Sat, 1 Oct 2016 00:51:59 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 5E0A0160AE5 for ; Sat, 1 Oct 2016 02:51:58 +0200 (CEST) Received: (qmail 3156 invoked by uid 500); 1 Oct 2016 00:51:57 -0000 Mailing-List: contact commits-help@hawq.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@hawq.incubator.apache.org Delivered-To: mailing list commits@hawq.incubator.apache.org Received: (qmail 3146 invoked by uid 99); 1 Oct 2016 00:51:57 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 01 Oct 2016 00:51:57 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id 148211804B5 for ; Sat, 1 Oct 2016 00:51:57 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -6.219 X-Spam-Level: X-Spam-Status: No, score=-6.219 tagged_above=-999 required=6.31 tests=[KAM_ASCII_DIVIDERS=0.8, KAM_LAZY_DOMAIN_SECURITY=1, RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RP_MATCHES_RCVD=-2.999] autolearn=disabled Received: from mx2-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id 2Ai6nBA8TewJ for ; Sat, 1 Oct 2016 00:51:55 +0000 (UTC) Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx2-lw-eu.apache.org (ASF Mail Server at mx2-lw-eu.apache.org) with SMTP id 08B285F479 for ; Sat, 1 Oct 2016 00:51:53 +0000 (UTC) Received: (qmail 2760 invoked by uid 99); 1 Oct 2016 00:51:53 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 01 Oct 2016 00:51:53 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id 01A94DFB81; Sat, 1 Oct 2016 00:51:53 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit From: yozie@apache.org To: commits@hawq.incubator.apache.org Date: Sat, 01 Oct 2016 00:51:52 -0000 Message-Id: X-Mailer: ASF-Git Admin Mailer Subject: [1/4] incubator-hawq-docs git commit: pxf install updates for new rpm packaging/dependencies archived-at: Sat, 01 Oct 2016 00:51:59 -0000 Repository: incubator-hawq-docs Updated Branches: refs/heads/master 3377f513f -> 135ab9b85 pxf install updates for new rpm packaging/dependencies Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/commit/4c585399 Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/tree/4c585399 Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/diff/4c585399 Branch: refs/heads/master Commit: 4c585399e8036f17b7fb18c296a910f206327dd0 Parents: 10cde80 Author: Lisa Owen Authored: Thu Sep 29 16:15:44 2016 -0700 Committer: Lisa Owen Committed: Fri Sep 30 13:28:00 2016 -0700 ---------------------------------------------------------------------- pxf/InstallPXFPlugins.html.md.erb | 133 ++++++++------------------------- 1 file changed, 30 insertions(+), 103 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/4c585399/pxf/InstallPXFPlugins.html.md.erb ---------------------------------------------------------------------- diff --git a/pxf/InstallPXFPlugins.html.md.erb b/pxf/InstallPXFPlugins.html.md.erb index 8d2dfe4..d638d6d 100644 --- a/pxf/InstallPXFPlugins.html.md.erb +++ b/pxf/InstallPXFPlugins.html.md.erb @@ -2,9 +2,9 @@ title: Installing PXF Plug-ins --- -This topic describes how to install the built-in PXF service plug-ins that are required to connect PXF to HDFS, Hive, and HBase. +This topic describes how to install the built-in PXF service plug-ins that are required to connect PXF to HDFS, Hive, HBase, and JSON. -**Note:** The PXF plug-ins require that you run Tomcat on the host machine. Tomcat reserves ports 8005, 8080, and 8009. If you have configured Oozie JXM reporting on a host that will run a PXF plug-in, make sure that the reporting service uses a port other than 8005. This helps to prevent port conflict errors from occurring when you start the PXF service. +**Note:** PXF requires that you run Tomcat on the host machine. Tomcat reserves ports 8005, 8080, and 8009. If you have configured Oozie JXM reporting on a host that will run PXF, make sure that the reporting service uses a port other than 8005. This helps to prevent port conflict errors from occurring when you start the PXF service. ## PXF Installation and Log File Directories @@ -19,123 +19,50 @@ Installing PXF plug-ins, regardless of method, creates directories and log files | `/var/run/pxf/catalina.pid` | PXF Tomcat container PID location | -## Installing PXF Plug-ins Using Ambari +## Installing PXF Using Ambari If you are using Ambari to install and manage your HAWQ cluster, you do *not* need to follow the manual installation steps in this topic. Installing using the Ambari web interface installs all of the necessary PXF plug-in components. -## Installing PXF Plug-ins from the Command Line +## Installing PXF from the Command Line Each PXF service plug-in resides in its own RPM. You may have built these RPMs in the Apache HAWQ open source project repository (see [PXF Build Instructions](https://github.com/apache/incubator-hawq/blob/master/pxf/README.md)), or these RPMs may have been included in a commercial product download package. -RPMs for PXF plug-ins must be installed on *each* node in your cluster. The PXF service must be restarted on each node after installation of plug-ins. +PXF software must be installed on *each* node in your cluster. -### Install Prerequisite Packages -All PXF plug-ins require that both the Tomcat and PXF service packages be installed on each node in your cluster. - -Install Tomcat: - -```shell -$ sudo rpm -i apache-tomcat-x.x.x-el6.noarch.rpm -``` - -where *x.x.x* corresponds to the version of Tomcat required by PXF. The appropriate version of Tomcat is included in the PXF RPM bundle. - -Install the PXF service: - -```shell -$ sudo rpm -i pxf-service-n.n.n-x.el6.noarch.rpm -``` - -where *n.n.n-x* corresponds to the PXF version and build number you wish to install. - -Installing the PXF service package: - - - creates a `/usr/lib/pxf-n.n.n` directory, adding a softlink from `/usr/lib/pxf` to this directory - - copies the PXF service JAR file `pxf-service-n.n.n.jar` to `/usr/lib/pxf-n.n.n/` - - creates a softlink to `pxf-service.jar` in `/usr/lib/pxf-n.n.n/` - - sets up the PXF service configuration files - - starts the PXF service - - -### Install the PXF HDFS Plug-in - -To install PXF support for HDFS, perform the following steps on *each* node in your cluster: - -1. Install the Tomcat and PXF service packages as described in the previous section. - -2. Install the PXF HDFS plug-in: +1. Install the PXF software, including Apache, the PXF service, and all PXF plug-ins: HDFS, HBase, Hive, JSON: ```shell - $ sudo rpm -i pxf-hdfs-n.n.n-x.el6.noarch.rpm + $ sudo yum install -y pxf ``` - The install copies the HDFS JAR file `pxf-hdfs-n.n.n.jar` to `/usr/lib/pxf-n.n.n/` and creates a softlink to `pxf-hdfs.jar` in that directory. - -### Install the PXF Hive Plug-in - -To install PXF support for Hive, perform the following steps on *each* node in your cluster: + Installing PXF in this manner: + * installs the required version of `apache-tomcat` + * creates a `/etc/pxf/pxf-n.n.n` directory, adding a softlink from `/etc/pxf` to this directory + * sets up the PXF service configuration files in `/etc/pxf` + * creates a `/usr/lib/pxf-n.n.n` directory, adding a softlink from `/usr/lib/pxf` to this directory + * copies the PXF service JAR file `pxf-service-n.n.n.jar` to `/usr/lib/pxf-n.n.n/` + * copies JAR files for each of the PXF plugs-ins to `/usr/lib/pxf-n.n.n/` + * creates softlinks from `pxf-xxx.jar` in `/usr/lib/pxf-n.n.n/` + * starts the PXF service -1. Install the Tomcat, PXF service, and PXF HDFS RPMs as previously described. +2. If you choose to use the HBase plug-in, perform the following configuration: -2. Install the PXF Hive plug-in: - - ```shell - $ sudo rpm -i pxf-hive-n.n.n-x.el6.noarch.rpm - ``` + 1. Add the PXF HBase plug-in JAR file to the HBase `CLASSPATH` by updating the `HBASE_CLASSPATH` environment variable setting in the HBase environment file `/etc/hbase/conf/hbase-env.sh`: - The install copies the Hive JAR file `pxf-hive-n.n.n.jar` to `/usr/lib/pxf-n.n.n` and creates a softlink to `pxf-hive.jar` in that directory. + ``` shell + export HBASE_CLASSPATH=${HBASE_CLASSPATH}:/usr/lib/pxf/pxf-hbase.jar + ``` -### Install the PXF HBase Plug-in + 3. Restart the HBase service after making this update to HBase configuration. -To install PXF support for HBase, perform the following steps on *each* node in your cluster. - -1. Install the Tomcat, PXF Service, and PXF HDFS RPMs as previously described. - -2. Install the PXF HBase plug-in: - - ```shell - $ sudo rpm -i pxf-hbase-n.n.n-x.el6.noarch.rpm - ``` - - The install copies the HBase JAR file `pxf-hbase-n.n.n.jar` to `/usr/lib/pxf-n.n.n` and creates a softlink to `pxf-hbase.jar` in that directory. - -3. Add the PXF HBase plug-in JAR file to the HBase `CLASSPATH` by updating the `HBASE_CLASSPATH` environment variable setting in the HBase environment file `/etc/hbase/conf/hbase-env.sh`: - - ```shell - export HBASE_CLASSPATH=${HBASE_CLASSPATH}:/usr/lib/pxf/pxf-hbase.jar - ``` - -4. Restart the HBase service after making this update to HBase configuration. - - If you are on the HBase Master node: - - ```shell - $ su -l hbase -c "/usr/hdp/current/hbase-master/bin/hbase-daemon.sh restart master; sleep 25" - ``` - - If you are on an HBase Region Server node: - - ```shell - $ su -l hbase -c "/usr/hdp/current/hbase-regionserver/bin/hbase-daemon.sh restart regionserver" - ``` - -### Install the PXF JSON Plug-in - -To install PXF support for JSON, perform the following steps on *each* node in your cluster: - -1. Install the Tomcat, PXF service, and PXF HDFS RPMs as previously described. - -2. Install the PXF JSON plug-in: - - ```shell - $ sudo rpm -i pxf-json-n.n.n-x.el6.noarch.rpm - ``` + On the HBase Master node: - The install copies the JSON JAR file `pxf-json-n.n.n.jar` to `/usr/lib/pxf-n.n.n` and creates a softlink to `pxf-json.jar` in that directory. + ``` shell + $ su -l hbase -c "/usr/hdp/current/hbase-master/bin/hbase-daemon. sh restart master; sleep 25" + ``` -### Restart the PXF Service -Restart PXF on *each* node after installing the PXF service and any PXF plug-ins. + On an HBase Region Server node: -``` shell -$ sudo service pxf-service restart -``` \ No newline at end of file + ```shell + $ su -l hbase -c "/usr/hdp/current/hbase-regionserver/bin/hbase-daemon.sh restart regionserver" + ```