Return-Path: X-Original-To: apmail-bigtop-commits-archive@www.apache.org Delivered-To: apmail-bigtop-commits-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 431441885A for ; Sun, 27 Mar 2016 10:56:15 +0000 (UTC) Received: (qmail 87368 invoked by uid 500); 27 Mar 2016 10:56:15 -0000 Delivered-To: apmail-bigtop-commits-archive@bigtop.apache.org Received: (qmail 87323 invoked by uid 500); 27 Mar 2016 10:56:15 -0000 Mailing-List: contact commits-help@bigtop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: bigtop-dev@bigtop.apache.org Delivered-To: mailing list commits@bigtop.apache.org Received: (qmail 87314 invoked by uid 99); 27 Mar 2016 10:56:15 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 27 Mar 2016 10:56:15 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id CCBA6DFC74; Sun, 27 Mar 2016 10:56:14 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: oflebbe@apache.org To: commits@bigtop.apache.org Message-Id: X-Mailer: ASF-Git Admin Mailer Subject: bigtop git commit: BIGTOP-2364: QFS packaging should use HADOOP_VERSION environment variable (Original Patch by Faraaz Sareshwala) Date: Sun, 27 Mar 2016 10:56:14 +0000 (UTC) Repository: bigtop Updated Branches: refs/heads/master c1af7dc91 -> bcf8b390f BIGTOP-2364: QFS packaging should use HADOOP_VERSION environment variable (Original Patch by Faraaz Sareshwala) Project: http://git-wip-us.apache.org/repos/asf/bigtop/repo Commit: http://git-wip-us.apache.org/repos/asf/bigtop/commit/bcf8b390 Tree: http://git-wip-us.apache.org/repos/asf/bigtop/tree/bcf8b390 Diff: http://git-wip-us.apache.org/repos/asf/bigtop/diff/bcf8b390 Branch: refs/heads/master Commit: bcf8b390f32d7f6baef2773ee75e8084536ea18f Parents: c1af7dc Author: Olaf Flebbe Authored: Fri Mar 25 12:48:55 2016 +0100 Committer: Olaf Flebbe Committed: Sun Mar 27 12:53:17 2016 +0200 ---------------------------------------------------------------------- .../vagrantconfig_centos-7.yaml | 2 +- bigtop-packages/src/common/qfs/do-component-build | 13 +++---------- bigtop-packages/src/common/qfs/install_qfs.sh | 16 ++++++++++------ bigtop-packages/src/deb/qfs/rules | 11 ++++------- bigtop-packages/src/rpm/qfs/SPECS/qfs.spec | 11 ++--------- 5 files changed, 20 insertions(+), 33 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/bigtop/blob/bcf8b390/bigtop-deploy/vm/vagrant-puppet-docker/vagrantconfig_centos-7.yaml ---------------------------------------------------------------------- diff --git a/bigtop-deploy/vm/vagrant-puppet-docker/vagrantconfig_centos-7.yaml b/bigtop-deploy/vm/vagrant-puppet-docker/vagrantconfig_centos-7.yaml index 49de573..92d7468 100644 --- a/bigtop-deploy/vm/vagrant-puppet-docker/vagrantconfig_centos-7.yaml +++ b/bigtop-deploy/vm/vagrant-puppet-docker/vagrantconfig_centos-7.yaml @@ -21,7 +21,7 @@ boot2docker: memory_size: "4096" number_cpus: "1" -repo: "http://bigtop-repos.s3.amazonaws.com/releases/1.1.0/centos/6/x86_64" +repo: "http://bigtop-repos.s3.amazonaws.com/releases/1.1.0/centos/7/x86_64" distro: centos components: [hadoop, yarn] namenode_ui_port: "50070" http://git-wip-us.apache.org/repos/asf/bigtop/blob/bcf8b390/bigtop-packages/src/common/qfs/do-component-build ---------------------------------------------------------------------- diff --git a/bigtop-packages/src/common/qfs/do-component-build b/bigtop-packages/src/common/qfs/do-component-build index dd269d0..3284cd3 100644 --- a/bigtop-packages/src/common/qfs/do-component-build +++ b/bigtop-packages/src/common/qfs/do-component-build @@ -15,18 +15,15 @@ # See the License for the specific language governing permissions and # limitations under the License. -# For debugging: -# set -ex +set -ex -# Autodetect JAVA_HOME if not defined -source /usr/lib/bigtop-utils/bigtop-detect-javahome +. `dirname $0`/bigtop.bom usage() { echo " usage: $0 Required not-so-options: --qfs-version=VERSION version of qfs we are installing - --hadoop-version=VERSION version of hadoop qfs built against --python=PATH path to python Optional options: @@ -38,7 +35,6 @@ usage: $0 OPTS=$(getopt \ -n $0 \ -o '' \ - -l 'hadoop-version:' \ -l 'qfs-version:' \ -l 'python:' \ -l 'python3:' \ @@ -51,9 +47,6 @@ fi eval set -- "$OPTS" while true ; do case "$1" in - --hadoop-version) - HADOOP_VERSION=$2 ; shift 2 - ;; --qfs-version) QFS_VERSION=$2 ; shift 2 ;; @@ -74,7 +67,7 @@ while true ; do esac done -for var in QFS_VERSION HADOOP_VERSION PYTHON_PATH ; do +for var in QFS_VERSION PYTHON_PATH ; do if [ -z "$(eval "echo \$$var")" ]; then echo Missing required param: $var usage http://git-wip-us.apache.org/repos/asf/bigtop/blob/bcf8b390/bigtop-packages/src/common/qfs/install_qfs.sh ---------------------------------------------------------------------- diff --git a/bigtop-packages/src/common/qfs/install_qfs.sh b/bigtop-packages/src/common/qfs/install_qfs.sh index 10ef537..6e7eb7d 100644 --- a/bigtop-packages/src/common/qfs/install_qfs.sh +++ b/bigtop-packages/src/common/qfs/install_qfs.sh @@ -22,8 +22,8 @@ usage() { usage: $0 Required not-so-options: --prefix=PREFIX path to install into + --source-dir=DIR path to package shared files dir --qfs-version=VERSION version of qfs we are installing - --hadoop-version=VERSION version of hadoop qfs built against --python=PYTHON the path to python Optional options: @@ -46,6 +46,7 @@ usage: $0 OPTS=$(getopt \ -n $0 \ -o '' \ + -l 'source-dir:' \ -l 'prefix:' \ -l 'bin-dir:' \ -l 'lib-dir:' \ @@ -55,7 +56,6 @@ OPTS=$(getopt \ -l 'var-dir:' \ -l 'hadoop-home:' \ -l 'qfs-version:' \ - -l 'hadoop-version:' \ -l 'python:' \ -l 'python3:' \ -l 'python-extra:' \ @@ -68,6 +68,9 @@ fi eval set -- "$OPTS" while true ; do case "$1" in + --source-dir) + SOURCE_DIR=$2 ; shift 2 + ;; --prefix) PREFIX=$2 ; shift 2 ;; @@ -95,9 +98,6 @@ while true ; do --qfs-version) QFS_VERSION=$2 ; shift 2 ;; - --hadoop-version) - HADOOP_VERSION=$2 ; shift 2 - ;; --python) PYTHON_PATH=$2 ; shift 2 ;; @@ -118,13 +118,17 @@ while true ; do esac done -for var in PREFIX QFS_VERSION HADOOP_VERSION PYTHON_PATH ; do +for var in SOURCE_DIR PREFIX QFS_VERSION PYTHON_PATH ; do if [ -z "$(eval "echo \$$var")" ]; then echo Missing required param: $var usage fi done +if [ -f "$SOURCE_DIR/bigtop.bom" ]; then + . $SOURCE_DIR/bigtop.bom +fi + PREFIX=$(readlink -f $PREFIX) BIN_DIR=$PREFIX/${BIN_DIR:-/usr/bin/qfs} LIB_DIR=$PREFIX/${LIB_DIR:-/usr/lib/qfs} http://git-wip-us.apache.org/repos/asf/bigtop/blob/bcf8b390/bigtop-packages/src/deb/qfs/rules ---------------------------------------------------------------------- diff --git a/bigtop-packages/src/deb/qfs/rules b/bigtop-packages/src/deb/qfs/rules index 56743ea..59d78ea 100755 --- a/bigtop-packages/src/deb/qfs/rules +++ b/bigtop-packages/src/deb/qfs/rules @@ -25,10 +25,10 @@ export DH_OPTIONS # Build configuration -- override with environment variables SHELL := /bin/bash -HADOOP_VERSION ?= 1.0.4 HADOOP_HOME ?= /usr/lib/hadoop PYTHON_PATH ?= $(shell command -v python) -PYTHON3_PATH ?= $(shell command -v python3) +# This will pick up a python3 dev environment, which is not yet provided by bigtop +# PYTHON3_PATH ?= $(shell command -v python3) BIN_DIR ?= /usr/bin LIB_DIR ?= /usr/lib/qfs @@ -45,17 +45,14 @@ SERVICES ?= metaserver chunkserver webui override_dh_auto_build: bash debian/do-component-build \ --qfs-version=$(QFS_BASE_VERSION) \ - --hadoop-version=$(HADOOP_VERSION) \ - --python=$(PYTHON_PATH) \ - --python3=$(PYTHON3_PATH) + --python=$(PYTHON_PATH) override_dh_auto_install: bash -x debian/install_qfs.sh \ + --source-dir=debian \ --prefix=debian/tmp \ --qfs-version=$(QFS_BASE_VERSION) \ - --hadoop-version=$(HADOOP_VERSION) \ --python=$(PYTHON_PATH) \ - --python3=$(PYTHON3_PATH) \ --python-extra='--install-layout=deb' \ --bin-dir=$(BIN_DIR) \ --lib-dir=$(LIB_DIR) \ http://git-wip-us.apache.org/repos/asf/bigtop/blob/bcf8b390/bigtop-packages/src/rpm/qfs/SPECS/qfs.spec ---------------------------------------------------------------------- diff --git a/bigtop-packages/src/rpm/qfs/SPECS/qfs.spec b/bigtop-packages/src/rpm/qfs/SPECS/qfs.spec index 2001d46..efce8cc 100644 --- a/bigtop-packages/src/rpm/qfs/SPECS/qfs.spec +++ b/bigtop-packages/src/rpm/qfs/SPECS/qfs.spec @@ -23,12 +23,6 @@ %define var_dir /var %define qfs_services chunkserver metaserver webui -%global HADOOP_VERSION %{hadoop_version} - -%if %{?!HADOOP_VERSION:1}0 -%global HADOOP_VERSION 1.0.4 -%endif - %if %{?!HADOOP_HOME:1}0 %global HADOOP_HOME /usr/lib/hadoop %endif @@ -177,7 +171,6 @@ PTYHON3_PATH=%{__python3} %endif # with_python3 bash $RPM_SOURCE_DIR/do-component-build \ - --hadoop-version=%{HADOOP_VERSION} \ --qfs-version=%{qfs_version} \ --python=%{__python} \ --python3=$PYTHON3_PATH @@ -187,8 +180,8 @@ rm -rf $RPM_BUILD_ROOT mkdir -p $RPM_BUILD_ROOT sh $RPM_SOURCE_DIR/install_qfs.sh \ + --source-dir=$RPM_SOURCE_DIR \ --prefix=$RPM_BUILD_ROOT \ - --hadoop-version=%{HADOOP_VERSION} \ --qfs-version=%{qfs_version} \ --python=%{__python} \ --python3=$PYTHON3_PATH \ @@ -342,7 +335,7 @@ rm -rf $RPM_BUILD_ROOT %files hadoop %defattr(-,root,root,-) -%{HADOOP_HOME}/lib/hadoop-%{HADOOP_VERSION}-qfs-%{qfs_version}.jar +%{HADOOP_HOME}/lib/hadoop-%{hadoop_version}-qfs-%{qfs_version}.jar %files java %defattr(-,root,root,-)