Return-Path: X-Original-To: apmail-trafodion-commits-archive@www.apache.org Delivered-To: apmail-trafodion-commits-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id EB5DC18694 for ; Mon, 7 Mar 2016 19:36:08 +0000 (UTC) Received: (qmail 44774 invoked by uid 500); 7 Mar 2016 19:36:08 -0000 Delivered-To: apmail-trafodion-commits-archive@trafodion.apache.org Received: (qmail 44744 invoked by uid 500); 7 Mar 2016 19:36:08 -0000 Mailing-List: contact commits-help@trafodion.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: commits@trafodion.apache.org Delivered-To: mailing list commits@trafodion.apache.org Received: (qmail 44735 invoked by uid 99); 7 Mar 2016 19:36:08 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 07 Mar 2016 19:36:08 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id 6BCE7C0679 for ; Mon, 7 Mar 2016 19:36:08 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -3.549 X-Spam-Level: X-Spam-Status: No, score=-3.549 tagged_above=-999 required=6.31 tests=[KAM_ASCII_DIVIDERS=0.8, KAM_LAZY_DOMAIN_SECURITY=1, RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RP_MATCHES_RCVD=-0.329] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id 6rn8w37rXoMN for ; Mon, 7 Mar 2016 19:36:06 +0000 (UTC) Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with SMTP id E46D05FB57 for ; Mon, 7 Mar 2016 19:36:04 +0000 (UTC) Received: (qmail 44089 invoked by uid 99); 7 Mar 2016 19:36:03 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 07 Mar 2016 19:36:03 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id 3C1CFDFB8A; Mon, 7 Mar 2016 19:36:03 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: amanda@apache.org To: commits@trafodion.incubator.apache.org Date: Mon, 07 Mar 2016 19:36:03 -0000 Message-Id: <178b8caa4e6e4beb93d4d55aad8a0881@git.apache.org> X-Mailer: ASF-Git Admin Mailer Subject: [1/4] incubator-trafodion git commit: JIRA TRAFODION 1001 1478 1691 1800 1821 Repository: incubator-trafodion Updated Branches: refs/heads/master 9111d8cc4 -> 57d3ebd1d JIRA TRAFODION 1001 1478 1691 1800 1821 1001 Finding zookeeper nodes 1478 Removing /hbase/archive/data/default from traf_hortonworks_mods98 1821 ldapconfigcheck changes 1691 Increase subscripts 1800 Backup resttore changes Project: http://git-wip-us.apache.org/repos/asf/incubator-trafodion/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-trafodion/commit/3c45de7c Tree: http://git-wip-us.apache.org/repos/asf/incubator-trafodion/tree/3c45de7c Diff: http://git-wip-us.apache.org/repos/asf/incubator-trafodion/diff/3c45de7c Branch: refs/heads/master Commit: 3c45de7c32cd8542e46a19c4d1390929a1f12363 Parents: 1824c7e Author: Amanda Moran Authored: Thu Mar 3 21:21:01 2016 +0000 Committer: Amanda Moran Committed: Thu Mar 3 21:21:01 2016 +0000 ---------------------------------------------------------------------- install/installer/dcs_installer | 8 +----- install/installer/parseHBaseSite.py | 34 ++++++++++++++++++++++++++ install/installer/traf_add_sudoAccess | 5 +++- install/installer/traf_config_check | 8 ++++-- install/installer/traf_hortonworks_mods98 | 26 -------------------- install/installer/traf_sqgen | 2 +- install/installer/trafodion_install | 19 +++++++++++--- 7 files changed, 62 insertions(+), 40 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/incubator-trafodion/blob/3c45de7c/install/installer/dcs_installer ---------------------------------------------------------------------- diff --git a/install/installer/dcs_installer b/install/installer/dcs_installer index 5d71aab..b07a748 100755 --- a/install/installer/dcs_installer +++ b/install/installer/dcs_installer @@ -110,7 +110,7 @@ while [[ $# -gt 0 ]]; do done checkOpts - +source $TRAF_CONFIG echo "***INFO: Start of DCS install" # Make sure required env vars have been set first @@ -155,12 +155,6 @@ rm dcs-env.temp 2>/dev/null echo "***INFO: modifying $DCS_DIR/conf/dcs-site.xml" -# get zookeeper quorum - -lineNumber=$(grep -n "zookeeper\.quorum" $TRAF_WORKDIR/hbase-site.xml | sed 's/\:.*//') -lineNumber=$((lineNumber+1)) - -ZOOKEEPER_NODES=$(sed "$lineNumber!d" $TRAF_WORKDIR/hbase-site.xml | sed 's/\/value.*//' | sed 's/.*>//' | sed 's/.$//') # add zookeeper quorum property to end of configuration rm dcs-site.temp 2>/dev/null http://git-wip-us.apache.org/repos/asf/incubator-trafodion/blob/3c45de7c/install/installer/parseHBaseSite.py ---------------------------------------------------------------------- diff --git a/install/installer/parseHBaseSite.py b/install/installer/parseHBaseSite.py new file mode 100755 index 0000000..4ac7e1b --- /dev/null +++ b/install/installer/parseHBaseSite.py @@ -0,0 +1,34 @@ +#!/usr/bin/python + +import os + +import xml.etree.ElementTree as ET + +hbaseMaster="hbase.master.info.port" +hbaseRegion="hbase.regionserver.info.port" +zooKeeperNodes="hbase.zookeeper.quorum" +pathToHome= os.environ['HOME'] + +tree = ET.parse( pathToHome + '/hbase-site.xml') + +root = tree.getroot() + + +for x in root.findall('property'): + name = str(x.find('name').text) + if name == hbaseMaster: + value = x.find('value').text + f = open( '/etc/trafodion/trafodion_config', 'a') + f.write ( 'export HBASE_MASTER_INFO_PORT="' + value + '"\n' ) + f.close() + if name == hbaseRegion: + value = x.find('value').text + f = open( '/etc/trafodion/trafodion_config', 'a') + f.write ( 'export REGIONSERVER_INFO_PORT="' + value + '"\n' ) + f.close() + if name == zooKeeperNodes: + value = x.find('value').text + f = open( '/etc/trafodion/trafodion_config', 'a') + f.write ( 'export ZOOKEEPER_NODES="' + value + '"\n' ) + f.close() + http://git-wip-us.apache.org/repos/asf/incubator-trafodion/blob/3c45de7c/install/installer/traf_add_sudoAccess ---------------------------------------------------------------------- diff --git a/install/installer/traf_add_sudoAccess b/install/installer/traf_add_sudoAccess index 9440977..1e5cc9b 100755 --- a/install/installer/traf_add_sudoAccess +++ b/install/installer/traf_add_sudoAccess @@ -38,6 +38,8 @@ echo "%trafodion ALL = NOPASSWD: IP, ARP" >> $sudoFile function backupRestore { +source $TRAF_CONFIG + echo "## Allow $BACKUP_USER id to run commands needed for backup and restore" >> $BACKUP_USER echo "%$BACKUP_USER ALL =(hbase) NOPASSWD: /usr/bin/hbase" >> $BACKUP_USER @@ -72,8 +74,9 @@ fi #Main echo "***INFO: Creating $TRAF_USER sudo access file" echo +touch trafodion -if [[ "$ENABLE_HA" == "Y" ]]; then +if [[ "$ENABLE_HA" == "true" ]]; then haSudoAdd copyTrafFile fi http://git-wip-us.apache.org/repos/asf/incubator-trafodion/blob/3c45de7c/install/installer/traf_config_check ---------------------------------------------------------------------- diff --git a/install/installer/traf_config_check b/install/installer/traf_config_check index 95e3f4a..ef8f603 100755 --- a/install/installer/traf_config_check +++ b/install/installer/traf_config_check @@ -426,7 +426,7 @@ else fi done fi - export PATH=$PATH:$JAVA_HOME/bin + export PATH="$PATH:$JAVA_HOME/bin" sudo chmod 777 $TRAF_CONFIG sed -i '/PATH\=/d' $TRAF_CONFIG echo "export PATH=\"$PATH\"" >>$TRAF_CONFIG @@ -489,11 +489,15 @@ fi function checkBackupUser { +TRAF_CONFIG="/etc/trafodion/trafodion_config" + +source $TRAF_CONFIG + if [ -z $BACKUP_USER ]; then echo "***ERROR: BACKUP_USER variable not set in config file." echo "***WARNING: BACKUP_USER will be set to trafodion." sudo chmod 777 $TRAF_CONFIG - sudo "export BACKUP_USER=\"trafodion\"" >> $TRAF_CONFIG + sudo echo "export BACKUP_USER=\"trafodion\"" >> $TRAF_CONFIG sudo chmod 777 $TRAF_CONFIG fi http://git-wip-us.apache.org/repos/asf/incubator-trafodion/blob/3c45de7c/install/installer/traf_hortonworks_mods98 ---------------------------------------------------------------------- diff --git a/install/installer/traf_hortonworks_mods98 b/install/installer/traf_hortonworks_mods98 index 7d2be7e..7070e44 100755 --- a/install/installer/traf_hortonworks_mods98 +++ b/install/installer/traf_hortonworks_mods98 @@ -557,32 +557,6 @@ if [ $? != 0 ]; then exit -1 fi -ssh -q -n $AMBARI_HOST 'sudo su hdfs --command "hdfs dfs -mkdir -p /apps/hbase/data/archive/data"' -if [ $? != 0 ]; then - echo "***ERROR: (hdfs dfs -mkdir -p /apps/hbase/data/archive/data) command failed" - exit -1 -fi - -ssh -q -n $AMBARI_HOST 'sudo su hdfs --command "hdfs dfs -chown hbase:hdfs /apps/hbase/data/archive/data"' -if [ $? != 0 ]; then - echo "***ERROR: (hdfs dfs -chown hbase:hdfs /apps/hbase/data/archive/data) command failed" - exit -1 -fi - -ssh -q -n $AMBARI_HOST 'sudo su hdfs --command "hdfs dfs -mkdir -p /apps/hbase/data/archive/data/default"' -if [ $? != 0 ]; then - echo "***ERROR: (hdfs dfs -mkdir -p /apps/hbase/data/archive/data/default) command failed" - exit -1 -fi - -ssh -q -n $AMBARI_HOST 'sudo su hdfs --command "hdfs dfs -chown hbase:hdfs /apps/hbase/data/archive/data/default"' -if [ $? != 0 ]; then - echo "***ERROR: (hdfs dfs -chown hbase:hdfs /apps/hbase/data/archive/data/default) command failed" - exit -1 -fi - - - # clean up files generated by Ambari's config.sh script ssh -q -n $AMBARI_HOST 'rm $HOME/doSet_version*' http://git-wip-us.apache.org/repos/asf/incubator-trafodion/blob/3c45de7c/install/installer/traf_sqgen ---------------------------------------------------------------------- diff --git a/install/installer/traf_sqgen b/install/installer/traf_sqgen index d3e2da2..271e719 100755 --- a/install/installer/traf_sqgen +++ b/install/installer/traf_sqgen @@ -73,7 +73,7 @@ if [[ "$LDAP_SECURITY" == "Y" ]]; then #Check traf_authentication_config for errors echo "***INFO: Checking LDAP Configuration file for errors." - $SQ_ROOT/export/bin64/ldapconfigcheck -file $SQ_ROOT/sql/scripts/traf_authentication_config + $SQ_ROOT/export/bin${SQ_MBTYPE}/ldapconfigcheck -file $SQ_ROOT/sql/scripts/traf_authentication_config if [ "$?" != "0" ]; then echo "***ERROR: traf_authentication_config not configured correctly." echo "***ERROR: Install will continue WITHOUT simple security turned on." http://git-wip-us.apache.org/repos/asf/incubator-trafodion/blob/3c45de7c/install/installer/trafodion_install ---------------------------------------------------------------------- diff --git a/install/installer/trafodion_install b/install/installer/trafodion_install index 4bb6649..7330c2c 100755 --- a/install/installer/trafodion_install +++ b/install/installer/trafodion_install @@ -103,9 +103,9 @@ do exit -1 fi else - supportedHBase=$(echo $version | grep "1.1.1" | wc -l) + supportedHBase=$(echo $version | grep "1.1.2" | wc -l) if [[ $supportedHBase -eq "0" ]]; then - echo "***ERROR: HBase installed must be 1.1.1 on node $node" + echo "***ERROR: HBase installed must be 1.1.2 on node $node" exit -1 fi fi @@ -759,7 +759,7 @@ if [[ "$LDAP_SECURITY" == "Y" ]]; then $TRAF_PDSH sudo chmod 750 $HOME_DIR/$TRAF_USER/$LDAP_AUTH_FILE if [[ $LDAP_LEVEL == "2" ]] || [[ $LDAP_LEVEL == "1" ]]; then echo "***INFO: Copying $LDAP_CERT to Trafodion home dir" - $TRAF_PDCP $LDAP_CERT $HOME + $TRAF_PCP $LDAP_CERT $HOME $TRAF_PDSH sudo cp -r $HOME/$LDAP_CERT_BASE $HOME_DIR/$TRAF_USER $TRAF_PDSH sudo chown trafodion.trafodion $HOME_DIR/$TRAF_USER/$LDAP_CERT_BASE $TRAF_PDSH sudo chmod 750 $HOME_DIR/$TRAF_USER/$LDAP_CERT_BASE @@ -803,6 +803,19 @@ fi echo "***INFO: Trafodion Mods ran successfully." +#Get HBaseSite.xml information +$LOCAL_WORKDIR/parseHBaseSite.py + +if [[ $HADOOP_TYPE == "cloudera" ]]; then + sudo chmod 777 $TRAF_CONFIG + sed -i '/HBASE_MASTER_INFO_PORT\=/d' $TRAF_CONFIG + echo "export HBASE_MASTER_INFO_PORT=\"60010\"" >> $TRAF_CONFIG + sed -i '/REGIONSERVER_INFO_PORT\=/d' $TRAF_CONFIG + echo "export REGIONSERVER_INFO_PORT=\"60030\"" >> $TRAF_CONFIG + sudo chmod 777 $TRAF_CONFIG + +fi + #============================================== #Install Trafodion echo