Return-Path: X-Original-To: apmail-atlas-commits-archive@minotaur.apache.org Delivered-To: apmail-atlas-commits-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 0DEFB17980 for ; Fri, 6 Nov 2015 09:02:20 +0000 (UTC) Received: (qmail 42140 invoked by uid 500); 6 Nov 2015 09:02:19 -0000 Delivered-To: apmail-atlas-commits-archive@atlas.apache.org Received: (qmail 42111 invoked by uid 500); 6 Nov 2015 09:02:19 -0000 Mailing-List: contact commits-help@atlas.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@atlas.incubator.apache.org Delivered-To: mailing list commits@atlas.incubator.apache.org Received: (qmail 42102 invoked by uid 99); 6 Nov 2015 09:02:19 -0000 Received: from Unknown (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 06 Nov 2015 09:02:19 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id 7457EC547B for ; Fri, 6 Nov 2015 09:02:19 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.791 X-Spam-Level: * X-Spam-Status: No, score=1.791 tagged_above=-999 required=6.31 tests=[KAM_ASCII_DIVIDERS=0.8, KAM_LAZY_DOMAIN_SECURITY=1, T_RP_MATCHES_RCVD=-0.01, URIBL_BLOCKED=0.001] autolearn=disabled Received: from mx1-us-west.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id TYcahikscGO4 for ; Fri, 6 Nov 2015 09:02:07 +0000 (UTC) Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx1-us-west.apache.org (ASF Mail Server at mx1-us-west.apache.org) with SMTP id 9279F21165 for ; Fri, 6 Nov 2015 09:02:07 +0000 (UTC) Received: (qmail 41972 invoked by uid 99); 6 Nov 2015 09:02:07 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 06 Nov 2015 09:02:07 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id 5951AE0A3C; Fri, 6 Nov 2015 09:02:07 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: sumasai@apache.org To: commits@atlas.incubator.apache.org Message-Id: <6554ffb4bfd34d9a8b4dd23c36acd9a8@git.apache.org> X-Mailer: ASF-Git Admin Mailer Subject: incubator-atlas git commit: ATLAS-257 import_hive.sh fails when run under cygwin(dkantor via sumasai) Date: Fri, 6 Nov 2015 09:02:07 +0000 (UTC) Repository: incubator-atlas Updated Branches: refs/heads/master aafc06c4c -> 27a39063d ATLAS-257 import_hive.sh fails when run under cygwin(dkantor via sumasai) Project: http://git-wip-us.apache.org/repos/asf/incubator-atlas/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-atlas/commit/27a39063 Tree: http://git-wip-us.apache.org/repos/asf/incubator-atlas/tree/27a39063 Diff: http://git-wip-us.apache.org/repos/asf/incubator-atlas/diff/27a39063 Branch: refs/heads/master Commit: 27a39063d17ceaa32b602f2661d7a7ffb6dbf3ab Parents: aafc06c Author: Suma Shivaprasad Authored: Fri Nov 6 14:31:57 2015 +0530 Committer: Suma Shivaprasad Committed: Fri Nov 6 14:31:57 2015 +0530 ---------------------------------------------------------------------- addons/hive-bridge/src/bin/import-hive.sh | 46 ++++++++++++++++++-------- release-log.txt | 1 + 2 files changed, 33 insertions(+), 14 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/incubator-atlas/blob/27a39063/addons/hive-bridge/src/bin/import-hive.sh ---------------------------------------------------------------------- diff --git a/addons/hive-bridge/src/bin/import-hive.sh b/addons/hive-bridge/src/bin/import-hive.sh index 8cb1868..21b84f7 100755 --- a/addons/hive-bridge/src/bin/import-hive.sh +++ b/addons/hive-bridge/src/bin/import-hive.sh @@ -16,6 +16,8 @@ # resolve links - $0 may be a softlink PRG="${0}" +[[ `uname -s` == *"CYGWIN"* ]] && CYGWIN=true + while [ -h "${PRG}" ]; do ls=`ls -ld "${PRG}"` link=`expr "$ls" : '.*-> \(.*\)$'` @@ -38,21 +40,23 @@ if [ -f "${METADATA_CONF}/atlas-env.sh" ]; then . "${METADATA_CONF}/atlas-env.sh" fi -if test -z ${JAVA_HOME} +if test -z "${JAVA_HOME}" then JAVA_BIN=`which java` JAR_BIN=`which jar` else - JAVA_BIN=${JAVA_HOME}/bin/java - JAR_BIN=${JAVA_HOME}/bin/jar + JAVA_BIN="${JAVA_HOME}/bin/java" + JAR_BIN="${JAVA_HOME}/bin/jar" fi export JAVA_BIN -if [ ! -e $JAVA_BIN ] || [ ! -e $JAR_BIN ]; then +if [ ! -e "${JAVA_BIN}" ] || [ ! -e "${JAR_BIN}" ]; then echo "$JAVA_BIN and/or $JAR_BIN not found on the system. Please make sure java and jar commands are available." exit 1 fi +# Construct classpath using Atlas conf directory +# and jars from bridge/hive and hook/hive directories. METADATACPPATH="$METADATA_CONF" for i in "${BASEDIR}/bridge/hive/"*.jar; do @@ -66,14 +70,8 @@ done # log dir for applications METADATA_LOG_DIR="${METADATA_LOG_DIR:-$BASEDIR/logs}" export METADATA_LOG_DIR +LOGFILE="$METADATA_LOG_DIR/import-hive.log" -JAVA_PROPERTIES="$METADATA_OPTS -Datlas.log.dir=$METADATA_LOG_DIR -Datlas.log.file=import-hive.log -Dlog4j.configuration=atlas-log4j.xml" -shift - -while [[ ${1} =~ ^\-D ]]; do - JAVA_PROPERTIES="${JAVA_PROPERTIES} ${1}" - shift -done TIME=`date +%Y%m%d%H%M%s` #Add hive conf in classpath @@ -88,10 +86,30 @@ else exit 1 fi export HIVE_CP -echo Using Hive configuration directory [$HIVE_CP] -echo "Logs for import are in $METADATA_LOG_DIR/import-hive.log" -${JAVA_BIN} ${JAVA_PROPERTIES} -cp ${HIVE_CP}:${METADATACPPATH} org.apache.atlas.hive.bridge.HiveMetaStoreBridge +CP="${HIVE_CP}:${METADATACPPATH}" + +# If running in cygwin, convert pathnames and classpath to Windows format. +if [ "${CYGWIN}" == "true" ] +then + METADATA_LOG_DIR=`cygpath -w ${METADATA_LOG_DIR}` + LOGFILE=`cygpath -w ${LOGFILE}` + HIVE_CP=`cygpath -w ${HIVE_CP}` + CP=`cygpath -w -p ${CP}` +fi + +JAVA_PROPERTIES="$METADATA_OPTS -Datlas.log.dir=$METADATA_LOG_DIR -Datlas.log.file=import-hive.log -Dlog4j.configuration=atlas-log4j.xml" +shift + +while [[ ${1} =~ ^\-D ]]; do + JAVA_PROPERTIES="${JAVA_PROPERTIES} ${1}" + shift +done + +echo Using Hive configuration directory ["$HIVE_CP"] +echo "Log file for import is $LOGFILE" + +"${JAVA_BIN}" ${JAVA_PROPERTIES} -cp "${CP}" org.apache.atlas.hive.bridge.HiveMetaStoreBridge RETVAL=$? [ $RETVAL -eq 0 ] && echo Hive Data Model imported successfully!!! http://git-wip-us.apache.org/repos/asf/incubator-atlas/blob/27a39063/release-log.txt ---------------------------------------------------------------------- diff --git a/release-log.txt b/release-log.txt index b5dc76e..1d7fbc5 100644 --- a/release-log.txt +++ b/release-log.txt @@ -10,6 +10,7 @@ ATLAS-54 Rename configs in hive hook (shwethags) ATLAS-3 Mixed Index creation fails with Date types (sumasai via shwethags) ALL CHANGES: +ATLAS-257 import_hive.sh fails when run under cygwin(dkantor via sumasai) ATLAS-255 Add log level setting for titan in atlas-log4j.xml(ayubkhan via sumasai) ATLAS-246 QuickStart uses integer data type for dates, which causes data loss (dkantor via sumasai) ATLAS-232 Fix the API incompatibility introduced in ATLAS-58(shwethags via sumasai)