Return-Path: X-Original-To: apmail-accumulo-commits-archive@www.apache.org Delivered-To: apmail-accumulo-commits-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id AD79610398 for ; Wed, 17 Jul 2013 16:03:32 +0000 (UTC) Received: (qmail 83133 invoked by uid 500); 17 Jul 2013 16:03:25 -0000 Delivered-To: apmail-accumulo-commits-archive@accumulo.apache.org Received: (qmail 83087 invoked by uid 500); 17 Jul 2013 16:03:25 -0000 Mailing-List: contact commits-help@accumulo.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@accumulo.apache.org Delivered-To: mailing list commits@accumulo.apache.org Received: (qmail 82974 invoked by uid 99); 17 Jul 2013 16:03:23 -0000 Received: from tyr.zones.apache.org (HELO tyr.zones.apache.org) (140.211.11.114) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 17 Jul 2013 16:03:23 +0000 Received: by tyr.zones.apache.org (Postfix, from userid 65534) id 9D7BA8AC0D0; Wed, 17 Jul 2013 16:03:22 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: ecn@apache.org To: commits@accumulo.apache.org Date: Wed, 17 Jul 2013 16:03:24 -0000 Message-Id: In-Reply-To: <923085cfcffc4fb4b0072e89b9a983a1@git.apache.org> References: <923085cfcffc4fb4b0072e89b9a983a1@git.apache.org> X-Mailer: ASF-Git Admin Mailer Subject: [3/4] git commit: ACCUMULO-1550 committing Jonathan Hsieh's patch ACCUMULO-1550 committing Jonathan Hsieh's patch Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/c82c4316 Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/c82c4316 Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/c82c4316 Branch: refs/heads/master Commit: c82c4316ad90ee7370c5919efb8ded65bb50b302 Parents: cfb01d4 Author: Eric Newton Authored: Wed Jul 17 11:57:15 2013 -0400 Committer: Eric Newton Committed: Wed Jul 17 11:57:15 2013 -0400 ---------------------------------------------------------------------- README | 10 ++--- bin/accumulo | 4 +- bin/config.sh | 41 +++++++++++++++----- bin/start-all.sh | 8 ++-- bin/start-here.sh | 6 +-- bin/start-server.sh | 2 +- bin/stop-all.sh | 4 +- bin/stop-here.sh | 4 +- bin/tdown.sh | 2 +- bin/tup.sh | 2 +- .../1GB/native-standalone/accumulo-env.sh | 4 +- conf/examples/1GB/standalone/accumulo-env.sh | 4 +- .../2GB/native-standalone/accumulo-env.sh | 4 +- conf/examples/2GB/standalone/accumulo-env.sh | 4 +- .../3GB/native-standalone/accumulo-env.sh | 4 +- conf/examples/3GB/standalone/accumulo-env.sh | 4 +- .../512MB/native-standalone/accumulo-env.sh | 4 +- conf/examples/512MB/standalone/accumulo-env.sh | 4 +- .../org/apache/accumulo/server/Accumulo.java | 4 +- .../start/classloader/AccumuloClassLoader.java | 9 +++-- test/system/continuous/agitator.pl | 2 +- test/system/continuous/magitator.pl | 6 +-- test/system/continuous/mapred-setup.sh | 2 +- test/system/continuous/start-stats.sh | 2 +- test/system/randomwalk/README | 2 +- test/system/randomwalk/bin/reset-cluster.sh | 10 ++--- test/system/randomwalk/bin/start-all.sh | 4 +- test/system/randomwalk/bin/start-local.sh | 4 +- test/system/scalability/run.py | 2 +- 29 files changed, 94 insertions(+), 68 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/accumulo/blob/c82c4316/README ---------------------------------------------------------------------- diff --git a/README b/README index 21b53e2..445af37 100644 --- a/README +++ b/README @@ -111,12 +111,12 @@ accumulo client code. These directories must be at the same location on every node in the cluster. If you are configuring a larger cluster you will need to create the configuration -files yourself and propogate the changes to the $ACCUMULO_HOME/conf directories: +files yourself and propogate the changes to the $ACCUMULO_CONF_DIR directories: - Create a "slaves" file in $ACCUMULO_HOME/conf/. This is a list of machines + Create a "slaves" file in $ACCUMULO_CONF_DIR/. This is a list of machines where tablet servers and loggers will run. - Create a "masters" file in $ACCUMULO_HOME/conf/. This is a list of + Create a "masters" file in $ACCUMULO_CONF_DIR/. This is a list of machines where the master server will run. Create conf/accumulo-env.sh following the template of @@ -229,7 +229,7 @@ server, but you can also glob them if you wish. kadmin.local -q "xst -k accumulo.keytab -glob accumulo*" -Place this file in $ACCUMULO_HOME/conf for every host. It should be owned by +Place this file in $ACCUMULO_CONF_DIR for every host. It should be owned by the accumulo user and chmodded to 400. Add the following to the accumulo-env.sh kinit -kt $ACCUMULO_HOME/conf/accumulo.keytab accumulo/`hostname -f` @@ -242,7 +242,7 @@ _HOST in lieu of your individual host names. general.kerberos.keytab - $ACCUMULO_HOME/conf/accumulo.keytab + $ACCUMULO_CONF_DIR/accumulo.keytab http://git-wip-us.apache.org/repos/asf/accumulo/blob/c82c4316/bin/accumulo ---------------------------------------------------------------------- diff --git a/bin/accumulo b/bin/accumulo index 7ee197c..3ae3095 100755 --- a/bin/accumulo +++ b/bin/accumulo @@ -37,7 +37,7 @@ locationByProgram() { RESULT=$( which "$1" ) if [[ "$?" != 0 && -z "$RESULT" ]]; then - echo "Cannot find '$1' and '$2' is not set in conf/accumulo-env.sh" + echo "Cannot find '$1' and '$2' is not set in $ACCUMULO_CONF_DIR/accumulo-env.sh" exit 1 fi while [ -h "$RESULT" ]; do # resolve $RESULT until the file is no longer a symlink @@ -80,7 +80,7 @@ logger) export ACCUMULO_OPTS="${ACCUMULO_GENERAL_OPTS} ${ACCUMULO_LOGGER_OPTS}" *) export ACCUMULO_OPTS="${ACCUMULO_GENERAL_OPTS} ${ACCUMULO_OTHER_OPTS}" ;; esac -XML_FILES=${ACCUMULO_HOME}/conf +XML_FILES=${ACCUMULO_CONF_DIR} LOG4J_JAR=$(find $HADOOP_PREFIX/lib $HADOOP_PREFIX/share/hadoop/common/lib -name 'log4j*.jar' -print 2>/dev/null | head -1) CLASSPATH=${XML_FILES}:${START_JAR}:${LOG4J_JAR} http://git-wip-us.apache.org/repos/asf/accumulo/blob/c82c4316/bin/config.sh ---------------------------------------------------------------------- diff --git a/bin/config.sh b/bin/config.sh index a58b33a..6c4d74a 100755 --- a/bin/config.sh +++ b/bin/config.sh @@ -15,6 +15,21 @@ # See the License for the specific language governing permissions and # limitations under the License. +# Guarantees that Accumulo and its environment variables are set. +# +# Values set by script that can be user provided. If not provided script attempts to infer. +# ACCUMULO_CONF_DIR Location where accumulo-env.sh, accumulo-site.xml and friends will be read from +# ACCUMULO_HOME Home directory for Accumulo +# ACCUMULO_LOG_DIR Directory for Accumulo daemon logs +# ACCUMULO_VERSION Accumulo version name +# HADOOP_PREFIX Prefix to the home dir for hadoop. +# +# Values always set by script. +# GC Machine to rn GC daemon on. Used by start-here.sh script +# MONITOR Machine to run monitor daemon on. Used by start-here.sh script +# SSH Default ssh parameters used to start daemons +# HADOOP_HOME Home dir for hadoop. TODO fix this. + # Start: Resolve Script Directory SOURCE="${BASH_SOURCE[0]}" while [ -h "$SOURCE" ]; do # resolve $SOURCE until the file is no longer a symlink @@ -29,8 +44,16 @@ script=$( basename "$SOURCE" ) ACCUMULO_HOME=$( cd -P ${bin}/.. && pwd ) export ACCUMULO_HOME -if [ -f $ACCUMULO_HOME/conf/accumulo-env.sh ] ; then - . $ACCUMULO_HOME/conf/accumulo-env.sh +ACCUMULO_CONF_DIR="${ACCUMULO_CONF_DIR:-$ACCUMULO_HOME/conf}" +export ACCUMULO_CONF_DIR +if [ -z "$ACCUMULO_CONF_DIR" -o ! -d "$ACCUMULO_CONF_DIR" ] +then + echo "ACCUMULO_CONF_DIR=$ACCUMULO_CONF_DIR is not a valid directory. Please make sure it exists" + exit 1 +fi + +if [ -f $ACCUMULO_CONF_DIR/accumulo-env.sh ] ; then + . $ACCUMULO_CONF_DIR/accumulo-env.sh elif [ -z "$ACCUMULO_TEST" ] ; then # # Attempt to bootstrap configuration and continue @@ -70,17 +93,17 @@ then fi export HADOOP_PREFIX -MASTER1=$(egrep -v '(^#|^\s*$)' "$ACCUMULO_HOME/conf/masters" | head -1) +MASTER1=$(egrep -v '(^#|^\s*$)' "$ACCUMULO_CONF_DIR/masters" | head -1) GC=$MASTER1 MONITOR=$MASTER1 -if [ -f "$ACCUMULO_HOME/conf/gc" ]; then - GC=$(egrep -v '(^#|^\s*$)' "$ACCUMULO_HOME/conf/gc" | head -1) +if [ -f "$ACCUMULO_CONF_DIR/gc" ]; then + GC=$(egrep -v '(^#|^\s*$)' "$ACCUMULO_CONF_DIR/gc" | head -1) fi -if [ -f "$ACCUMULO_HOME/conf/monitor" ]; then - MONITOR=$(egrep -v '(^#|^\s*$)' "$ACCUMULO_HOME/conf/monitor" | head -1) +if [ -f "$ACCUMULO_CONF_DIR/monitor" ]; then + MONITOR=$(egrep -v '(^#|^\s*$)' "$ACCUMULO_CONF_DIR/monitor" | head -1) fi -if [ ! -f "$ACCUMULO_HOME/conf/tracers" ]; then - echo "$MASTER1" > "$ACCUMULO_HOME/conf/tracers" +if [ ! -f "$ACCUMULO_CONF_DIR/tracers" ]; then + echo "$MASTER1" > "$ACCUMULO_CONF_DIR/tracers" fi SSH='ssh -qnf -o ConnectTimeout=2' http://git-wip-us.apache.org/repos/asf/accumulo/blob/c82c4316/bin/start-all.sh ---------------------------------------------------------------------- diff --git a/bin/start-all.sh b/bin/start-all.sh index 8494e18..8470a40 100755 --- a/bin/start-all.sh +++ b/bin/start-all.sh @@ -28,8 +28,8 @@ bin="$( cd -P "$( dirname "$SOURCE" )" && pwd )" . "$bin"/config.sh unset DISPLAY -if [ ! -f $ACCUMULO_HOME/conf/accumulo-env.sh ] ; then - echo "${ACCUMULO_HOME}/conf/accumulo-env.sh does not exist. Please make sure you configure Accumulo before you run anything" +if [ ! -f $ACCUMULO_CONF_DIR/accumulo-env.sh ] ; then + echo "${ACCUMULO_CONF_DIR}/accumulo-env.sh does not exist. Please make sure you configure Accumulo before you run anything" echo "We provide examples you can copy in ${ACCUMULO_HOME}/conf/examples/ which are set up for your memory footprint" exit 1 fi @@ -54,12 +54,12 @@ if [ "$1" != "--notSlaves" ]; then fi ${bin}/accumulo org.apache.accumulo.server.master.state.SetGoalState NORMAL -for master in `egrep -v '(^#|^\s*$)' "$ACCUMULO_HOME/conf/masters"`; do +for master in `egrep -v '(^#|^\s*$)' "$ACCUMULO_CONF_DIR/masters"`; do ${bin}/start-server.sh $master master done ${bin}/start-server.sh $GC gc "garbage collector" -for tracer in `egrep -v '(^#|^\s*$)' "$ACCUMULO_HOME/conf/tracers"`; do +for tracer in `egrep -v '(^#|^\s*$)' "$ACCUMULO_CONF_DIR/tracers"`; do ${bin}/start-server.sh $tracer tracer done http://git-wip-us.apache.org/repos/asf/accumulo/blob/c82c4316/bin/start-here.sh ---------------------------------------------------------------------- diff --git a/bin/start-here.sh b/bin/start-here.sh index feb3d5e..2952975 100755 --- a/bin/start-here.sh +++ b/bin/start-here.sh @@ -42,14 +42,14 @@ fi HOSTS="`hostname -a` `hostname` localhost 127.0.0.1 $ip" for host in $HOSTS; do - if grep -q "^${host}\$" $ACCUMULO_HOME/conf/slaves; then + if grep -q "^${host}\$" $ACCUMULO_CONF_DIR/slaves; then ${bin}/start-server.sh $host tserver "tablet server" break fi done for host in $HOSTS; do - if grep -q "^${host}\$" $ACCUMULO_HOME/conf/masters; then + if grep -q "^${host}\$" $ACCUMULO_CONF_DIR/masters; then ${bin}/accumulo org.apache.accumulo.server.master.state.SetGoalState NORMAL ${bin}/start-server.sh $host master break @@ -71,7 +71,7 @@ for host in $HOSTS; do done for host in $HOSTS; do - if grep -q "^${host}\$" $ACCUMULO_HOME/conf/tracers; then + if grep -q "^${host}\$" $ACCUMULO_CONF_DIR/tracers; then ${bin}/start-server.sh $host tracer break fi http://git-wip-us.apache.org/repos/asf/accumulo/blob/c82c4316/bin/start-server.sh ---------------------------------------------------------------------- diff --git a/bin/start-server.sh b/bin/start-server.sh index 75d9342..e38c084 100755 --- a/bin/start-server.sh +++ b/bin/start-server.sh @@ -39,7 +39,7 @@ LONGNAME="$3" if [ -z "$LONGNAME" ]; then LONGNAME="$2" fi -SLAVES=$( wc -l < ${ACCUMULO_HOME}/conf/slaves ) +SLAVES=$( wc -l < ${ACCUMULO_CONF_DIR}/slaves ) IFCONFIG=/sbin/ifconfig if [ ! -x $IFCONFIG ]; then http://git-wip-us.apache.org/repos/asf/accumulo/blob/c82c4316/bin/stop-all.sh ---------------------------------------------------------------------- diff --git a/bin/stop-all.sh b/bin/stop-all.sh index e2724aa..348d067 100755 --- a/bin/stop-all.sh +++ b/bin/stop-all.sh @@ -44,7 +44,7 @@ sleep 5 #look for master and gc processes not killed by 'admin stopAll' for signal in TERM KILL ; do - for master in `grep -v '^#' "$ACCUMULO_HOME/conf/masters"`; do + for master in `grep -v '^#' "$ACCUMULO_CONF_DIR/masters"`; do ${bin}/stop-server.sh $master "$ACCUMULO_HOME/lib/accumulo-start.jar" master $signal done @@ -52,7 +52,7 @@ for signal in TERM KILL ; do ${bin}/stop-server.sh "$MONITOR" "$ACCUMULO_HOME/.*/accumulo-start.*.jar" monitor $signal - for tracer in `egrep -v '(^#|^\s*$)' "$ACCUMULO_HOME/conf/tracers"`; do + for tracer in `egrep -v '(^#|^\s*$)' "$ACCUMULO_CONF_DIR/tracers"`; do ${bin}/stop-server.sh $tracer "$ACCUMULO_HOME/.*/accumulo-start.*.jar" tracer $signal done done http://git-wip-us.apache.org/repos/asf/accumulo/blob/c82c4316/bin/stop-here.sh ---------------------------------------------------------------------- diff --git a/bin/stop-here.sh b/bin/stop-here.sh index d23b228..8a3b513 100755 --- a/bin/stop-here.sh +++ b/bin/stop-here.sh @@ -33,11 +33,11 @@ bin="$( cd -P "$( dirname "$SOURCE" )" && pwd )" ACCUMULO="$ACCUMULO_HOME/lib/accumulo-start.jar" -if egrep -q localhost\|127.0.0.1 $ACCUMULO_HOME/conf/slaves; then +if egrep -q localhost\|127.0.0.1 $ACCUMULO_CONF_DIR/slaves; then $bin/accumulo admin stop localhost else for host in `hostname -a`; do - if grep -q ${host} $ACCUMULO_HOME/conf/slaves; then + if grep -q ${host} $ACCUMULO_CONF_DIR/slaves; then ${bin}/accumulo admin stop $host fi done http://git-wip-us.apache.org/repos/asf/accumulo/blob/c82c4316/bin/tdown.sh ---------------------------------------------------------------------- diff --git a/bin/tdown.sh b/bin/tdown.sh index 3dfc856..141ad24 100755 --- a/bin/tdown.sh +++ b/bin/tdown.sh @@ -28,7 +28,7 @@ bin="$( cd -P "$( dirname "$SOURCE" )" && pwd )" . "$bin"/config.sh HADOOP_CMD=$HADOOP_PREFIX/bin/hadoop -SLAVES=$ACCUMULO_HOME/conf/slaves +SLAVES=$ACCUMULO_CONF_DIR/slaves SLAVE_HOSTS=$(egrep -v '(^#|^\s*$)' "${SLAVES}") echo "Stopping unresponsive tablet servers (if any)..." http://git-wip-us.apache.org/repos/asf/accumulo/blob/c82c4316/bin/tup.sh ---------------------------------------------------------------------- diff --git a/bin/tup.sh b/bin/tup.sh index 07b4609..b26def5 100755 --- a/bin/tup.sh +++ b/bin/tup.sh @@ -27,7 +27,7 @@ bin="$( cd -P "$( dirname "$SOURCE" )" && pwd )" . "$bin"/config.sh -SLAVES=$ACCUMULO_HOME/conf/slaves +SLAVES=$ACCUMULO_CONF_DIR/slaves echo -n "Starting tablet servers ..." http://git-wip-us.apache.org/repos/asf/accumulo/blob/c82c4316/conf/examples/1GB/native-standalone/accumulo-env.sh ---------------------------------------------------------------------- diff --git a/conf/examples/1GB/native-standalone/accumulo-env.sh b/conf/examples/1GB/native-standalone/accumulo-env.sh index 04c85e7..16779c4 100755 --- a/conf/examples/1GB/native-standalone/accumulo-env.sh +++ b/conf/examples/1GB/native-standalone/accumulo-env.sh @@ -42,9 +42,9 @@ test -z "$HADOOP_CONF_DIR" && export HADOOP_CONF_DIR="$HADOOP_PREFIX/conf" test -z "$JAVA_HOME" && export JAVA_HOME=/path/to/java test -z "$ZOOKEEPER_HOME" && export ZOOKEEPER_HOME=/path/to/zookeeper test -z "$ACCUMULO_LOG_DIR" && export ACCUMULO_LOG_DIR=$ACCUMULO_HOME/logs -if [ -f ${ACCUMULO_HOME}/conf/accumulo.policy ] +if [ -f ${ACCUMULO_CONF_DIR}/accumulo.policy ] then - POLICY="-Djava.security.manager -Djava.security.policy=${ACCUMULO_HOME}/conf/accumulo.policy" + POLICY="-Djava.security.manager -Djava.security.policy=${ACCUMULO_CONF_DIR}/accumulo.policy" fi test -z "$ACCUMULO_TSERVER_OPTS" && export ACCUMULO_TSERVER_OPTS="${POLICY} -Xmx128m -Xms128m " test -z "$ACCUMULO_MASTER_OPTS" && export ACCUMULO_MASTER_OPTS="${POLICY} -Xmx128m -Xms128m" http://git-wip-us.apache.org/repos/asf/accumulo/blob/c82c4316/conf/examples/1GB/standalone/accumulo-env.sh ---------------------------------------------------------------------- diff --git a/conf/examples/1GB/standalone/accumulo-env.sh b/conf/examples/1GB/standalone/accumulo-env.sh index f830256..10c1ab9 100755 --- a/conf/examples/1GB/standalone/accumulo-env.sh +++ b/conf/examples/1GB/standalone/accumulo-env.sh @@ -42,9 +42,9 @@ test -z "$HADOOP_CONF_DIR" && export HADOOP_CONF_DIR="$HADOOP_PREFIX/conf" test -z "$JAVA_HOME" && export JAVA_HOME=/path/to/java test -z "$ZOOKEEPER_HOME" && export ZOOKEEPER_HOME=/path/to/zookeeper test -z "$ACCUMULO_LOG_DIR" && export ACCUMULO_LOG_DIR=$ACCUMULO_HOME/logs -if [ -f ${ACCUMULO_HOME}/conf/accumulo.policy ] +if [ -f ${ACCUMULO_CONF_DIR}/accumulo.policy ] then - POLICY="-Djava.security.manager -Djava.security.policy=${ACCUMULO_HOME}/conf/accumulo.policy" + POLICY="-Djava.security.manager -Djava.security.policy=${ACCUMULO_CONF_DIR}/accumulo.policy" fi test -z "$ACCUMULO_TSERVER_OPTS" && export ACCUMULO_TSERVER_OPTS="${POLICY} -Xmx384m -Xms384m " test -z "$ACCUMULO_MASTER_OPTS" && export ACCUMULO_MASTER_OPTS="${POLICY} -Xmx128m -Xms128m" http://git-wip-us.apache.org/repos/asf/accumulo/blob/c82c4316/conf/examples/2GB/native-standalone/accumulo-env.sh ---------------------------------------------------------------------- diff --git a/conf/examples/2GB/native-standalone/accumulo-env.sh b/conf/examples/2GB/native-standalone/accumulo-env.sh index 7db28c1..4302a1a 100755 --- a/conf/examples/2GB/native-standalone/accumulo-env.sh +++ b/conf/examples/2GB/native-standalone/accumulo-env.sh @@ -41,9 +41,9 @@ test -z "$HADOOP_CONF_DIR" && export HADOOP_CONF_DIR="$HADOOP_PREFIX/conf" test -z "$JAVA_HOME" && export JAVA_HOME=/path/to/java test -z "$ZOOKEEPER_HOME" && export ZOOKEEPER_HOME=/path/to/zookeeper test -z "$ACCUMULO_LOG_DIR" && export ACCUMULO_LOG_DIR=$ACCUMULO_HOME/logs -if [ -f ${ACCUMULO_HOME}/conf/accumulo.policy ] +if [ -f ${ACCUMULO_CONF_DIR}/accumulo.policy ] then - POLICY="-Djava.security.manager -Djava.security.policy=${ACCUMULO_HOME}/conf/accumulo.policy" + POLICY="-Djava.security.manager -Djava.security.policy=${ACCUMULO_CONF_DIR}/accumulo.policy" fi test -z "$ACCUMULO_TSERVER_OPTS" && export ACCUMULO_TSERVER_OPTS="${POLICY} -Xmx256m -Xms256m " test -z "$ACCUMULO_MASTER_OPTS" && export ACCUMULO_MASTER_OPTS="${POLICY} -Xmx256m -Xms256m" http://git-wip-us.apache.org/repos/asf/accumulo/blob/c82c4316/conf/examples/2GB/standalone/accumulo-env.sh ---------------------------------------------------------------------- diff --git a/conf/examples/2GB/standalone/accumulo-env.sh b/conf/examples/2GB/standalone/accumulo-env.sh index a833476..0b29672 100755 --- a/conf/examples/2GB/standalone/accumulo-env.sh +++ b/conf/examples/2GB/standalone/accumulo-env.sh @@ -42,9 +42,9 @@ test -z "$HADOOP_CONF_DIR" && export HADOOP_CONF_DIR="$HADOOP_PREFIX/conf" test -z "$JAVA_HOME" && export JAVA_HOME=/path/to/java test -z "$ZOOKEEPER_HOME" && export ZOOKEEPER_HOME=/path/to/zookeeper test -z "$ACCUMULO_LOG_DIR" && export ACCUMULO_LOG_DIR=$ACCUMULO_HOME/logs -if [ -f ${ACCUMULO_HOME}/conf/accumulo.policy ] +if [ -f ${ACCUMULO_CONF_DIR}/accumulo.policy ] then - POLICY="-Djava.security.manager -Djava.security.policy=${ACCUMULO_HOME}/conf/accumulo.policy" + POLICY="-Djava.security.manager -Djava.security.policy=${ACCUMULO_CONF_DIR}/accumulo.policy" fi test -z "$ACCUMULO_TSERVER_OPTS" && export ACCUMULO_TSERVER_OPTS="${POLICY} -Xmx768m -Xms768m " test -z "$ACCUMULO_MASTER_OPTS" && export ACCUMULO_MASTER_OPTS="${POLICY} -Xmx256m -Xms256m" http://git-wip-us.apache.org/repos/asf/accumulo/blob/c82c4316/conf/examples/3GB/native-standalone/accumulo-env.sh ---------------------------------------------------------------------- diff --git a/conf/examples/3GB/native-standalone/accumulo-env.sh b/conf/examples/3GB/native-standalone/accumulo-env.sh index 5c8fffb..82f4de5 100755 --- a/conf/examples/3GB/native-standalone/accumulo-env.sh +++ b/conf/examples/3GB/native-standalone/accumulo-env.sh @@ -41,9 +41,9 @@ test -z "$HADOOP_CONF_DIR" && export HADOOP_CONF_DIR="$HADOOP_PREFIX/conf" test -z "$JAVA_HOME" && export JAVA_HOME=/path/to/java test -z "$ZOOKEEPER_HOME" && export ZOOKEEPER_HOME=/path/to/zookeeper test -z "$ACCUMULO_LOG_DIR" && export ACCUMULO_LOG_DIR=$ACCUMULO_HOME/logs -if [ -f ${ACCUMULO_HOME}/conf/accumulo.policy ] +if [ -f ${ACCUMULO_CONF_DIR}/accumulo.policy ] then - POLICY="-Djava.security.manager -Djava.security.policy=${ACCUMULO_HOME}/conf/accumulo.policy" + POLICY="-Djava.security.manager -Djava.security.policy=${ACCUMULO_CONF_DIR}/accumulo.policy" fi test -z "$ACCUMULO_TSERVER_OPTS" && export ACCUMULO_TSERVER_OPTS="${POLICY} -Xmx1g -Xms1g -XX:NewSize=500m -XX:MaxNewSize=500m " test -z "$ACCUMULO_MASTER_OPTS" && export ACCUMULO_MASTER_OPTS="${POLICY} -Xmx1g -Xms1g" http://git-wip-us.apache.org/repos/asf/accumulo/blob/c82c4316/conf/examples/3GB/standalone/accumulo-env.sh ---------------------------------------------------------------------- diff --git a/conf/examples/3GB/standalone/accumulo-env.sh b/conf/examples/3GB/standalone/accumulo-env.sh index 04c5791..ec8fae4 100755 --- a/conf/examples/3GB/standalone/accumulo-env.sh +++ b/conf/examples/3GB/standalone/accumulo-env.sh @@ -41,9 +41,9 @@ test -z "$HADOOP_CONF_DIR" && export HADOOP_CONF_DIR="$HADOOP_PREFIX/conf" test -z "$JAVA_HOME" && export JAVA_HOME=/path/to/java test -z "$ZOOKEEPER_HOME" && export ZOOKEEPER_HOME=/path/to/zookeeper test -z "$ACCUMULO_LOG_DIR" && export ACCUMULO_LOG_DIR=$ACCUMULO_HOME/logs -if [ -f ${ACCUMULO_HOME}/conf/accumulo.policy ] +if [ -f ${ACCUMULO_CONF_DIR}/accumulo.policy ] then - POLICY="-Djava.security.manager -Djava.security.policy=${ACCUMULO_HOME}/conf/accumulo.policy" + POLICY="-Djava.security.manager -Djava.security.policy=${ACCUMULO_CONF_DIR}/accumulo.policy" fi test -z "$ACCUMULO_TSERVER_OPTS" && export ACCUMULO_TSERVER_OPTS="${POLICY} -Xmx2g -Xms2g -XX:NewSize=1G -XX:MaxNewSize=1G " test -z "$ACCUMULO_MASTER_OPTS" && export ACCUMULO_MASTER_OPTS="${POLICY} -Xmx1g -Xms1g" http://git-wip-us.apache.org/repos/asf/accumulo/blob/c82c4316/conf/examples/512MB/native-standalone/accumulo-env.sh ---------------------------------------------------------------------- diff --git a/conf/examples/512MB/native-standalone/accumulo-env.sh b/conf/examples/512MB/native-standalone/accumulo-env.sh index 86de0e0..e7f3846 100755 --- a/conf/examples/512MB/native-standalone/accumulo-env.sh +++ b/conf/examples/512MB/native-standalone/accumulo-env.sh @@ -41,9 +41,9 @@ test -z "$HADOOP_CONF_DIR" && export HADOOP_CONF_DIR="$HADOOP_PREFIX/conf" test -z "$JAVA_HOME" && export JAVA_HOME=/path/to/java test -z "$ZOOKEEPER_HOME" && export ZOOKEEPER_HOME=/path/to/zookeeper test -z "$ACCUMULO_LOG_DIR" && export ACCUMULO_LOG_DIR=$ACCUMULO_HOME/logs -if [ -f ${ACCUMULO_HOME}/conf/accumulo.policy ] +if [ -f ${ACCUMULO_CONF_DIR}/accumulo.policy ] then - POLICY="-Djava.security.manager -Djava.security.policy=${ACCUMULO_HOME}/conf/accumulo.policy" + POLICY="-Djava.security.manager -Djava.security.policy=${ACCUMULO_CONF_DIR}/accumulo.policy" fi test -z "$ACCUMULO_TSERVER_OPTS" && export ACCUMULO_TSERVER_OPTS="${POLICY} -Xmx48m -Xms48m " test -z "$ACCUMULO_MASTER_OPTS" && export ACCUMULO_MASTER_OPTS="${POLICY} -Xmx128m -Xms128m" http://git-wip-us.apache.org/repos/asf/accumulo/blob/c82c4316/conf/examples/512MB/standalone/accumulo-env.sh ---------------------------------------------------------------------- diff --git a/conf/examples/512MB/standalone/accumulo-env.sh b/conf/examples/512MB/standalone/accumulo-env.sh index 36987a7..1034632 100755 --- a/conf/examples/512MB/standalone/accumulo-env.sh +++ b/conf/examples/512MB/standalone/accumulo-env.sh @@ -41,9 +41,9 @@ test -z "$HADOOP_CONF_DIR" && export HADOOP_CONF_DIR="$HADOOP_PREFIX/conf" test -z "$JAVA_HOME" && export JAVA_HOME=/path/to/java test -z "$ZOOKEEPER_HOME" && export ZOOKEEPER_HOME=/path/to/zookeeper test -z "$ACCUMULO_LOG_DIR" && export ACCUMULO_LOG_DIR=$ACCUMULO_HOME/logs -if [ -f ${ACCUMULO_HOME}/conf/accumulo.policy ] +if [ -f ${ACCUMULO_CONF_DIR}/accumulo.policy ] then - POLICY="-Djava.security.manager -Djava.security.policy=${ACCUMULO_HOME}/conf/accumulo.policy" + POLICY="-Djava.security.manager -Djava.security.policy=${ACCUMULO_CONF_DIR}/accumulo.policy" fi test -z "$ACCUMULO_TSERVER_OPTS" && export ACCUMULO_TSERVER_OPTS="${POLICY} -Xmx128m -Xms128m " test -z "$ACCUMULO_MASTER_OPTS" && export ACCUMULO_MASTER_OPTS="${POLICY} -Xmx128m -Xms128m" http://git-wip-us.apache.org/repos/asf/accumulo/blob/c82c4316/server/src/main/java/org/apache/accumulo/server/Accumulo.java ---------------------------------------------------------------------- diff --git a/server/src/main/java/org/apache/accumulo/server/Accumulo.java b/server/src/main/java/org/apache/accumulo/server/Accumulo.java index b4f24a3..f8ca31a 100644 --- a/server/src/main/java/org/apache/accumulo/server/Accumulo.java +++ b/server/src/main/java/org/apache/accumulo/server/Accumulo.java @@ -100,10 +100,10 @@ public class Accumulo { System.setProperty("org.apache.accumulo.core.host.log", localhost); // Use a specific log config, if it exists - String logConfig = String.format("%s/conf/%s_logger.xml", System.getenv("ACCUMULO_HOME"), application); + String logConfig = String.format("%s/%s_logger.xml", System.getenv("ACCUMULO_CONF_DIR"), application); if (!new File(logConfig).exists()) { // otherwise, use the generic config - logConfig = String.format("%s/conf/generic_logger.xml", System.getenv("ACCUMULO_HOME")); + logConfig = String.format("%s/generic_logger.xml", System.getenv("ACCUMULO_CONF_DIR")); } // Turn off messages about not being able to reach the remote logger... we protect against that. LogLog.setQuietMode(true); http://git-wip-us.apache.org/repos/asf/accumulo/blob/c82c4316/start/src/main/java/org/apache/accumulo/start/classloader/AccumuloClassLoader.java ---------------------------------------------------------------------- diff --git a/start/src/main/java/org/apache/accumulo/start/classloader/AccumuloClassLoader.java b/start/src/main/java/org/apache/accumulo/start/classloader/AccumuloClassLoader.java index 2cac8d9..594ad8f 100644 --- a/start/src/main/java/org/apache/accumulo/start/classloader/AccumuloClassLoader.java +++ b/start/src/main/java/org/apache/accumulo/start/classloader/AccumuloClassLoader.java @@ -46,7 +46,7 @@ public class AccumuloClassLoader { public static final String CLASSPATH_PROPERTY_NAME = "general.classpaths"; public static final String ACCUMULO_CLASSPATH_VALUE = - "$ACCUMULO_HOME/conf,\n" + + "$ACCUMULO_CONF_DIR,\n" + "$ACCUMULO_HOME/lib/[^.].*.jar,\n" + "$ZOOKEEPER_HOME/zookeeper[^.].*.jar,\n" + "$HADOOP_CONF_DIR,\n" + @@ -66,8 +66,11 @@ public class AccumuloClassLoader { static { String configFile = System.getProperty("org.apache.accumulo.config.file", "accumulo-site.xml"); - if (System.getenv("ACCUMULO_HOME") != null) { - // accumulo home should be set + if (System.getenv("ACCUMULO_CONF_DIR") != null) { + // accumulo conf dir should be set + SITE_CONF = System.getenv("ACCUMULO_CONF_DIR"); + } else if (System.getenv("ACCUMULO_HOME") != null) { + // if no accumulo conf dir, try accumulo home default SITE_CONF = System.getenv("ACCUMULO_HOME") + "/conf/" + configFile; } else { SITE_CONF = null; http://git-wip-us.apache.org/repos/asf/accumulo/blob/c82c4316/test/system/continuous/agitator.pl ---------------------------------------------------------------------- diff --git a/test/system/continuous/agitator.pl b/test/system/continuous/agitator.pl index f769942..d9c5395 100755 --- a/test/system/continuous/agitator.pl +++ b/test/system/continuous/agitator.pl @@ -64,7 +64,7 @@ if($minKill > $maxKill){ die("minKill > maxKill $minKill > $maxKill"); } -@slavesRaw = `cat $ACCUMULO_HOME/conf/slaves`; +@slavesRaw = `cat $ACCUMULO_CONF_DIR/slaves`; chomp(@slavesRaw); for $slave (@slavesRaw){ http://git-wip-us.apache.org/repos/asf/accumulo/blob/c82c4316/test/system/continuous/magitator.pl ---------------------------------------------------------------------- diff --git a/test/system/continuous/magitator.pl b/test/system/continuous/magitator.pl index 8b22505..a0cab60 100755 --- a/test/system/continuous/magitator.pl +++ b/test/system/continuous/magitator.pl @@ -29,7 +29,7 @@ $ACCUMULO_HOME="../../.."; $sleep1 = $ARGV[0]; $sleep2 = $ARGV[1]; -@mastersRaw = `cat $ACCUMULO_HOME/conf/masters`; +@mastersRaw = `cat $ACCUMULO_CONF_DIR/masters`; chomp(@mastersRaw); for $master (@mastersRaw){ @@ -52,10 +52,10 @@ while(1){ system($cmd); }else{ print STDERR "$t Killing all masters\n"; - $cmd = "pssh -h $ACCUMULO_HOME/conf/masters \"pkill -f '[ ]org.apache.accumulo.start.*master'\" < /dev/null"; + $cmd = "pssh -h $ACCUMULO_CONF_DIR/masters \"pkill -f '[ ]org.apache.accumulo.start.*master'\" < /dev/null"; print "$t $cmd\n"; system($cmd); - $cmd = "pssh -h $ACCUMULO_HOME/conf/masters \"pkill -f '[ ]org.apache.accumulo.start.*gc'\" < /dev/null"; + $cmd = "pssh -h $ACCUMULO_CONF_DIR/masters \"pkill -f '[ ]org.apache.accumulo.start.*gc'\" < /dev/null"; print "$t $cmd\n"; system($cmd); } http://git-wip-us.apache.org/repos/asf/accumulo/blob/c82c4316/test/system/continuous/mapred-setup.sh ---------------------------------------------------------------------- diff --git a/test/system/continuous/mapred-setup.sh b/test/system/continuous/mapred-setup.sh index aa2b11b..c348131 100755 --- a/test/system/continuous/mapred-setup.sh +++ b/test/system/continuous/mapred-setup.sh @@ -19,6 +19,6 @@ # for running a map reduce job . continuous-env.sh -. $ACCUMULO_HOME/conf/accumulo-env.sh +. $ACCUMULO_CONF_DIR/accumulo-env.sh SERVER_LIBJAR="$ACCUMULO_HOME/lib/accumulo-test.jar" http://git-wip-us.apache.org/repos/asf/accumulo/blob/c82c4316/test/system/continuous/start-stats.sh ---------------------------------------------------------------------- diff --git a/test/system/continuous/start-stats.sh b/test/system/continuous/start-stats.sh index 4e42930..3aad413 100755 --- a/test/system/continuous/start-stats.sh +++ b/test/system/continuous/start-stats.sh @@ -22,7 +22,7 @@ mkdir -p $CONTINUOUS_LOG_DIR CONFIG_OUT=$CONTINUOUS_LOG_DIR/`date +%Y%m%d%H%M%S`_`hostname`_config.out -cat $ACCUMULO_HOME/conf/accumulo-env.sh > $CONFIG_OUT +cat $ACCUMULO_CONF_DIR/accumulo-env.sh > $CONFIG_OUT echo >> $CONFIG_OUT echo -e "config -np\nconfig -t $TABLE -np\nquit" | $ACCUMULO_HOME/bin/accumulo shell -u $USER -p $PASS >> $CONFIG_OUT echo >> $CONFIG_OUT http://git-wip-us.apache.org/repos/asf/accumulo/blob/c82c4316/test/system/randomwalk/README ---------------------------------------------------------------------- diff --git a/test/system/randomwalk/README b/test/system/randomwalk/README index 0c33310..78a50c6 100644 --- a/test/system/randomwalk/README +++ b/test/system/randomwalk/README @@ -4,7 +4,7 @@ The randomwalk framework needs to be configured for your Accumulo instance by doing the following steps: 1. Make sure you have both ACCUMULO_HOME and HADOOP_HOME set in your - $ACCUMULO_HOME/conf/accumulo-env.sh. + $ACCUMULO_CONF_DIR/accumulo-env.sh. 2. Create 'randomwalk.conf' file in the conf directory containing settings needed by walkers to connect to Accumulo. http://git-wip-us.apache.org/repos/asf/accumulo/blob/c82c4316/test/system/randomwalk/bin/reset-cluster.sh ---------------------------------------------------------------------- diff --git a/test/system/randomwalk/bin/reset-cluster.sh b/test/system/randomwalk/bin/reset-cluster.sh index 15d791e..b5beb65 100755 --- a/test/system/randomwalk/bin/reset-cluster.sh +++ b/test/system/randomwalk/bin/reset-cluster.sh @@ -34,8 +34,8 @@ this="$bin/$script" ACCUMULO_HOME=`dirname "$this"`/../../../.. export ACCUMULO_HOME=`cd $ACCUMULO_HOME; pwd` -if [ -f $ACCUMULO_HOME/conf/accumulo-env.sh ] ; then -. $ACCUMULO_HOME/conf/accumulo-env.sh +if [ -f $ACCUMULO_CONF_DIR/accumulo-env.sh ] ; then +. $ACCUMULO_CONF_DIR/accumulo-env.sh fi if [ -z $HADOOP_PREFIX ] ; then @@ -49,7 +49,7 @@ if [ "$1" = "" ] ; then fi echo 'killing accumulo' -pssh -h $ACCUMULO_HOME/conf/slaves "pkill -f org.apache.accumulo.start" < /dev/null +pssh -h $ACCUMULO_CONF_DIR/slaves "pkill -f org.apache.accumulo.start" < /dev/null pkill -f org.apache.accumulo.start pkill -f agitator.pl @@ -64,10 +64,10 @@ rm -f $ACCUMULO_HOME/test/system/continuous/logs/* rm -f ~/rwlogs/* echo 'removing old code' -pssh -h $ACCUMULO_HOME/conf/slaves "rm -rf $ACCUMULO_HOME" < /dev/null +pssh -h $ACCUMULO_CONF_DIR/slaves "rm -rf $ACCUMULO_HOME" < /dev/null echo 'pushing new code' -prsync -r -h $ACCUMULO_HOME/conf/slaves $ACCUMULO_HOME /opt/dev +prsync -r -h $ACCUMULO_CONF_DIR/slaves $ACCUMULO_HOME /opt/dev echo 'removing /accumulo dir' $HADOOP_PREFIX/bin/hadoop fs -rmr /accumulo http://git-wip-us.apache.org/repos/asf/accumulo/blob/c82c4316/test/system/randomwalk/bin/start-all.sh ---------------------------------------------------------------------- diff --git a/test/system/randomwalk/bin/start-all.sh b/test/system/randomwalk/bin/start-all.sh index 217c3b4..de9a9f1 100755 --- a/test/system/randomwalk/bin/start-all.sh +++ b/test/system/randomwalk/bin/start-all.sh @@ -34,8 +34,8 @@ this="$bin/$script" ACCUMULO_HOME=`dirname "$this"`/../../../.. export ACCUMULO_HOME=`cd $ACCUMULO_HOME; pwd` -if [ -f $ACCUMULO_HOME/conf/accumulo-env.sh ] ; then -. $ACCUMULO_HOME/conf/accumulo-env.sh +if [ -f $ACCUMULO_CONF_DIR/accumulo-env.sh ] ; then +. $ACCUMULO_CONF_DIR/accumulo-env.sh fi http://git-wip-us.apache.org/repos/asf/accumulo/blob/c82c4316/test/system/randomwalk/bin/start-local.sh ---------------------------------------------------------------------- diff --git a/test/system/randomwalk/bin/start-local.sh b/test/system/randomwalk/bin/start-local.sh index 262a37c..ebb1e92 100755 --- a/test/system/randomwalk/bin/start-local.sh +++ b/test/system/randomwalk/bin/start-local.sh @@ -35,8 +35,8 @@ this="$bin/$script" ACCUMULO_HOME=`dirname "$this"`/../../../.. export ACCUMULO_HOME=`cd $ACCUMULO_HOME; pwd` -if [ -f $ACCUMULO_HOME/conf/accumulo-env.sh ] ; then -. $ACCUMULO_HOME/conf/accumulo-env.sh +if [ -f $ACCUMULO_CONF_DIR/accumulo-env.sh ] ; then +. $ACCUMULO_CONF_DIR/accumulo-env.sh fi if [ -z "$HADOOP_PREFIX" ] ; then http://git-wip-us.apache.org/repos/asf/accumulo/blob/c82c4316/test/system/scalability/run.py ---------------------------------------------------------------------- diff --git a/test/system/scalability/run.py b/test/system/scalability/run.py index bca64c1..ad98d63 100755 --- a/test/system/scalability/run.py +++ b/test/system/scalability/run.py @@ -57,7 +57,7 @@ def runTest(testName, siteConfig, testDir, numNodes, fdata): syscall('head -n %d %s > %s' % (numNodes,slavesPath,nodesPath)) log('Copying slaves file to accumulo config') - syscall('cp '+nodesPath+' $ACCUMULO_HOME/conf/slaves'); + syscall('cp '+nodesPath+' $ACCUMULO_CONF_DIR/slaves'); log('Removing /accumulo directory in HDFS') syscall("hadoop fs -rmr /accumulo")