fluo-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ktur...@apache.org
Subject incubator-fluo git commit: fixes #247 Made Accumulo, Hadoop, and ZK jars configurable in tarball
Date Mon, 27 Jun 2016 22:20:15 GMT
Repository: incubator-fluo
Updated Branches:
  refs/heads/master 2b2d784fc -> 6fd86298c


fixes #247 Made Accumulo,Hadoop,and ZK jars configurable in tarball


Project: http://git-wip-us.apache.org/repos/asf/incubator-fluo/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-fluo/commit/6fd86298
Tree: http://git-wip-us.apache.org/repos/asf/incubator-fluo/tree/6fd86298
Diff: http://git-wip-us.apache.org/repos/asf/incubator-fluo/diff/6fd86298

Branch: refs/heads/master
Commit: 6fd86298c7d6c4759df98e9d6633fe91ec036f99
Parents: 2b2d784
Author: Keith Turner <kturner@apache.org>
Authored: Fri Jun 17 19:50:59 2016 -0400
Committer: Keith Turner <kturner@apache.org>
Committed: Mon Jun 27 18:16:37 2016 -0400

----------------------------------------------------------------------
 README.md                                       |   2 -
 docs/mini-fluo-setup.md                         | 143 -----------------
 docs/prod-fluo-setup.md                         |  29 ++--
 .../fluo/cluster/command/FluoCommand.java       |   5 +-
 .../fluo/cluster/command/MiniFluoCommand.java   | 123 --------------
 .../apache/fluo/cluster/runner/AppRunner.java   |  62 -------
 .../fluo/cluster/runner/ClasspathOptions.java   |  54 -------
 .../fluo/cluster/runner/MiniAppRunner.java      |  44 -----
 .../fluo/cluster/runner/it/MiniAppRunnerIT.java |  89 -----------
 modules/distribution/pom.xml                    |  15 +-
 modules/distribution/src/main/assembly/bin.xml  |  80 +++-------
 .../distribution/src/main/config/fluo-env.sh    |  62 +++++++
 modules/distribution/src/main/deps/README.md    |  14 ++
 modules/distribution/src/main/deps/pom.xml      |  64 ++++++++
 modules/distribution/src/main/scripts/fluo      |  23 +--
 .../src/main/scripts/impl/config.sh             |   8 +-
 .../distribution/src/main/scripts/local-fluo    |  10 +-
 modules/distribution/src/main/scripts/mini-fluo | 160 -------------------
 18 files changed, 207 insertions(+), 780 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/incubator-fluo/blob/6fd86298/README.md
----------------------------------------------------------------------
diff --git a/README.md b/README.md
index 0e89697..5488013 100644
--- a/README.md
+++ b/README.md
@@ -14,7 +14,6 @@ news and general information.
 There are several ways to run Fluo (listed in order of increasing difficulty):
 
 * [quickstart] - Starts a MiniFluo instance that is configured to run a word count application
-* [MiniFluo] - Sets up a minimal Fluo instance that writes its data to single directory
 * [fluo-dev] - Command-line tool for running Fluo and its dependencies on a single machine
 * [Zetten] - Command-line tool that launches an AWS cluster and sets up Fluo/Accumulo on it
 * [Production] - Sets up Fluo on a cluster where Accumulo, Hadoop & Zookeeper are running
@@ -51,7 +50,6 @@ Below are helpful resources for Fluo application developers:
 [phrasecount]: https://github.com/fluo-io/phrasecount
 [fluo-stress]: https://github.com/fluo-io/fluo-stress
 [webindex]: https://github.com/fluo-io/webindex
-[MiniFluo]: docs/mini-fluo-setup.md
 [Production]: docs/prod-fluo-setup.md
 [apps]: docs/applications.md
 [api]: https://fluo.apache.org/apidocs/

http://git-wip-us.apache.org/repos/asf/incubator-fluo/blob/6fd86298/docs/mini-fluo-setup.md
----------------------------------------------------------------------
diff --git a/docs/mini-fluo-setup.md b/docs/mini-fluo-setup.md
deleted file mode 100644
index 3ee3f56..0000000
--- a/docs/mini-fluo-setup.md
+++ /dev/null
@@ -1,143 +0,0 @@
-MiniFluo Setup
-==============
-
-Below are instructions for running a test/development instance of Fluo
-called MiniFluo.  MiniFluo runs its own Accumulo and Zookeeper but can
-only run on one machine.  While MiniFluo is easy to set up and has 
-all of the features of Fluo, it is not recommended for production use as 
-all of its data is lost when its stopped.
-
-If you looking to run Fluo in a production environment, read the
-[production setup instructions][prod-fluo].
-
-Obtaining a distribution
-------------------------
-
-Before you can install MiniFluo, you will need to obtain a distribution tarball.  You
-can either download the [latest release][release] or build a distribution tarball
-by following these steps:
-
-First, clone Fluo:
-
-    git clone https://github.com/apache/incubator-fluo.git
-    cd fluo/
-
-Optionally, check out a stable tag (if you don't want to build a release from master):
-
-    git checkout 1.0.0-beta-2
-
-Next, build a distribution which will be created in `modules/distribution/target`.
-
-    mvn package
-
-Install Fluo distribution
--------------------------
-
-When you have a distribution tarball built for your environment, follow these steps
-to install Fluo.
-
-First, choose a directory with plenty of space and untar the distribution:
-
-    tar -xvzf fluo-1.0.0-beta-2-bin.tar.gz
-
-Verify that your distribution has the same versions of Hadoop & Accumulo as your environment:
-
-    cd fluo-1.0.0-beta-2
-    ls lib/hadoop-* lib/accumulo-*
-
-Next, copy the example configuration to the base of your configuration directory to create
-the default configuration for your Fluo install:
-
-    cp conf/examples/* conf/
-
-The default configuration will be used as the base configuration for each new application. 
-Therefore, you should modify [fluo.properties] for your environment. However, you should
-not configure any application settings (like observers).
- 
-As you are running a MiniFluo instance, most properties in [fluo.properties] do not need to be 
-set by you. For example, all "Client properties" can be left unset as MiniFluo will start up 
-its own Accumulo and Zookeeper.  If you would rather have MiniFluo connect to an existing 
-Accumulo instance, change `org.apache.fluo.mini.start.accumulo` to `false` and set all "Client properties".
-If you have an existing Accumulo instance, you should also consider running a Fluo instance by 
-following the [production setup instructions][prod-fluo].
-
-MiniFluo command script
------------------------
-
-The MiniFluo command script is located at `bin/mini-fluo` of your Fluo installation.  All
-MiniFluo commands are invoked by this script.
-
-Modify and add the following to your `~/.bashrc` if you want to be able to execute the
-`mini-fluo` script from any directory:
-
-    export PATH=/path/to/fluo-1.0.0-beta-2/bin:$PATH
-
-Source your `.bashrc` for the changes to take effect and test the script
-
-    source ~/.bashrc
-    mini-fluo
-
-Running the script without any arguments prints a description of all commands.
-
-Configure a MiniFluo application
---------------------------------
-
-You are now ready to configure a MiniFluo application.  Use the command below to create the
-configuration necessary for a new application.  Feel free to pick a different name (other
-than `myapp`) for your application:
-
-    mini-fluo new myapp
-
-This command will create a directory for your application at `apps/myapp` of your Fluo
-install which will contain a `conf` and `lib`.
-
-The `apps/myapp/conf` directory contains a copy of the `fluo.properties` from your default
-configuration.  This should be configured for your application:
-
-    vim apps/myapp/fluo.properties
-
-When configuring the observer section in [fluo.properties], you can configure your application
-to run the [phrasecount] example if you have not developed your own application. See
-the [phrasecount] example for instructions. You can also choose not to configure any
-observers but MiniFluo will be idle when started.
-
-The `apps/myapp/lib` directory should contain any observer jars for your application. If 
-you configured [fluo.properties] for observers, copy any jars containing these
-observer classes this directory.
- 
-Run a MiniFluo application
---------------------------
-
-A MiniFluo application can be started with the following command:
-
-    mini-fluo start myapp
-
-The application will output its logs to `app/myapp/logs/` directory of your Fluo installation.
-
-MiniFluo starts its own cluster of Accumulo and Zookeeper.  All data for this
-cluster is written by default to `app/myapp/mini/` directory of your Fluo installation 
-but this can be configured in `fluo.properties`.  Logs for underlying cluster
-can be viewed in `mini/logs`.
-
-Due to MiniFluo starting its own cluster, it generates and writes a `client.properties`
-file to its data directory.  This file can be used by Fluo clients to connect
-to the MiniFluo instance.
-
-When you have data in your fluo instance, you can view it using the command `mini-fluo scan myapp`.
-Pipe the output to `less` using the command `mini-fluo scan myapp | less` if you want to page 
-through the data.
-
-Stop a MiniFluo application
----------------------------
-
-A MiniFluo application can be stopped by running the following command:
-
-    mini-fluo stop myapp
-
-If MiniFluo started its own Accumulo cluster, it will also remove the `mini/` data 
-directory and any data in your instance will be lost.
-
-[prod-fluo]: prod-fluo-setup.md
-[release]: https://github.com/apache/incubator-fluo/releases
-[phrasecount]: https://github.com/fluo-io/phrasecount
-[fluo.properties]: ../modules/distribution/src/main/config/fluo.properties

http://git-wip-us.apache.org/repos/asf/incubator-fluo/blob/6fd86298/docs/prod-fluo-setup.md
----------------------------------------------------------------------
diff --git a/docs/prod-fluo-setup.md b/docs/prod-fluo-setup.md
index 0a8f3e1..dd633a0 100644
--- a/docs/prod-fluo-setup.md
+++ b/docs/prod-fluo-setup.md
@@ -3,7 +3,7 @@ Production Setup
 
 Below are instructions for running Fluo in a production environment where Accumulo,
 Hadoop & Zookeeper are installed and running.  If you want to avoid setting up
-these dependencies, consider using [fluo-dev] or [MiniFluo]. 
+these dependencies, consider using [fluo-dev]. 
 
 Requirements
 ------------
@@ -19,25 +19,14 @@ your local machine or cluster:
 Obtain a distribution
 ---------------------
 
-Before you can install Fluo, you will need to obtain a distribution tarball that
-works for your environment. Fluo distributions are built for specific releases
-of Hadoop and Accumulo. If you are using Accumulo 1.7.1 and Hadoop 2.6.3,
-you can download the [latest release][release].  If you need a release for a
-different environment or one built from the master branch, follow these steps:
-
-First, clone Fluo:
+Before you can install Fluo, you will need to obtain a distribution tarball.  It is
+recommended that you download the [latest release][release].  You can also build
+a distribution from the master branch by following these steps which create a tarball
+in `modules/distribution/target`:
 
     git clone https://github.com/apache/incubator-fluo.git
     cd fluo/
-
-Optionally, check out a stable tag (if you don't want to build a release from master):
-
-    git checkout 1.0.0-beta-2
-
-Next, build a distribution for your environment. The tarball will be created in 
-`modules/distribution/target`.
-
-    mvn package -Daccumulo.version=1.7.1 -Dhadoop.version=2.7.1
+    mvn package
 
 Install Fluo
 ------------
@@ -67,6 +56,11 @@ NOTE - All properties that have a default are set with it.  Uncomment a property
 to use a value different than the default.  Properties that are unset and uncommented must be
 set by the user.
 
+To run, Fluo requires the Hadoop, Accumulo, and Zookeeper jars for the versions
+running on the cluster where Fluo will run.  In order to configure this a
+classpath must be constructed with these jars for Fluo.  This can be done by
+editing `conf/fluo-env.sh`, which also contains further documentation.  
+
 Fluo command script
 -------------------
 
@@ -208,7 +202,6 @@ Use the following commands to stop a local Fluo process:
 In a distributed environment, you will need to deploy and configure a Fluo 
 distribution on every node in your cluster.
 
-[MiniFluo]: mini-fluo-setup.md
 [fluo-dev]: https://github.com/fluo-io/fluo-dev
 [Accumulo]: https://accumulo.apache.org/
 [Hadoop]: http://hadoop.apache.org/

http://git-wip-us.apache.org/repos/asf/incubator-fluo/blob/6fd86298/modules/cluster/src/main/java/org/apache/fluo/cluster/command/FluoCommand.java
----------------------------------------------------------------------
diff --git a/modules/cluster/src/main/java/org/apache/fluo/cluster/command/FluoCommand.java b/modules/cluster/src/main/java/org/apache/fluo/cluster/command/FluoCommand.java
index dbf5939..24852f1 100644
--- a/modules/cluster/src/main/java/org/apache/fluo/cluster/command/FluoCommand.java
+++ b/modules/cluster/src/main/java/org/apache/fluo/cluster/command/FluoCommand.java
@@ -61,9 +61,6 @@ public class FluoCommand {
 
     try (YarnAppRunner runner = new YarnAppRunner(hadoopPrefix)) {
       switch (command.toLowerCase()) {
-        case "classpath":
-          runner.classpath(fluoHomeDir, remainArgs);
-          break;
         case "init":
           runner.init(fluoInstall.getAppConfiguration(appName),
               fluoInstall.getAppPropsPath(appName), remainArgs);
@@ -102,7 +99,7 @@ public class FluoCommand {
           runner.waitUntilFinished(fluoInstall.resolveFluoConfiguration(appName));
           break;
         case "exec":
-          runner.exec(fluoInstall.resolveFluoConfiguration(appName), remainArgs);
+          runner.exec(fluoInstall.resolveFluoConfiguration(appName, false), remainArgs);
           break;
         default:
           System.err.println("Unknown command: " + command);

http://git-wip-us.apache.org/repos/asf/incubator-fluo/blob/6fd86298/modules/cluster/src/main/java/org/apache/fluo/cluster/command/MiniFluoCommand.java
----------------------------------------------------------------------
diff --git a/modules/cluster/src/main/java/org/apache/fluo/cluster/command/MiniFluoCommand.java b/modules/cluster/src/main/java/org/apache/fluo/cluster/command/MiniFluoCommand.java
deleted file mode 100644
index 713c3fe..0000000
--- a/modules/cluster/src/main/java/org/apache/fluo/cluster/command/MiniFluoCommand.java
+++ /dev/null
@@ -1,123 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more contributor license
- * agreements. See the NOTICE file distributed with this work for additional information regarding
- * copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance with the License. You may obtain a
- * copy of the License at
- * 
- * http://www.apache.org/licenses/LICENSE-2.0
- * 
- * Unless required by applicable law or agreed to in writing, software distributed under the License
- * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
- * or implied. See the License for the specific language governing permissions and limitations under
- * the License.
- */
-
-package org.apache.fluo.cluster.command;
-
-import java.io.File;
-import java.util.Arrays;
-
-import ch.qos.logback.classic.Level;
-import ch.qos.logback.classic.Logger;
-import org.apache.fluo.api.config.FluoConfiguration;
-import org.apache.fluo.cluster.runner.MiniAppRunner;
-import org.apache.fluo.cluster.util.FluoInstall;
-import org.apache.fluo.mini.MiniFluoImpl;
-import org.slf4j.LoggerFactory;
-
-public class MiniFluoCommand {
-
-  static FluoInstall fluoInstall;
-
-  public static void verifyNoArgs(String[] remainArgs) {
-    if (remainArgs.length != 0) {
-      System.err.println("ERROR - Received unexpected command-line arguments: "
-          + Arrays.toString(remainArgs));
-      System.exit(-1);
-    }
-  }
-
-  public static FluoConfiguration chooseConfig(String appName) {
-
-    FluoConfiguration chosenConfig = null;
-    String propsPath;
-    FluoConfiguration fluoConfig = fluoInstall.resolveFluoConfiguration(appName, false);
-    if (fluoConfig.hasRequiredClientProps()) {
-      chosenConfig = fluoConfig;
-      propsPath = fluoInstall.getFluoPropsPath();
-    } else {
-      File miniDataDir = new File(fluoConfig.getMiniDataDir());
-      if (!miniDataDir.exists()) {
-        System.err.println("Cannot connect to Fluo '" + fluoConfig.getApplicationName()
-            + "' application!  Client properties are not set in fluo.properties and "
-            + " a MiniAccumuloCluster is not running at " + miniDataDir.getAbsolutePath());
-        System.exit(-1);
-      }
-      if (!fluoConfig.hasRequiredMiniFluoProps()) {
-        System.err.println("Fluo properties are not configured correctly!");
-        System.exit(-1);
-      }
-      propsPath = MiniFluoImpl.clientPropsPath(fluoConfig);
-      File clientPropsFile = new File(propsPath);
-      if (!clientPropsFile.exists()) {
-        System.err.println("MiniFluo client.properties do not exist at "
-            + clientPropsFile.getAbsolutePath());
-        System.exit(-1);
-      }
-      chosenConfig = new FluoConfiguration(clientPropsFile);
-    }
-
-    System.out.println("Connecting to MiniFluo instance (" + chosenConfig.getInstanceZookeepers()
-        + ") using config (" + fluoInstall.stripFluoHomeDir(propsPath) + ")");
-    return chosenConfig;
-  }
-
-  public static void main(String[] args) throws Exception {
-
-    if (args.length < 3) {
-      System.err.println("ERROR - Expected at least two arguments.  "
-          + "Usage: MiniFluoCommand <fluoHomeDir> <command> <appName> ...");
-      System.exit(-1);
-    }
-
-    String fluoHomeDir = args[0];
-    String command = args[1];
-    String appName = args[2];
-    String[] remainArgs = Arrays.copyOfRange(args, 3, args.length);
-
-    if (command.equalsIgnoreCase("scan")) {
-      for (String logger : new String[] {Logger.ROOT_LOGGER_NAME, "org.apache.fluo"}) {
-        ((Logger) LoggerFactory.getLogger(logger)).setLevel(Level.ERROR);
-      }
-    } else if (command.equalsIgnoreCase("wait")) {
-      ((Logger) LoggerFactory.getLogger(FluoConfiguration.class)).setLevel(Level.ERROR);
-    }
-
-    fluoInstall = new FluoInstall(fluoHomeDir);
-    MiniAppRunner runner = new MiniAppRunner();
-
-    switch (command.toLowerCase()) {
-      case "classpath":
-        runner.classpath(fluoHomeDir, remainArgs);
-        break;
-      case "scan":
-        runner.scan(chooseConfig(appName), remainArgs);
-        break;
-      case "cleanup":
-        verifyNoArgs(remainArgs);
-        runner.cleanup(fluoInstall.resolveFluoConfiguration(appName, false));
-        break;
-      case "wait":
-        verifyNoArgs(remainArgs);
-        runner.waitUntilFinished(chooseConfig(appName));
-        break;
-      case "exec":
-        runner.exec(chooseConfig(appName), remainArgs);
-        break;
-      default:
-        System.err.println("Unknown command: " + command);
-        break;
-    }
-  }
-}

http://git-wip-us.apache.org/repos/asf/incubator-fluo/blob/6fd86298/modules/cluster/src/main/java/org/apache/fluo/cluster/runner/AppRunner.java
----------------------------------------------------------------------
diff --git a/modules/cluster/src/main/java/org/apache/fluo/cluster/runner/AppRunner.java b/modules/cluster/src/main/java/org/apache/fluo/cluster/runner/AppRunner.java
index c906ac5..ac0da34 100644
--- a/modules/cluster/src/main/java/org/apache/fluo/cluster/runner/AppRunner.java
+++ b/modules/cluster/src/main/java/org/apache/fluo/cluster/runner/AppRunner.java
@@ -15,7 +15,6 @@
 
 package org.apache.fluo.cluster.runner;
 
-import java.io.File;
 import java.lang.reflect.Method;
 import java.util.Arrays;
 import java.util.Map;
@@ -234,67 +233,6 @@ public abstract class AppRunner {
     return entriesFound;
   }
 
-  private static void appendLib(StringBuilder classpath, String libDirName, boolean useLibJarsFormat) {
-    File libDir = new File(libDirName);
-    if (!libDir.exists()) {
-      System.err.println("ERROR - Directory needed for classpath does not exist: " + libDirName);
-      System.exit(-1);
-    }
-
-    if (useLibJarsFormat) {
-      File[] files = libDir.listFiles();
-      if (files != null) {
-        Arrays.sort(files);
-        for (File f : files) {
-          if (f.isFile() && f.getName().endsWith(".jar")) {
-            if (classpath.length() != 0) {
-              classpath.append(",");
-            }
-            classpath.append(f.getAbsolutePath());
-          }
-        }
-      }
-    } else {
-      if (classpath.length() != 0) {
-        classpath.append(":");
-      }
-      classpath.append(libDir.getAbsolutePath()).append("/*");
-    }
-  }
-
-  public void classpath(String fluoHomeDir, String[] args) {
-    ClasspathOptions options = new ClasspathOptions();
-    JCommander jcommand = new JCommander(options);
-    jcommand.setProgramName(scriptName + " classpath");
-    try {
-      jcommand.parse(args);
-    } catch (ParameterException e) {
-      System.err.println(e.getMessage());
-      jcommand.usage();
-      System.exit(-1);
-    }
-
-    if (options.help) {
-      jcommand.usage();
-      System.exit(0);
-    }
-
-    StringBuilder classpath = new StringBuilder();
-
-    appendLib(classpath, fluoHomeDir + "/lib/fluo-client", options.getLibJars());
-    if (options.getAccumulo()) {
-      appendLib(classpath, fluoHomeDir + "/lib/accumulo", options.getLibJars());
-    }
-    if (options.getZookeepers()) {
-      appendLib(classpath, fluoHomeDir + "/lib/zookeeper", options.getLibJars());
-    }
-    if (options.getHadoop()) {
-      appendLib(classpath, fluoHomeDir + "/lib/hadoop-client", options.getLibJars());
-    }
-
-    System.out.println(classpath.toString());
-  }
-
   private long calculateSleep(long notifyCount, long numWorkers) {
     long sleep = notifyCount / numWorkers / 100;
     if (sleep < MIN_SLEEP_SEC) {

http://git-wip-us.apache.org/repos/asf/incubator-fluo/blob/6fd86298/modules/cluster/src/main/java/org/apache/fluo/cluster/runner/ClasspathOptions.java
----------------------------------------------------------------------
diff --git a/modules/cluster/src/main/java/org/apache/fluo/cluster/runner/ClasspathOptions.java b/modules/cluster/src/main/java/org/apache/fluo/cluster/runner/ClasspathOptions.java
deleted file mode 100644
index 61727bb..0000000
--- a/modules/cluster/src/main/java/org/apache/fluo/cluster/runner/ClasspathOptions.java
+++ /dev/null
@@ -1,54 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more contributor license
- * agreements. See the NOTICE file distributed with this work for additional information regarding
- * copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance with the License. You may obtain a
- * copy of the License at
- * 
- * http://www.apache.org/licenses/LICENSE-2.0
- * 
- * Unless required by applicable law or agreed to in writing, software distributed under the License
- * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
- * or implied. See the License for the specific language governing permissions and limitations under
- * the License.
- */
-
-package org.apache.fluo.cluster.runner;
-
-import com.beust.jcommander.Parameter;
-
-public class ClasspathOptions {
-
-  @Parameter(names = {"-a", "--accumulo"}, description = "Additionally prints Accumulo client jars")
-  private boolean accumulo = false;
-
-  @Parameter(names = {"-z", "--zookeeper"},
-      description = "Additionally prints Zookeeper client jars")
-  private boolean zookeeper = false;
-
-  @Parameter(names = {"-H", "--hadoop"}, description = "Additionally prints Hadoop client jars")
-  private boolean hadoop = false;
-
-  @Parameter(names = {"-l", "--lib-jars"},
-      description = "Prints classpath in format suitable for Hadoop -libjars")
-  private boolean libJars = false;
-
-  @Parameter(names = {"-h", "-help", "--help"}, help = true, description = "Prints help")
-  public boolean help;
-
-  public boolean getAccumulo() {
-    return accumulo;
-  }
-
-  public boolean getHadoop() {
-    return hadoop;
-  }
-
-  public boolean getZookeepers() {
-    return zookeeper;
-  }
-
-  public boolean getLibJars() {
-    return libJars;
-  }
-}

http://git-wip-us.apache.org/repos/asf/incubator-fluo/blob/6fd86298/modules/cluster/src/main/java/org/apache/fluo/cluster/runner/MiniAppRunner.java
----------------------------------------------------------------------
diff --git a/modules/cluster/src/main/java/org/apache/fluo/cluster/runner/MiniAppRunner.java b/modules/cluster/src/main/java/org/apache/fluo/cluster/runner/MiniAppRunner.java
deleted file mode 100644
index bbffa37..0000000
--- a/modules/cluster/src/main/java/org/apache/fluo/cluster/runner/MiniAppRunner.java
+++ /dev/null
@@ -1,44 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more contributor license
- * agreements. See the NOTICE file distributed with this work for additional information regarding
- * copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance with the License. You may obtain a
- * copy of the License at
- * 
- * http://www.apache.org/licenses/LICENSE-2.0
- * 
- * Unless required by applicable law or agreed to in writing, software distributed under the License
- * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
- * or implied. See the License for the specific language governing permissions and limitations under
- * the License.
- */
-
-package org.apache.fluo.cluster.runner;
-
-import java.io.File;
-import java.io.IOException;
-
-import org.apache.commons.io.FileUtils;
-import org.apache.fluo.api.config.FluoConfiguration;
-
-/**
- * Runs applications in MiniFluo
- */
-public class MiniAppRunner extends AppRunner {
-
-  public MiniAppRunner() {
-    super("mini-fluo");
-  }
-
-  public void cleanup(FluoConfiguration appConfig) {
-    File dataDir = new File(appConfig.getMiniDataDir());
-    if (dataDir.exists() && appConfig.getMiniStartAccumulo()) {
-      try {
-        FileUtils.deleteDirectory(dataDir);
-      } catch (IOException e) {
-        throw new IllegalStateException(e);
-      }
-    }
-  }
-
-}

http://git-wip-us.apache.org/repos/asf/incubator-fluo/blob/6fd86298/modules/cluster/src/test/java/org/apache/fluo/cluster/runner/it/MiniAppRunnerIT.java
----------------------------------------------------------------------
diff --git a/modules/cluster/src/test/java/org/apache/fluo/cluster/runner/it/MiniAppRunnerIT.java b/modules/cluster/src/test/java/org/apache/fluo/cluster/runner/it/MiniAppRunnerIT.java
deleted file mode 100644
index 12948c3..0000000
--- a/modules/cluster/src/test/java/org/apache/fluo/cluster/runner/it/MiniAppRunnerIT.java
+++ /dev/null
@@ -1,89 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more contributor license
- * agreements. See the NOTICE file distributed with this work for additional information regarding
- * copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance with the License. You may obtain a
- * copy of the License at
- * 
- * http://www.apache.org/licenses/LICENSE-2.0
- * 
- * Unless required by applicable law or agreed to in writing, software distributed under the License
- * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
- * or implied. See the License for the specific language governing permissions and limitations under
- * the License.
- */
-
-package org.apache.fluo.cluster.runner.it;
-
-import java.util.Collections;
-import java.util.List;
-
-import org.apache.fluo.api.config.ObserverConfiguration;
-import org.apache.fluo.api.data.Bytes;
-import org.apache.fluo.api.data.Column;
-import org.apache.fluo.api.types.StringEncoder;
-import org.apache.fluo.api.types.TypeLayer;
-import org.apache.fluo.api.types.TypedObserver;
-import org.apache.fluo.api.types.TypedTransaction;
-import org.apache.fluo.api.types.TypedTransactionBase;
-import org.apache.fluo.cluster.runner.MiniAppRunner;
-import org.apache.fluo.core.impl.Environment;
-import org.apache.fluo.integration.ITBaseMini;
-import org.junit.Assert;
-import org.junit.Test;
-
-public class MiniAppRunnerIT extends ITBaseMini {
-
-  private static TypeLayer tl = new TypeLayer(new StringEncoder());
-
-  public static class TestObserver extends TypedObserver {
-
-    @Override
-    public ObservedColumn getObservedColumn() {
-      return new ObservedColumn(new Column("ingest", "num"), NotificationType.STRONG);
-    }
-
-    @Override
-    public void process(TypedTransactionBase tx, Bytes row, Column col) {
-      try {
-        Thread.sleep(1000);
-      } catch (InterruptedException e) {
-        throw new IllegalStateException(e);
-      }
-      Column countColumn = new Column(Bytes.of("count"), col.getQualifier());
-      int ingest = tx.get().row(row).col(col).toInteger(0);
-      int count = tx.get().row(row).col(countColumn).toInteger(0);
-      tx.mutate().row(row).col(countColumn).set(count + ingest);
-      tx.mutate().row(row).col(col).delete();
-    }
-  }
-
-  @Override
-  protected List<ObserverConfiguration> getObservers() {
-    return Collections.singletonList(new ObserverConfiguration(TestObserver.class.getName()));
-  }
-
-  @Test
-  public void testSimple() {
-    try (TypedTransaction tx1 = tl.wrap(client.newTransaction())) {
-      tx1.mutate().row("row1").fam("ingest").qual("num").set(1);
-      tx1.mutate().row("row2").fam("ingest").qual("num").set(1);
-      tx1.mutate().row("row3").fam("ingest").qual("num").set(1);
-      tx1.commit();
-    }
-
-    Environment env = new Environment(config);
-    MiniAppRunner runner = new MiniAppRunner();
-
-    Assert.assertEquals(3, runner.countNotifications(env));
-
-    runner.waitUntilFinished(config);
-
-    Assert.assertEquals(0, runner.countNotifications(env));
-    Assert.assertEquals(3, runner.scan(config, new String[] {}));
-    Assert.assertEquals(1, runner.scan(config, new String[] {"-r", "row1"}));
-    Assert.assertEquals(1, runner.scan(config, new String[] {"-r", "row2"}));
-    Assert.assertEquals(3, runner.scan(config, new String[] {"-c", "count"}));
-    Assert.assertEquals(0, runner.scan(config, new String[] {"-c", "ingest"}));
-  }
-}

http://git-wip-us.apache.org/repos/asf/incubator-fluo/blob/6fd86298/modules/distribution/pom.xml
----------------------------------------------------------------------
diff --git a/modules/distribution/pom.xml b/modules/distribution/pom.xml
index 2a1ed43..bc2aacf 100644
--- a/modules/distribution/pom.xml
+++ b/modules/distribution/pom.xml
@@ -15,21 +15,17 @@
 -->
 <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
   <modelVersion>4.0.0</modelVersion>
-
   <parent>
     <groupId>org.apache.fluo</groupId>
     <artifactId>fluo-parent</artifactId>
     <version>1.0.0-beta-3-SNAPSHOT</version>
     <relativePath>../../pom.xml</relativePath>
   </parent>
-
   <artifactId>fluo-distribution</artifactId>
   <packaging>pom</packaging>
-
   <name>Fluo Distribution</name>
   <description>This module produces a tarball distribution of Fluo.  It contains all of the default configuration
     and scripts required for the distribution.</description>
-
   <!-- NOTE: These dependencies are bundled in this assembly -->
   <dependencies>
     <dependency>
@@ -65,7 +61,6 @@
       <artifactId>slf4j-log4j12</artifactId>
     </dependency>
   </dependencies>
-
   <build>
     <plugins>
       <plugin>
@@ -86,9 +81,17 @@
           </execution>
         </executions>
       </plugin>
+      <plugin>
+        <groupId>org.apache.rat</groupId>
+        <artifactId>apache-rat-plugin</artifactId>
+        <configuration>
+          <excludes>
+            <exclude>src/main/deps/**</exclude>
+          </excludes>
+        </configuration>
+      </plugin>
     </plugins>
   </build>
-
   <profiles>
     <profile>
       <id>full</id>

http://git-wip-us.apache.org/repos/asf/incubator-fluo/blob/6fd86298/modules/distribution/src/main/assembly/bin.xml
----------------------------------------------------------------------
diff --git a/modules/distribution/src/main/assembly/bin.xml b/modules/distribution/src/main/assembly/bin.xml
index 522c465..da9d0e0 100644
--- a/modules/distribution/src/main/assembly/bin.xml
+++ b/modules/distribution/src/main/assembly/bin.xml
@@ -27,22 +27,6 @@
       <scope>runtime</scope>
       <outputFileNameMapping>${artifact.artifactId}${dashClassifier?}-${artifact.version}.${artifact.extension}</outputFileNameMapping>
       <useTransitiveDependencies>true</useTransitiveDependencies>
-      <excludes>
-        <exclude>*:slf4j-log4j12</exclude>
-        <exclude>*:log4j</exclude>
-        <exclude>*:logback-classic</exclude>
-        <exclude>*:logback-core</exclude>
-        <exclude>*:log4j-over-slf4j</exclude>
-      </excludes>
-    </dependencySet>
-    <dependencySet>
-      <outputDirectory>lib/fluo-client</outputDirectory>
-      <directoryMode>0755</directoryMode>
-      <fileMode>0644</fileMode>
-      <useProjectArtifact>true</useProjectArtifact>
-      <scope>runtime</scope>
-      <outputFileNameMapping>${artifact.artifactId}${dashClassifier?}-${artifact.version}.${artifact.extension}</outputFileNameMapping>
-      <useTransitiveDependencies>true</useTransitiveDependencies>
       <useTransitiveFiltering>true</useTransitiveFiltering>
       <includes>
         <include>org.apache.fluo:fluo-api</include>
@@ -50,59 +34,32 @@
         <include>org.apache.fluo:fluo-core</include>
         <include>org.apache.fluo:fluo-mini</include>
         <include>org.apache.fluo:fluo-mapreduce</include>
+        <include>org.apache.fluo:fluo-cluster</include>
         <include>org.slf4j:slf4j-api</include>
-        <!--fluo-core has direct dependencies on following, but for some reason maven does not include -->
         <include>org.apache.curator:curator-framework</include>
         <include>org.apache.curator:curator-recipes</include>
       </includes>
       <excludes>
         <exclude>org.apache.accumulo:*</exclude>
+        <exclude>org.apache.thrift:*</exclude>
         <exclude>org.apache.hadoop:*</exclude>
         <exclude>org.apache.zookeeper:zookeeper</exclude>
+        <exclude>*:slf4j-log4j12</exclude>
+        <exclude>*:log4j</exclude>
+        <exclude>*:logback-classic</exclude>
+        <exclude>*:logback-core</exclude>
+        <exclude>*:log4j-over-slf4j</exclude>
       </excludes>
     </dependencySet>
     <dependencySet>
-      <outputDirectory>lib/accumulo</outputDirectory>
-      <directoryMode>0755</directoryMode>
-      <fileMode>0644</fileMode>
-      <useProjectArtifact>false</useProjectArtifact>
-      <scope>runtime</scope>
-      <outputFileNameMapping>${artifact.artifactId}${dashClassifier?}-${artifact.version}.${artifact.extension}</outputFileNameMapping>
-      <useTransitiveDependencies>true</useTransitiveDependencies>
-      <useTransitiveFiltering>true</useTransitiveFiltering>
-      <includes>
-        <include>org.apache.accumulo:accumulo-core</include>
-        <include>org.apache.accumulo:accumulo-minicluster</include>
-      </includes>
-      <excludes>
-        <exclude>org.apache.hadoop:hadoop-minicluster</exclude>
-      </excludes>
-    </dependencySet>
-    <dependencySet>
-      <outputDirectory>lib/hadoop-client</outputDirectory>
-      <directoryMode>0755</directoryMode>
-      <fileMode>0644</fileMode>
-      <useProjectArtifact>false</useProjectArtifact>
-      <scope>runtime</scope>
-      <outputFileNameMapping>${artifact.artifactId}${dashClassifier?}-${artifact.version}.${artifact.extension}</outputFileNameMapping>
-      <useTransitiveDependencies>true</useTransitiveDependencies>
-      <useTransitiveFiltering>true</useTransitiveFiltering>
-      <includes>
-        <include>org.apache.hadoop:hadoop-client</include>
-      </includes>
-    </dependencySet>
-    <dependencySet>
-      <outputDirectory>lib/zookeeper</outputDirectory>
-      <directoryMode>0755</directoryMode>
-      <fileMode>0644</fileMode>
-      <useProjectArtifact>false</useProjectArtifact>
-      <scope>runtime</scope>
-      <outputFileNameMapping>${artifact.artifactId}${dashClassifier?}-${artifact.version}.${artifact.extension}</outputFileNameMapping>
-      <useTransitiveDependencies>true</useTransitiveDependencies>
-      <useTransitiveFiltering>true</useTransitiveFiltering>
-      <includes>
-        <include>org.apache.zookeeper:zookeeper</include>
-      </includes>
+        <!-- this is in separate section because it was transitively filtered -->
+        <outputDirectory>lib</outputDirectory>
+        <directoryMode>0755</directoryMode>
+        <fileMode>0644</fileMode>
+        <!-- was having a problem with version 2.2.2 from Accumulo-->
+        <includes>
+          <include>com.google.code.gson:gson</include>
+        </includes>
     </dependencySet>
     <dependencySet>
       <outputDirectory>lib/log4j</outputDirectory>
@@ -160,6 +117,13 @@
       </includes>
     </fileSet>
     <fileSet>
+      <directory>src/main/deps</directory>
+      <outputDirectory>lib/deps</outputDirectory>
+      <includes>
+        <include>*</include>
+      </includes>
+    </fileSet>
+    <fileSet>
       <directory>${project.basedir}/../../docs</directory>
       <outputDirectory>docs</outputDirectory>
       <includes>

http://git-wip-us.apache.org/repos/asf/incubator-fluo/blob/6fd86298/modules/distribution/src/main/config/fluo-env.sh
----------------------------------------------------------------------
diff --git a/modules/distribution/src/main/config/fluo-env.sh b/modules/distribution/src/main/config/fluo-env.sh
index 8594c78..b344775 100755
--- a/modules/distribution/src/main/config/fluo-env.sh
+++ b/modules/distribution/src/main/config/fluo-env.sh
@@ -14,4 +14,66 @@
 # Sets HADOOP_PREFIX if it is not already set.  Please modify the
 # export statement to use the correct directory.  Remove the test
 # statement to override any previously set environment.
+
+
 test -z "$HADOOP_PREFIX" && export HADOOP_PREFIX=/path/to/hadoop
+
+# The classpath for Fluo must be defined.  The Fluo tarball does not include
+# jars for Accumulo, Zookeeper, or Hadoop.  This example env file offers two
+# ways to setup the classpath with these jars.  Go to the end of the file for
+# more info.
+
+addToClasspath() 
+{
+  local dir=$1
+  local filterRegex=$2
+
+  if [ ! -d "$dir" ]; then
+    echo "ERROR $dir does not exist or not a directory"
+    exit 1
+  fi
+
+  for jar in $dir/*.jar; do
+    if ! [[ $jar =~ $filterRegex ]]; then
+       CLASSPATH="$CLASSPATH:$jar"
+    fi
+  done
+}
+
+
+# This function attemps to obtain Accumulo, Hadoop, and Zookeeper jars from the
+# location where those dependencies are installed on the system.
+setupClasspathFromSystem()
+{
+  test -z "$ACCUMULO_HOME" && ACCUMULO_HOME=/path/to/accumulo
+  test -z "$ZOOKEEPER_HOME" && ZOOKEEPER_HOME=/path/to/zookeeper
+
+  CLASSPATH="$FLUO_HOME/lib/*:$FLUO_HOME/lib/logback/*"
+
+  #any jars matching this pattern is excluded from classpath
+  EXCLUDE_RE="(.*log4j.*)|(.*asm.*)|(.*guava.*)|(.*gson.*)"
+
+  addToClasspath "$ACCUMULO_HOME/lib" $EXCLUDE_RE
+  addToClasspath "$ZOOKEEPER_HOME" $EXCLUDE_RE
+  addToClasspath "$ZOOKEEPER_HOME/lib" $EXCLUDE_RE
+  addToClasspath "$HADOOP_PREFIX/share/hadoop/common" $EXCLUDE_RE;
+  addToClasspath "$HADOOP_PREFIX/share/hadoop/common/lib" $EXCLUDE_RE;
+  addToClasspath "$HADOOP_PREFIX/share/hadoop/hdfs" $EXCLUDE_RE;
+  addToClasspath "$HADOOP_PREFIX/share/hadoop/hdfs/lib" $EXCLUDE_RE;
+  addToClasspath "$HADOOP_PREFIX/share/hadoop/yarn" $EXCLUDE_RE;
+  addToClasspath "$HADOOP_PREFIX/share/hadoop/yarn/lib" $EXCLUDE_RE;
+}
+
+
+# This function obtains Accumulo, Hadoop, and Zookeeper jars from
+# $FLUO_HOME/lib/deps/. Before using this function, go to that directoy and
+# follow the readme to download the dependencies.
+setupClasspathFromLib(){
+  CLASSPATH="$FLUO_HOME/lib/*:$FLUO_HOME/lib/logback/*:$FLUO_HOME/lib/deps/*"
+}
+
+# Call one of the following functions to setup the classpath or write your own
+# bash code to setup the classpath for Fluo.
+
+setupClasspathFromSystem
+#setupClasspathFromLib

http://git-wip-us.apache.org/repos/asf/incubator-fluo/blob/6fd86298/modules/distribution/src/main/deps/README.md
----------------------------------------------------------------------
diff --git a/modules/distribution/src/main/deps/README.md b/modules/distribution/src/main/deps/README.md
new file mode 100644
index 0000000..0457fe8
--- /dev/null
+++ b/modules/distribution/src/main/deps/README.md
@@ -0,0 +1,14 @@
+Fluo Dependencies
+-----------------
+
+This directory provides a simple maven pom that can be used to fetch Accumulo,
+Hadoop, and Zookeeper depedencies.   You should fetch the versions of these
+dependencies that are installed on your system.
+
+    cd $FLUO_HOME/lib/deps/
+    #edit pom to reflect versions of Software on your system
+    vim pom.xml
+    rm *.jar;mvn package clean
+
+After doing this edit `$FLUO_HOME/conf/fluo-env.sh` to add
+`$FLUO_HOME/lib/deps/*` to the classpath.

http://git-wip-us.apache.org/repos/asf/incubator-fluo/blob/6fd86298/modules/distribution/src/main/deps/pom.xml
----------------------------------------------------------------------
diff --git a/modules/distribution/src/main/deps/pom.xml b/modules/distribution/src/main/deps/pom.xml
new file mode 100644
index 0000000..68c2576
--- /dev/null
+++ b/modules/distribution/src/main/deps/pom.xml
@@ -0,0 +1,64 @@
+<project>
+  
+  <modelVersion>4.0.0</modelVersion>
+  <packaging>jar</packaging>
+  <groupId>a</groupId>
+  <artifactId>a</artifactId>
+  <version>a</version>
+
+  <properties>
+    <accumulo.version>1.7.1</accumulo.version>
+    <hadoop.version>2.6.3</hadoop.version>
+    <zookeeper.version>3.4.8</zookeeper.version>
+  </properties>
+
+  <dependencies>
+    <dependency>
+      <groupId>org.apache.accumulo</groupId>
+      <artifactId>accumulo-core</artifactId>
+      <version>${accumulo.version}</version>
+    </dependency>
+    <dependency>
+      <groupId>org.apache.hadoop</groupId>
+      <artifactId>hadoop-client</artifactId>
+      <version>${hadoop.version}</version>
+    </dependency>
+    <dependency>
+      <groupId>org.apache.hadoop</groupId>
+      <artifactId>hadoop-yarn-client</artifactId>
+      <version>${hadoop.version}</version>
+    </dependency>
+    <dependency>
+      <groupId>org.apache.zookeeper</groupId>
+      <artifactId>zookeeper</artifactId>
+      <version>${zookeeper.version}</version>
+    </dependency>
+  </dependencies>
+
+  <build>
+    <plugins>
+      <plugin>
+        <groupId>org.apache.maven.plugins</groupId>
+        <artifactId>maven-dependency-plugin</artifactId>
+        <version>2.10</version>
+        <executions>
+          <execution>
+            <id>copy-dependencies</id>
+            <phase>package</phase>
+            <goals>
+              <goal>copy-dependencies</goal>
+            </goals>
+            <configuration>
+              <outputDirectory>${project.build.directory}/..</outputDirectory>
+              <overWriteReleases>false</overWriteReleases>
+              <overWriteSnapshots>false</overWriteSnapshots>
+              <overWriteIfNewer>true</overWriteIfNewer>
+              <excludeArtifactIds>guava,gson,asm,log4j,slf4j-api,slf4j-log4j12</excludeArtifactIds>
+              <includeScope>runtime</includeScope>
+            </configuration>
+          </execution>
+        </executions>
+      </plugin>
+    </plugins>
+  </build>
+</project>

http://git-wip-us.apache.org/repos/asf/incubator-fluo/blob/6fd86298/modules/distribution/src/main/scripts/fluo
----------------------------------------------------------------------
diff --git a/modules/distribution/src/main/scripts/fluo b/modules/distribution/src/main/scripts/fluo
index ae24c64..b9cd984 100755
--- a/modules/distribution/src/main/scripts/fluo
+++ b/modules/distribution/src/main/scripts/fluo
@@ -26,6 +26,8 @@ script=$( basename "$SOURCE" )
 
 . "$bin"/impl/config.sh
 
+export CLASSPATH
+
 # stop if any command fails
 set -e
 
@@ -57,7 +59,7 @@ function print_usage {
   echo "  status <app>  Prints status of Fluo application"
   echo "  info <app>    Prints information about containers of Fluo application"
   echo "  scan <app>    Prints snapshot of data in Fluo application"
-  echo "  classpath     Prints the classpath needed for fluo clients.  Run with -h to learn more."
+  echo "  classpath     Prints the classpath setup in fluo-env.sh"
   echo "  wait <app>    Waits until all notifications are processed"
   echo "  exec <app> <class>  {<argument>}";
   echo " "
@@ -76,7 +78,7 @@ function validate_app {
 
 function validate_app_full {
   validate_app
-  java -cp "$FLUO_LIB_DIR/*:$FLUO_LIB_DIR/logback/*" org.apache.fluo.cluster.util.ValidateAppName $APP
+  java org.apache.fluo.cluster.util.ValidateAppName $APP
   if [[ ! -d $APP_DIR || ! -d $APP_CONF_DIR || ! -d $APP_LIB_DIR  ]]; then
     echo "ERROR - The Fluo '$APP' application needs to be configured in apps/ with a conf/ and lib/ directory.  Use 'fluo new $APP' to create this configuration"
     exit 1
@@ -97,7 +99,7 @@ function check_hadoop {
 case "$1" in
 new)
   validate_app
-  java -cp "$FLUO_LIB_DIR/*:$FLUO_LIB_DIR/logback/*" org.apache.fluo.cluster.util.ValidateAppName $APP
+  java org.apache.fluo.cluster.util.ValidateAppName $APP
   if [ -d $APP_DIR ]; then
     echo "The Fluo '$APP' application already has a directory in apps/"
     exit 1
@@ -120,27 +122,30 @@ init)
     echo "Copying `ls $FLUO_HOME/lib/fluo-accumulo-*.jar` to HDFS"
     $HADOOP_PREFIX/bin/hdfs dfs -copyFromLocal -f $FLUO_HOME/lib/fluo-accumulo-*.jar /fluo/lib/
   fi
-  java -cp "$FLUO_LIB_DIR/*:$FLUO_LIB_DIR/logback/*:$APP_LIB_DIR/*" org.apache.fluo.cluster.command.FluoCommand $FLUO_HOME $HADOOP_PREFIX "$@"
+  export CLASSPATH="$APP_LIB_DIR/*:$CLASSPATH"
+  java org.apache.fluo.cluster.command.FluoCommand $FLUO_HOME $HADOOP_PREFIX "$@"
   ;;
 start)
   validate_app_full
   check_hadoop
-  java -cp "$FLUO_LIB_DIR/*:$FLUO_LIB_DIR/logback/*:$APP_LIB_DIR/*" org.apache.fluo.cluster.command.FluoCommand $FLUO_HOME $HADOOP_PREFIX "$@"
+  export CLASSPATH="$APP_LIB_DIR/*:$CLASSPATH"
+  java org.apache.fluo.cluster.command.FluoCommand $FLUO_HOME $HADOOP_PREFIX "$@"
   ;;
 stop|kill|status|info|scan|wait)
   validate_app
   check_hadoop
-  java -cp "$FLUO_LIB_DIR/*:$FLUO_LIB_DIR/logback/*" org.apache.fluo.cluster.command.FluoCommand $FLUO_HOME $HADOOP_PREFIX "$@"
+  java org.apache.fluo.cluster.command.FluoCommand $FLUO_HOME $HADOOP_PREFIX "$@"
   ;;
 list)
   check_hadoop
-  java -cp "$FLUO_LIB_DIR/*:$FLUO_LIB_DIR/logback/*" org.apache.fluo.cluster.command.FluoCommand $FLUO_HOME $HADOOP_PREFIX $1 app ${*:2}
+  java org.apache.fluo.cluster.command.FluoCommand $FLUO_HOME $HADOOP_PREFIX $1 app ${*:2}
   ;;
 classpath)
-  java -cp "$FLUO_LIB_DIR/*:$FLUO_LIB_DIR/logback/*" org.apache.fluo.cluster.command.FluoCommand $FLUO_HOME none $1 app ${*:2}
+  echo "$CLASSPATH"
   ;;
 exec)
-  java -cp "$FLUO_LIB_DIR/*:$FLUO_LIB_DIR/logback/*:$APP_LIB_DIR/*" org.apache.fluo.cluster.command.FluoCommand $FLUO_HOME $HADOOP_PREFIX "$@"
+  export CLASSPATH="$APP_LIB_DIR/*:$CLASSPATH"
+  java org.apache.fluo.cluster.command.FluoCommand $FLUO_HOME $HADOOP_PREFIX "$@"
   ;;
 *)
   print_usage

http://git-wip-us.apache.org/repos/asf/incubator-fluo/blob/6fd86298/modules/distribution/src/main/scripts/impl/config.sh
----------------------------------------------------------------------
diff --git a/modules/distribution/src/main/scripts/impl/config.sh b/modules/distribution/src/main/scripts/impl/config.sh
index b386a77..7abbbe5 100755
--- a/modules/distribution/src/main/scripts/impl/config.sh
+++ b/modules/distribution/src/main/scripts/impl/config.sh
@@ -25,10 +25,6 @@ bin="$( cd -P "$( dirname "$impl" )" && pwd )"
 script=$( basename "$SOURCE" )
 # Stop: Resolve Script Directory
 
-if [ -f "$bin"/../conf/fluo-env.sh ]; then
-    . "$bin"/../conf/fluo-env.sh
-fi
-
 # Determine FLUO_HOME - Use env variable set by user.  If none set, calculate using bin dir
 FLUO_HOME="${FLUO_HOME:-$( cd -P ${bin}/.. && pwd )}"
 export FLUO_HOME
@@ -56,6 +52,10 @@ then
   exit 1
 fi
 
+if [ -f "$bin"/../conf/fluo-env.sh ]; then
+    . "$bin"/../conf/fluo-env.sh
+fi
+
 if [[ "$OSTYPE" == "darwin"* ]]; then
   export MD5=md5
   export SED="sed -i .bak"

http://git-wip-us.apache.org/repos/asf/incubator-fluo/blob/6fd86298/modules/distribution/src/main/scripts/local-fluo
----------------------------------------------------------------------
diff --git a/modules/distribution/src/main/scripts/local-fluo b/modules/distribution/src/main/scripts/local-fluo
index 748620d..16ce77e 100755
--- a/modules/distribution/src/main/scripts/local-fluo
+++ b/modules/distribution/src/main/scripts/local-fluo
@@ -26,6 +26,8 @@ script=$( basename "$SOURCE" )
 
 . "$bin"/impl/config.sh
 
+export CLASSPATH
+
 # stop if any command fails
 set -e
 
@@ -54,7 +56,7 @@ function validate_app_full {
   if [[ $APP = *"-h"* ]]; then
     print_usage
   fi
-  java -cp "$FLUO_LIB_DIR/*:$FLUO_LIB_DIR/logback/*" org.apache.fluo.cluster.util.ValidateAppName $APP
+  java org.apache.fluo.cluster.util.ValidateAppName $APP
 }
 
 function check_dirs {
@@ -78,14 +80,13 @@ function kill_if_running {
 
 LOGHOST=$(hostname)
 LOCAL_OPTS="$APP_CONF_DIR/fluo.properties"
-LOCAL_LIB="$FLUO_LIB_DIR/*:$FLUO_LIB_DIR/logback/*"
 
 case "$1" in
 start-oracle)
   validate_app_full
   check_dirs
   SERVICE="oracle"
-  java -Dfluo.app=$APP -cp "$LOCAL_LIB" org.apache.fluo.core.oracle.FluoOracleImpl $LOCAL_OPTS >${APP_LOG_DIR}/${SERVICE}_${LOGHOST}.out 2>${APP_LOG_DIR}/${SERVICE}_${LOGHOST}.err &
+  java -Dfluo.app=$APP org.apache.fluo.core.oracle.FluoOracleImpl $LOCAL_OPTS >${APP_LOG_DIR}/${SERVICE}_${LOGHOST}.out 2>${APP_LOG_DIR}/${SERVICE}_${LOGHOST}.err &
   echo "Started oracle for Fluo '$APP' application.  View its logs at $APP_LOG_DIR"
   ;;
 kill-oracle)
@@ -95,7 +96,8 @@ start-worker)
   validate_app_full
   check_dirs
   SERVICE="worker"
-  java -Dfluo.app=$APP -cp "$LOCAL_LIB:$APP_LIB_DIR/*" org.apache.fluo.core.worker.FluoWorkerImpl $LOCAL_OPTS >${APP_LOG_DIR}/${SERVICE}_${LOGHOST}.out 2>${APP_LOG_DIR}/${SERVICE}_${LOGHOST}.err &
+  export CLASSPATH="$APP_LIB_DIR/*:$CLASSPATH"
+  java -Dfluo.app=$APP org.apache.fluo.core.worker.FluoWorkerImpl $LOCAL_OPTS >${APP_LOG_DIR}/${SERVICE}_${LOGHOST}.out 2>${APP_LOG_DIR}/${SERVICE}_${LOGHOST}.err &
   echo "Started worker for Fluo '$APP' application.  View its logs at $APP_LOG_DIR"
   ;;
 kill-worker)

http://git-wip-us.apache.org/repos/asf/incubator-fluo/blob/6fd86298/modules/distribution/src/main/scripts/mini-fluo
----------------------------------------------------------------------
diff --git a/modules/distribution/src/main/scripts/mini-fluo b/modules/distribution/src/main/scripts/mini-fluo
deleted file mode 100755
index 074b872..0000000
--- a/modules/distribution/src/main/scripts/mini-fluo
+++ /dev/null
@@ -1,160 +0,0 @@
-#! /usr/bin/env bash
-
-# Licensed to the Apache Software Foundation (ASF) under one or more contributor license
-# agreements.  See the NOTICE file distributed with this work for additional information regarding
-# copyright ownership.  The ASF licenses this file to you under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with the License.  You may
-# obtain a copy of the License at
-#
-#    http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software distributed under the License
-# is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
-# or implied. See the License for the specific language governing permissions and limitations under
-# the License.
-
-# Start: Resolve Script Directory
-SOURCE="${BASH_SOURCE[0]}"
-while [ -h "$SOURCE" ]; do # resolve $SOURCE until the file is no longer a symlink
-   bin="$( cd -P "$( dirname "$SOURCE" )" && pwd )"
-   SOURCE="$(readlink "$SOURCE")"
-   [[ $SOURCE != /* ]] && SOURCE="$bin/$SOURCE" # if $SOURCE was a relative symlink, we need to resolve it relative to the path where the symlink file was located
-done
-bin="$( cd -P "$( dirname "$SOURCE" )" && pwd )"
-script=$( basename "$SOURCE" )
-# Stop: Resolve Script Directory
-
-. "$bin"/impl/config.sh
-
-# stop if any command fails
-set -e
-
-APP=$2
-APP_DIR=$FLUO_HOME/apps/$APP
-APP_LIB_DIR=$APP_DIR/lib
-APP_CONF_DIR=$APP_DIR/conf
-APP_LOG_DIR=$APP_DIR/logs
-APP_MINI_DIR=$APP_DIR/mini
-APP_CLIENT_PROPS=$APP_MINI_DIR/client.properties
-
-function print_usage {
-  echo -e "Usage: mini-fluo <command> (<argument> ..)\n"
-  echo -e "Possible commands:\n"
-  echo "  new <app>         Creates configuration for new MiniFluo application in apps/"
-  echo "  start <app>       Starts application in MiniFluo on local machine"
-  echo "  kill <app>        Kills application in MiniFluo on local machine"
-  echo "  scan <app>        Prints snapshot of data in MiniFluo application"
-  echo "  classpath         Prints the classpath needed for fluo clients.  Run with -h to learn more."
-  echo "  wait <app>        Waits until all notifications are processed"
-  echo "  exec <app> <class>  {<argument>}";
-  echo " "
-  exit 1
-}
-
-function validate_app {
-  if [ -z "$APP" ]; then
-    echo -e "The application name (set by <app>) cannot be an empty string!\n"
-    print_usage
-  fi
-  if [[ $APP = *"-h"* ]]; then
-    print_usage
-  fi
-}
-
-function validate_app_full {  
-  validate_app
-  java -cp "$FLUO_LIB_DIR/*:$FLUO_LIB_DIR/logback/*" org.apache.fluo.cluster.util.ValidateAppName $APP
-}
-
-function check_dirs {
-  if [[ ! -d $APP_DIR || ! -d $APP_CONF_DIR || ! -d $APP_LIB_DIR  ]]; then
-    echo "ERROR - The Fluo '$APP' application needs to be configured in apps/ with a conf/ and lib/ directory.  Use 'mini-fluo new $APP' to create this configuration"
-    exit 1
-  fi
-  mkdir -p $APP_LOG_DIR 2>/dev/null
-}
-
-function copy_config {
-  if [ -f $FLUO_CONF_DIR/$1 ]; then
-    cp $FLUO_CONF_DIR/$1 $APP_CONF_DIR/
-  elif [ -f $FLUO_CONF_DIR/examples/$1 ]; then
-    cp FLUO_CONF_DIR/examples/$1 $APP_CONF_DIR
-  else
-    echo "Config file $1 not found in $FLUO_CONF_DIR or $FLUO_CONF_DIR/examples"
-    exit 1
-  fi
-}
-
-function kill_if_running {
-  PID=`jps -m | grep $1 | grep $APP | cut -f 1 -d ' '`
-  if [ -z "$PID" ]; then
-    echo "ERROR - A $1 process for the Fluo application '$APP' is not running"
-    exit 1
-  else
-    kill $PID
-  fi
-}
-
-FLUO_LOG_HOST=$(hostname)
-FLUO_APP="mini"
-MINI_OPTS="$APP_CONF_DIR/fluo.properties"
-MINI_OUT=${APP_LOG_DIR}/${FLUO_APP}_${FLUO_LOG_HOST}.out
-MINI_ERR=${APP_LOG_DIR}/${FLUO_APP}_${FLUO_LOG_HOST}.err
-MINI_LIB="$APP_CONF_DIR/*:$FLUO_LIB_DIR/*:$FLUO_LIB_DIR/log4j/*"
-
-START_OPTS="-Dlog4j.configuration=file:///$APP_CONF_DIR/log4j.xml -Dorg.apache.fluo.log.app=$FLUO_APP -Dorg.apache.fluo.log.host=$FLUO_LOG_HOST -Dorg.apache.fluo.log.dir=$APP_LOG_DIR"
-
-case "$1" in
-new)
-  if [ -d $APP_DIR ]; then
-    echo "The Fluo '$APP' application already has a directory in apps/"
-    exit 1
-  fi
-  validate_app_full
-  mkdir -p $APP_DIR
-  mkdir -p $APP_CONF_DIR
-  mkdir -p $APP_LIB_DIR
-  copy_config fluo.properties
-  $SED "s/org.apache.fluo.client.application.name=/org.apache.fluo.client.application.name=$APP/g" $APP_CONF_DIR/fluo.properties
-  $SED "s/org.apache.fluo.client.accumulo/#org.apache.fluo.client.accumulo/g" $APP_CONF_DIR/fluo.properties
-  $SED "s/org.apache.fluo.admin.accumulo.classpath/#org.apache.fluo.admin.accumulo.classpath/g" $APP_CONF_DIR/fluo.properties
-  $SED "s/#org.apache.fluo.mini/org.apache.fluo.mini/g" $APP_CONF_DIR/fluo.properties
-  copy_config log4j.xml
-  ;;
-start)
-  validate_app
-  check_dirs
-  java $START_OPTS -cp "$MINI_LIB:$FLUO_LIB_DIR/logback/*:$APP_LIB_DIR/*" org.apache.fluo.mini.MiniFluoImpl $MINI_OPTS >$MINI_OUT 2>$MINI_ERR &
-  echo "Started '$APP' application in MiniFluo.  View logs at $APP_LOG_DIR"
-  echo "MiniFluo will create properties file for Fluo clients at $APP_CLIENT_PROPS"
-  echo -n "Waiting for MiniFluo to create client.properties..."
-  while [ ! -f $APP_CLIENT_PROPS ]
-  do
-    echo -n "."
-    sleep 1
-  done
-  echo "."
-  echo "Client properties were created.  MiniFluo has started successfully!"
-  ;;
-kill)
-  validate_app
-  check_dirs
-  kill_if_running MiniFluoMain
-  java -cp "$FLUO_LIB_DIR/*:$FLUO_LIB_DIR/logback/*" org.apache.fluo.cluster.command.MiniFluoCommand $FLUO_HOME cleanup $APP
-  echo "Killed '$APP' application running in MiniFluo"
-  ;;
-scan|wait)
-  validate_app
-  check_dirs
-  java -cp "$FLUO_LIB_DIR/*:$FLUO_LIB_DIR/logback/*" org.apache.fluo.cluster.command.MiniFluoCommand $FLUO_HOME "$@"
-  ;;
-classpath)
-  java -cp "$FLUO_LIB_DIR/*:$FLUO_LIB_DIR/logback/*:$APP_LIB_DIR/*" org.apache.fluo.cluster.command.MiniFluoCommand $FLUO_HOME $1 app ${*:2}
-  ;;
-exec)
-  java -cp "$FLUO_LIB_DIR/*:$FLUO_LIB_DIR/logback/*:$APP_LIB_DIR/*" org.apache.fluo.cluster.command.MiniFluoCommand $FLUO_HOME "$@"
-  ;;
- 
-*)
-  print_usage
-esac


Mime
View raw message