fluo-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From mwa...@apache.org
Subject [1/2] incubator-fluo git commit: Avoid guava beta dependency and updated docs
Date Mon, 25 Jul 2016 14:18:47 GMT
Repository: incubator-fluo
Updated Branches:
  refs/heads/master 8316f94f5 -> 6c988a72e


Avoid guava beta dependency and updated docs

* Update oracle and worker to avoid using Guava abstract service which is beta
* Update documentation to recommend certain versions of Hadoop, Accumulo, etc
* Renamed prod-fluo-setup.md to install.md


Project: http://git-wip-us.apache.org/repos/asf/incubator-fluo/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-fluo/commit/6aa0d438
Tree: http://git-wip-us.apache.org/repos/asf/incubator-fluo/tree/6aa0d438
Diff: http://git-wip-us.apache.org/repos/asf/incubator-fluo/diff/6aa0d438

Branch: refs/heads/master
Commit: 6aa0d4389ef627ae1b8ef84d98a703f25fe4b27e
Parents: 8316f94
Author: Mike Walch <mwalch@gmail.com>
Authored: Fri Jul 22 13:05:48 2016 -0400
Committer: Mike Walch <mwalch@gmail.com>
Committed: Mon Jul 25 09:53:31 2016 -0400

----------------------------------------------------------------------
 README.md                                       |   8 +-
 docs/install.md                                 | 227 +++++++++++++++++++
 docs/prod-fluo-setup.md                         | 223 ------------------
 .../apache/fluo/core/oracle/FluoOracleImpl.java |  67 ++----
 .../apache/fluo/core/worker/FluoWorkerImpl.java |  57 ++---
 modules/distribution/src/main/lib/ahz/pom.xml   |   4 +-
 modules/distribution/src/main/lib/fetch.sh      |   2 +-
 pom.xml                                         |   1 +
 8 files changed, 274 insertions(+), 315 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/incubator-fluo/blob/6aa0d438/README.md
----------------------------------------------------------------------
diff --git a/README.md b/README.md
index 5488013..e434a9b 100644
--- a/README.md
+++ b/README.md
@@ -14,9 +14,9 @@ news and general information.
 There are several ways to run Fluo (listed in order of increasing difficulty):
 
 * [quickstart] - Starts a MiniFluo instance that is configured to run a word count application
-* [fluo-dev] - Command-line tool for running Fluo and its dependencies on a single machine
-* [Zetten] - Command-line tool that launches an AWS cluster and sets up Fluo/Accumulo on
it
-* [Production] - Sets up Fluo on a cluster where Accumulo, Hadoop & Zookeeper are running
+* [fluo-dev] - Automated tool that sets up Fluo and its dependencies on a single machine
+* [Zetten] - Automated tool that launches an AWS cluster and sets up Fluo/Accumulo on it
+* [Install instructions][install] - Manually set up Fluo on a cluster where Accumulo, Hadoop
& Zookeeper are running
 
 Except for [quickstart], all above will set up a Fluo application that will be idle unless
you
 create client & observer code for your application. You can either [create your own
@@ -50,7 +50,7 @@ Below are helpful resources for Fluo application developers:
 [phrasecount]: https://github.com/fluo-io/phrasecount
 [fluo-stress]: https://github.com/fluo-io/fluo-stress
 [webindex]: https://github.com/fluo-io/webindex
-[Production]: docs/prod-fluo-setup.md
+[install]: docs/install.md
 [apps]: docs/applications.md
 [api]: https://fluo.apache.org/apidocs/
 [recipes]: https://github.com/apache/incubator-fluo-recipes

http://git-wip-us.apache.org/repos/asf/incubator-fluo/blob/6aa0d438/docs/install.md
----------------------------------------------------------------------
diff --git a/docs/install.md b/docs/install.md
new file mode 100644
index 0000000..9856ff9
--- /dev/null
+++ b/docs/install.md
@@ -0,0 +1,227 @@
+Fluo Install Instructions
+=========================
+
+Install instructions for running Fluo on machine or cluster where Accumulo, Hadoop,
+and Zookeeper are installed and running.  If you want to avoid setting up these
+dependencies, consider using [fluo-dev] or [Zetten].
+
+Requirements
+------------
+
+Before you install Fluo, the following software must be installed and running on
+your local machine or cluster:
+
+| Software    | Recommended Version | Minimum Version |
+|-------------|---------------------|-----------------|
+| [Accumulo]  | 1.7.2               | 1.6.1           |
+| [Hadoop]    | 2.7.2               | 2.6.0           |
+| [Zookeeper] | 3.4.8               |                 |
+| [Java]      | JDK 8               | JDK 8           |
+
+Obtain a distribution
+---------------------
+
+Before you can install Fluo, you will need to obtain a distribution tarball.  It is
+recommended that you download the [latest release][release].  You can also build
+a distribution from the master branch by following these steps which create a tarball
+in `modules/distribution/target`:
+
+    git clone https://github.com/apache/incubator-fluo.git
+    cd fluo/
+    mvn package
+
+Install Fluo
+------------
+
+After you obtain a Fluo distribution tarball, follow these steps to install Fluo.
+
+1. Choose a directory with plenty of space and untar the distribution:
+
+        tar -xvzf fluo-1.0.0-incubating-bin.tar.gz
+
+2. Copy the example configuration to the base of your configuration directory to create
+the default configuration for your Fluo install:
+
+        cp conf/examples/* conf/
+
+    The default configuration will be used as the base configuration for each new application.
+
+3. Modify [fluo.properties] for your environment.  However, you should not configure any

+application settings (like observers). 
+
+    NOTE - All properties that have a default are set with it.  Uncomment a property if you
want 
+to use a value different than the default.  Properties that are unset and uncommented must
be
+set by the user.
+
+4. Fluo needs to build its classpath using jars from the versions of Hadoop, Accumulo, and
+Zookeeper that you are using. Choose one of the two ways below to make these jars available
+to Fluo:
+
+    * Set `HADOOP_PREFIX`, `ACCUMULO_HOME`, and `ZOOKEEPER_HOME` in your environment or configure
+    these variables in [fluo-env.sh].  Fluo will look in these locations for jars.
+    * Run `./lib/fetch.sh ahz` to download Hadoop, Accumulo, and Zookeeper jars to `lib/ahz`
and
+    configure [fluo-env.sh] to look in this directory. By default, this command will download
the
+    default versions set in [lib/ahz/pom.xml]. If you are not using the default versions,
you can
+    override them:
+    
+            ./lib/fetch.sh ahz -Daccumulo.version=1.7.2 -Dhadoop.version=2.7.2 -Dzookeeper.version=3.4.8
+
+5. Fluo needs more dependencies than what is available from Hadoop, Accumulo, and Zookeeper.
+These extra dependencies need to be downloaded to `lib/` using the command below:
+
+        ./lib/fetch.sh extra
+
+You are now ready to use the Fluo command script.
+
+Fluo command script
+-------------------
+
+The Fluo command script is located at `bin/fluo` of your Fluo installation.  All Fluo
+commands are invoked by this script.
+
+Modify and add the following to your `~/.bashrc` if you want to be able to execute the
+fluo script from any directory:
+
+    export PATH=/path/to/fluo-1.0.0-incubating/bin:$PATH
+
+Source your `.bashrc` for the changes to take effect and test the script
+
+    source ~/.bashrc
+    fluo
+
+Running the script without any arguments prints a description of all commands.
+
+    ./bin/fluo
+
+Configure a Fluo application
+----------------------------
+
+You are now ready to configure a Fluo application.  Use the command below to create the
+configuration necessary for a new application.  Feel free to pick a different name (other
+than `myapp`) for your application:
+
+    fluo new myapp
+
+This command will create a directory for your application at `apps/myapp` of your Fluo
+install which will contain a `conf` and `lib`.
+
+The `apps/myapp/conf` directory contains a copy of the `fluo.properties` from your default
+configuration.  This should be configured for your application:
+
+    vim apps/myapp/fluo.properties
+
+When configuring the observer section in fluo.properties, you can configure your instance
+for the [phrasecount] application if you have not created your own application. See
+the [phrasecount] example for instructions. You can also choose not to configure any
+observers but your workers will be idle when started.
+
+The `apps/myapp/lib` directory should contain any observer jars for your application. If

+you configured [fluo.properties] for observers, copy any jars containing these
+observer classes to this directory.
+ 
+Initialize your application
+---------------------------
+
+After your application has been configured, use the command below to initialize it:
+
+    fluo init myapp
+
+This only needs to be called once and stores configuration in Zookeeper.
+
+Start your application
+----------------------
+
+A Fluo application consists of one oracle process and multiple worker processes.
+Before starting your application, you can configure the number of worker process
+in your [fluo.properties] file.
+
+When you are ready to start your Fluo application on your YARN cluster, run the
+command below:
+
+    fluo start myapp
+
+The start command above will work for a single-node or a large cluster.  By
+using YARN, you do not need to deploy the Fluo binaries to every node on your
+cluster or start processes on every node.
+
+You can use the following command to check the status of your instance:
+
+    fluo status myapp
+
+For more detailed information on the YARN containers running Fluo:
+
+    fluo info myapp
+
+You can also use `yarn application -list` to check the status of your Fluo instance
+in YARN.  Logs are viewable within YARN.
+
+When you have data in your fluo instance, you can view it using the command `fluo scan`.
+Pipe the output to `less` using the command `fluo scan | less` if you want to page 
+through the data.
+
+Stop your Fluo application
+--------------------------
+
+Use the following command to stop your Fluo application:
+
+    fluo stop myapp
+
+If stop fails, there is also a kill command.
+
+    fluo kill myapp
+
+Tuning Accumulo
+---------------
+
+Fluo will reread the same data frequently when it checks conditions on
+mutations.   When Fluo initializes a table it enables data caching to make
+this more efficient.  However you may need to increase the amount of memory
+available for caching in the tserver by increasing `tserver.cache.data.size`.
+Increasing this may require increasing the maximum tserver java heap size in
+`accumulo-env.sh`.
+
+Fluo will run many client threads, will want to ensure the tablet server
+has enough threads.  Should probably increase the
+`tserver.server.threads.minimum` Accumulo setting.
+
+Using at least Accumulo 1.6.1 is recommended because multiple performance bugs
+were fixed.
+
+Tuning YARN
+-----------
+
+When running Fluo oracles and workers in YARN, the number of instances, max memory, and number
+of cores for Fluo processes can be configured in [fluo.properties]. If YARN is killing processes
+consider increasing `twill.java.reserved.memory.mb` (which defaults to 200 and is set in
yarn-site.xml).
+The `twill.java.reserved.memory.mb` config determines the gap between the YARN memory limit
set in
+[fluo.properties] and the java -Xmx setting.  For example, if max memory is 1024 and twill
reserved
+memory is 200, the java -Xmx setting will be 1024-200 = 824 MB.
+
+Run locally without YARN
+------------------------
+
+If you do not have YARN set up, you can start the oracle and worker as a local 
+Fluo process using the following commands:
+
+    local-fluo start-oracle
+    local-fluo start-worker
+
+Use the following commands to stop a local Fluo process:
+
+    local-fluo stop-worker
+    local-fluo stop-oracle
+
+In a distributed environment, you will need to deploy and configure a Fluo 
+distribution on every node in your cluster.
+
+[fluo-dev]: https://github.com/fluo-io/fluo-dev
+[Zetten]: https://github.com/fluo-io/zetten
+[Accumulo]: https://accumulo.apache.org/
+[Hadoop]: http://hadoop.apache.org/
+[Zookeeper]: http://zookeeper.apache.org/
+[Java]: http://openjdk.java.net/
+[release]: https://fluo.apache.org/download/
+[phrasecount]: https://github.com/fluo-io/phrasecount
+[fluo.properties]: ../modules/distribution/src/main/config/fluo.properties
+[fluo-env.sh]: ../modules/distribution/src/main/config/fluo-env.sh
+[lib/ahz/pom.xml]: ../modules/distribution/src/main/lib/ahz/pom.xml

http://git-wip-us.apache.org/repos/asf/incubator-fluo/blob/6aa0d438/docs/prod-fluo-setup.md
----------------------------------------------------------------------
diff --git a/docs/prod-fluo-setup.md b/docs/prod-fluo-setup.md
deleted file mode 100644
index e21350b..0000000
--- a/docs/prod-fluo-setup.md
+++ /dev/null
@@ -1,223 +0,0 @@
-Production Setup
-================
-
-Below are instructions for running Fluo in a production environment where Accumulo,
-Hadoop & Zookeeper are installed and running.  If you want to avoid setting up
-these dependencies, consider using [fluo-dev]. 
-
-Requirements
-------------
-
-Before you install Fluo, you will need the following installed and running on
-your local machine or cluster:
-
-* [Accumulo][Accumulo] (version 1.6+)
-* [Hadoop][Hadoop] (version 2.6+)
-* [Zookeeper]
-* [Java][Java] (version 8+)
-
-Obtain a distribution
----------------------
-
-Before you can install Fluo, you will need to obtain a distribution tarball.  It is
-recommended that you download the [latest release][release].  You can also build
-a distribution from the master branch by following these steps which create a tarball
-in `modules/distribution/target`:
-
-    git clone https://github.com/apache/incubator-fluo.git
-    cd fluo/
-    mvn package
-
-Install Fluo
-------------
-
-After you obtain a Fluo distribution tarball, follow these steps to install Fluo.
-
-1. Choose a directory with plenty of space and untar the distribution:
-
-        tar -xvzf fluo-1.0.0-incubating-bin.tar.gz
-
-2. Copy the example configuration to the base of your configuration directory to create
-the default configuration for your Fluo install:
-
-        cp conf/examples/* conf/
-
-    The default configuration will be used as the base configuration for each new application.
-
-3. Modify [fluo.properties] for your environment.  However, you should not configure any

-application settings (like observers). 
-
-    NOTE - All properties that have a default are set with it.  Uncomment a property if you
want 
-to use a value different than the default.  Properties that are unset and uncommented must
be
-set by the user.
-
-4. Fluo needs build its classpath using jars from the version of Hadoop, Accumulo, and Zookeeper
-that you are using. Choose one of the two ways below to make these jars available to Fluo:
-
-    * Set `HADOOP_PREFIX`, `ACCUMULO_HOME`, and `ZOOKEEPER_HOME` in your environment or configure
-    these variables in [fluo-env.sh].  Fluo will look in these locations for jars.
-    * Run `./lib/fetch.sh ahz` to download Hadoop, Accumulo, and Zookeeper jars to `lib/ahz`
and
-    configure [fluo-env.sh] to look in this directory. By default, this command will download
the
-    default versions set in [lib/ahz/pom.xml]. If you are not using the default versions,
you can
-    override them:
-    
-            ./lib/fetch.sh ahz -Daccumulo.version=1.7.1 -Dhadoop.version=2.6.3 -Dzookeeper.version=3.4.8
-
-5. Fluo needs more dependencies than what is available from Hadoop, Accumulo, and Zookeeper.
-These extra dependencies need to be downloaded to `lib/` using the command below:
-
-        ./lib/fetch.sh extra
-
-You are now ready to use the Fluo command script.
-
-Fluo command script
--------------------
-
-The Fluo command script is located at `bin/fluo` of your Fluo installation.  All Fluo
-commands are invoked by this script.
-
-Modify and add the following to your `~/.bashrc` if you want to be able to execute the
-fluo script from any directory:
-
-    export PATH=/path/to/fluo-1.0.0-incubating/bin:$PATH
-
-Source your `.bashrc` for the changes to take effect and test the script
-
-    source ~/.bashrc
-    fluo
-
-Running the script without any arguments prints a description of all commands.
-
-    ./bin/fluo
-
-Configure a Fluo application
-----------------------------
-
-You are now ready to configure a Fluo application.  Use the command below to create the
-configuration necessary for a new application.  Feel free to pick a different name (other
-than `myapp`) for your application:
-
-    fluo new myapp
-
-This command will create a directory for your application at `apps/myapp` of your Fluo
-install which will contain a `conf` and `lib`.
-
-The `apps/myapp/conf` directory contains a copy of the `fluo.properties` from your default
-configuration.  This should be configured for your application:
-
-    vim apps/myapp/fluo.properties
-
-When configuring the observer section in fluo.properties, you can configure your instance
-for the [phrasecount] application if you have not created your own application. See
-the [phrasecount] example for instructions. You can also choose not to configure any
-observers but your workers will be idle when started.
-
-The `apps/myapp/lib` directory should contain any observer jars for your application. If

-you configured [fluo.properties] for observers, copy any jars containing these
-observer classes to this directory.
- 
-Initialize your application
----------------------------
-
-After your application has been configured, use the command below to initialize it:
-
-    fluo init myapp
-
-This only needs to be called once and stores configuration in Zookeeper.
-
-Start your application
-----------------------
-
-A Fluo application consists of one oracle process and multiple worker processes.
-Before starting your application, you can configure the number of worker process
-in your [fluo.properties] file.
-
-When you are ready to start your Fluo application on your YARN cluster, run the
-command below:
-
-    fluo start myapp
-
-The start command above will work for a single-node or a large cluster.  By
-using YARN, you do not need to deploy the Fluo binaries to every node on your
-cluster or start processes on every node.
-
-You can use the following command to check the status of your instance:
-
-    fluo status myapp
-
-For more detailed information on the YARN containers running Fluo:
-
-    fluo info myapp
-
-You can also use `yarn application -list` to check the status of your Fluo instance
-in YARN.  Logs are viewable within YARN.
-
-When you have data in your fluo instance, you can view it using the command `fluo scan`.
-Pipe the output to `less` using the command `fluo scan | less` if you want to page 
-through the data.
-
-Stop your Fluo application
---------------------------
-
-Use the following command to stop your Fluo application:
-
-    fluo stop myapp
-
-If stop fails, there is also a kill command.
-
-    fluo kill myapp
-
-Tuning Accumulo
----------------
-
-Fluo will reread the same data frequently when it checks conditions on
-mutations.   When Fluo initializes a table it enables data caching to make
-this more efficient.  However you may need to increase the amount of memory
-available for caching in the tserver by increasing `tserver.cache.data.size`.
-Increasing this may require increasing the maximum tserver java heap size in
-`accumulo-env.sh`.
-
-Fluo will run many client threads, will want to ensure the tablet server
-has enough threads.  Should probably increase the
-`tserver.server.threads.minimum` Accumulo setting.
-
-Using at least Accumulo 1.6.1 is recommended because multiple performance bugs
-were fixed.
-
-Tuning YARN
------------
-
-When running Fluo oracles and workers in YARN, the number of instances, max memory, and number
-of cores for Fluo processes can be configured in [fluo.properties]. If YARN is killing processes
-consider increasing `twill.java.reserved.memory.mb` (which defaults to 200 and is set in
yarn-site.xml).
-The `twill.java.reserved.memory.mb` config determines the gap between the YARN memory limit
set in
-[fluo.properties] and the java -Xmx setting.  For example, if max memory is 1024 and twill
reserved
-memory is 200, the java -Xmx setting will be 1024-200 = 824 MB.
-
-Run locally without YARN
-------------------------
-
-If you do not have YARN set up, you can start the oracle and worker as a local 
-Fluo process using the following commands:
-
-    local-fluo start-oracle
-    local-fluo start-worker
-
-Use the following commands to stop a local Fluo process:
-
-    local-fluo stop-worker
-    local-fluo stop-oracle
-
-In a distributed environment, you will need to deploy and configure a Fluo 
-distribution on every node in your cluster.
-
-[fluo-dev]: https://github.com/fluo-io/fluo-dev
-[Accumulo]: https://accumulo.apache.org/
-[Hadoop]: http://hadoop.apache.org/
-[Zookeeper]: http://zookeeper.apache.org/
-[Java]: http://openjdk.java.net/
-[release]: https://fluo.apache.org/download/
-[phrasecount]: https://github.com/fluo-io/phrasecount
-[fluo.properties]: ../modules/distribution/src/main/config/fluo.properties
-[fluo-env.sh]: ../modules/distribution/src/main/config/fluo-env.sh
-[lib/ahz/pom.xml]: ../modules/distribution/src/main/lib/ahz/pom.xml

http://git-wip-us.apache.org/repos/asf/incubator-fluo/blob/6aa0d438/modules/core/src/main/java/org/apache/fluo/core/oracle/FluoOracleImpl.java
----------------------------------------------------------------------
diff --git a/modules/core/src/main/java/org/apache/fluo/core/oracle/FluoOracleImpl.java b/modules/core/src/main/java/org/apache/fluo/core/oracle/FluoOracleImpl.java
index 6851e01..c0f5f11 100644
--- a/modules/core/src/main/java/org/apache/fluo/core/oracle/FluoOracleImpl.java
+++ b/modules/core/src/main/java/org/apache/fluo/core/oracle/FluoOracleImpl.java
@@ -19,8 +19,6 @@ import java.io.File;
 import java.util.Objects;
 
 import com.google.common.base.Preconditions;
-import com.google.common.util.concurrent.AbstractIdleService;
-import com.google.common.util.concurrent.UncheckedExecutionException;
 import org.apache.curator.framework.recipes.cache.NodeCache;
 import org.apache.fluo.api.config.FluoConfiguration;
 import org.apache.fluo.api.exceptions.FluoException;
@@ -36,55 +34,28 @@ public class FluoOracleImpl implements FluoOracle {
 
   private static final Logger log = LoggerFactory.getLogger(FluoOracleImpl.class);
 
-  private OracleService oracleService;
+  private FluoConfiguration config;
+  private Environment env;
+  private AutoCloseable reporters;
+  private OracleServer oracleServer;
+  private NodeCache appIdCache;
 
   public FluoOracleImpl(FluoConfiguration config) {
-    this.oracleService = new OracleService(config);
-  }
-
-  @Override
-  public void start() {
+    Objects.requireNonNull(config);
+    Preconditions.checkArgument(config.hasRequiredOracleProps());
+    // any client in oracle should retry forever
+    config.setClientRetryTimeout(-1);
     try {
-      oracleService.startAndWait();
-    } catch (UncheckedExecutionException e) {
-      throw new FluoException(e);
+      config.validate();
+    } catch (Exception e) {
+      throw new IllegalArgumentException("Invalid FluoConfiguration", e);
     }
+    this.config = config;
   }
 
   @Override
-  public void stop() {
+  public void start() {
     try {
-      oracleService.stopAndWait();
-    } catch (UncheckedExecutionException e) {
-      throw new FluoException(e);
-    }
-  }
-
-  private static class OracleService extends AbstractIdleService {
-
-    private static final Logger log = LoggerFactory.getLogger(OracleService.class);
-
-    private FluoConfiguration config;
-    private Environment env;
-    private AutoCloseable reporters;
-    private OracleServer oracleServer;
-    private NodeCache appIdCache;
-
-    OracleService(FluoConfiguration config) {
-      Objects.requireNonNull(config);
-      Preconditions.checkArgument(config.hasRequiredOracleProps());
-      // any client in oracle should retry forever
-      config.setClientRetryTimeout(-1);
-      try {
-        config.validate();
-      } catch (Exception e) {
-        throw new IllegalArgumentException("Invalid FluoConfiguration", e);
-      }
-      this.config = config;
-    }
-
-    @Override
-    protected void startUp() throws Exception {
       env = new Environment(config);
       reporters = ReporterUtil.setupReporters(env);
       appIdCache = CuratorUtil.startAppIdWatcher(env);
@@ -95,14 +66,20 @@ public class FluoOracleImpl implements FluoOracle {
 
       oracleServer = new OracleServer(env);
       oracleServer.start();
+    } catch (Exception e) {
+      throw new FluoException(e);
     }
+  }
 
-    @Override
-    protected void shutDown() throws Exception {
+  @Override
+  public void stop() {
+    try {
       oracleServer.stop();
       appIdCache.close();
       reporters.close();
       env.close();
+    } catch (Exception e) {
+      throw new FluoException(e);
     }
   }
 

http://git-wip-us.apache.org/repos/asf/incubator-fluo/blob/6aa0d438/modules/core/src/main/java/org/apache/fluo/core/worker/FluoWorkerImpl.java
----------------------------------------------------------------------
diff --git a/modules/core/src/main/java/org/apache/fluo/core/worker/FluoWorkerImpl.java b/modules/core/src/main/java/org/apache/fluo/core/worker/FluoWorkerImpl.java
index e5b1bf2..47c9333 100644
--- a/modules/core/src/main/java/org/apache/fluo/core/worker/FluoWorkerImpl.java
+++ b/modules/core/src/main/java/org/apache/fluo/core/worker/FluoWorkerImpl.java
@@ -19,8 +19,6 @@ import java.io.File;
 import java.util.Objects;
 
 import com.google.common.base.Preconditions;
-import com.google.common.util.concurrent.AbstractIdleService;
-import com.google.common.util.concurrent.UncheckedExecutionException;
 import org.apache.curator.framework.recipes.cache.NodeCache;
 import org.apache.fluo.api.config.FluoConfiguration;
 import org.apache.fluo.api.exceptions.FluoException;
@@ -36,49 +34,22 @@ public class FluoWorkerImpl implements FluoWorker {
 
   private static final Logger log = LoggerFactory.getLogger(FluoWorkerImpl.class);
 
-  private WorkerService workerService;
+  private FluoConfiguration config;
+  private Environment env;
+  private AutoCloseable reporters;
+  private NotificationProcessor np;
+  private NotificationFinder notificationFinder;
+  private NodeCache appIdCache;
 
   public FluoWorkerImpl(FluoConfiguration config) {
-    this.workerService = new WorkerService(config);
+    Objects.requireNonNull(config);
+    Preconditions.checkArgument(config.hasRequiredWorkerProps());
+    this.config = config;
   }
 
   @Override
   public void start() {
     try {
-      workerService.startAndWait();
-    } catch (UncheckedExecutionException e) {
-      throw new FluoException(e);
-    }
-  }
-
-  @Override
-  public void stop() {
-    try {
-      workerService.stopAndWait();
-    } catch (UncheckedExecutionException e) {
-      throw new FluoException(e);
-    }
-  }
-
-  private static class WorkerService extends AbstractIdleService {
-
-    private static final Logger log = LoggerFactory.getLogger(WorkerService.class);
-
-    private FluoConfiguration config;
-    private Environment env;
-    private AutoCloseable reporters;
-    private NotificationProcessor np;
-    private NotificationFinder notificationFinder;
-    private NodeCache appIdCache;
-
-    WorkerService(FluoConfiguration config) {
-      Objects.requireNonNull(config);
-      Preconditions.checkArgument(config.hasRequiredWorkerProps());
-      this.config = config;
-    }
-
-    @Override
-    protected void startUp() throws Exception {
       env = new Environment(config);
       reporters = ReporterUtil.setupReporters(env);
       appIdCache = CuratorUtil.startAppIdWatcher(env);
@@ -91,15 +62,21 @@ public class FluoWorkerImpl implements FluoWorker {
       notificationFinder = NotificationFinderFactory.newNotificationFinder(env.getConfiguration());
       notificationFinder.init(env, np);
       notificationFinder.start();
+    } catch (Exception e) {
+      throw new FluoException(e);
     }
+  }
 
-    @Override
-    protected void shutDown() throws Exception {
+  @Override
+  public void stop() {
+    try {
       notificationFinder.stop();
       np.close();
       appIdCache.close();
       reporters.close();
       env.close();
+    } catch (Exception e) {
+      throw new FluoException(e);
     }
   }
 

http://git-wip-us.apache.org/repos/asf/incubator-fluo/blob/6aa0d438/modules/distribution/src/main/lib/ahz/pom.xml
----------------------------------------------------------------------
diff --git a/modules/distribution/src/main/lib/ahz/pom.xml b/modules/distribution/src/main/lib/ahz/pom.xml
index d08c597..ef8a6bd 100644
--- a/modules/distribution/src/main/lib/ahz/pom.xml
+++ b/modules/distribution/src/main/lib/ahz/pom.xml
@@ -20,8 +20,8 @@
   <version>a</version>
 
   <properties>
-    <accumulo.version>1.7.1</accumulo.version>
-    <hadoop.version>2.6.3</hadoop.version>
+    <accumulo.version>1.7.2</accumulo.version>
+    <hadoop.version>2.7.2</hadoop.version>
     <zookeeper.version>3.4.8</zookeeper.version>
   </properties>
 

http://git-wip-us.apache.org/repos/asf/incubator-fluo/blob/6aa0d438/modules/distribution/src/main/lib/fetch.sh
----------------------------------------------------------------------
diff --git a/modules/distribution/src/main/lib/fetch.sh b/modules/distribution/src/main/lib/fetch.sh
index 37fbff2..95765a2 100755
--- a/modules/distribution/src/main/lib/fetch.sh
+++ b/modules/distribution/src/main/lib/fetch.sh
@@ -108,6 +108,6 @@ extra)
   echo -e "   extra     Download extra Fluo dependencies\n"
   echo "For 'ahz', the versions of Hadoop, Accumulo, & Zookeeper are specifed in ahz/pom.xml."
   echo -e "However, you can override them using the command below:\n"
-  echo "./fetch.sh ahz -Daccumulo.version=1.7.1 -Dhadoop.version=2.6.3 -Dzookeeper.version=3.4.8"
+  echo "./fetch.sh ahz -Daccumulo.version=1.7.2 -Dhadoop.version=2.7.2 -Dzookeeper.version=3.4.8"
 esac
 

http://git-wip-us.apache.org/repos/asf/incubator-fluo/blob/6aa0d438/pom.xml
----------------------------------------------------------------------
diff --git a/pom.xml b/pom.xml
index 2d0b100..9904e49 100644
--- a/pom.xml
+++ b/pom.xml
@@ -83,6 +83,7 @@
         <version>1.32</version>
       </dependency>
       <dependency>
+        <!-- Guava 13.0.1 is required by Twill (due to beta method usage) -->
         <groupId>com.google.guava</groupId>
         <artifactId>guava</artifactId>
         <version>13.0.1</version>


Mime
View raw message