fluo-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From mwa...@apache.org
Subject [fluo-yarn] branch master updated: Initial implementation of Apache Fluo YARN launcher (#1)
Date Tue, 15 Aug 2017 15:01:03 GMT
This is an automated email from the ASF dual-hosted git repository.

mwalch pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/fluo-yarn.git


The following commit(s) were added to refs/heads/master by this push:
     new 5df10b3  Initial implementation of Apache Fluo YARN launcher (#1)
5df10b3 is described below

commit 5df10b34e43f8ab600935102286dd298c0d77292
Author: Mike Walch <mwalch@apache.org>
AuthorDate: Tue Aug 15 11:01:01 2017 -0400

    Initial implementation of Apache Fluo YARN launcher (#1)
---
 .gitignore                                         |   8 +
 README.md                                          |  90 ++++++++
 core/pom.xml                                       |  68 ++++++
 .../org/apache/fluo/yarn/core/FluoYarnEnv.java     | 132 +++++++++++
 .../apache/fluo/yarn/core/FluoYarnLauncher.java    | 149 +++++++++++++
 .../apache/fluo/yarn/core/FluoYarnProperties.java  |  66 ++++++
 distribution/bin/fluo-yarn                         | 102 +++++++++
 distribution/conf/fluo-yarn-env.sh                 |  73 +++++++
 distribution/conf/fluo-yarn.properties             |  51 +++++
 distribution/conf/log4j.properties                 |  25 +++
 distribution/conf/logback.xml                      |  35 +++
 distribution/lib/fetch.sh                          |  83 +++++++
 distribution/pom.xml                               |  90 ++++++++
 distribution/src/main/assembly/bin.xml             |  73 +++++++
 pom.xml                                            | 241 +++++++++++++++++++++
 15 files changed, 1286 insertions(+)

diff --git a/.gitignore b/.gitignore
new file mode 100644
index 0000000..ccc651c
--- /dev/null
+++ b/.gitignore
@@ -0,0 +1,8 @@
+.classpath
+.project
+.settings
+target
+logs
+*.iml
+.idea
+.DS_Store
diff --git a/README.md b/README.md
index ac14fd2..25b3a3e 100644
--- a/README.md
+++ b/README.md
@@ -18,3 +18,93 @@ limitations under the License.
 # Fluo YARN launcher
 
 A tool for running Apache Fluo applications in Hadoop YARN.
+
+## Requirements
+
+The Fluo YARN launcher requires [Apache Fluo][Fluo] and [Hadoop YARN][YARN]:
+
+| Software    | Recommended Version | Minimum Version |
+|-------------|---------------------|-----------------|
+| [Fluo]      | 1.2.0               | 1.2.0           |
+| [YARN]      | 2.7.2               | 2.6.0           |
+
+See the [related projects page][related] for external projects that may help in setting up these dependencies.
+
+## Set up your Fluo application
+
+Before you can launch a Fluo application in YARN, you should follow Fluo's [install instructions][install]
+to install Fluo and initialize a Fluo application. After your application has been initialized, follow
+the insructions below to install the Fluo YARN launcher and run your application in YARN. Avoid using the
+`fluo` command to start local oracle and worker processes if you are running in YARN.
+
+## Install and Configure Fluo YARN launcher
+
+Before you can install the Fluo YARN launcher, you will need to obtain a distribution tarball. It is
+recommended that you download the [latest release][release]. You can also build a distribution from the
+master branch by following these steps which create a tarball in `distribution/target`:
+
+    git clone https://github.com/apache/fluo-yarn.git
+    cd fluo-yarn/
+    mvn package
+
+After you obtain a Fluo YARN distribution tarball, follow these steps to install Fluo.
+
+1. Choose a directory with plenty of space, untar the distribution, and run `fetch.sh` to retrieve dependencies:
+
+        tar -xvzf fluo-yarn-1.0.0-bin.tar.gz
+        cd fluo-yarn-1.0.0
+        ./lib/fetch.sh
+
+    The distribution contains a `fluo-yarn` script in `bin/` that administers Fluo and the
+    following configuration files in `conf/`:
+
+    | Configuration file          | Description                                                             |
+    |-----------------------------|-------------------------------------------------------------------------|
+    | [fluo-yarn-env.sh]          | Configures classpath for `fluo-yarn` script. Required for all commands. |
+    | [fluo-yarn.properties]      | Configures how application runs in YARN.  Required for `start` command. |
+    | [log4j.properties]          | Configures logging                                                      |
+
+2. Configure [fluo-yarn-env.sh]
+
+    * Set `FLUO_HOME` if it is not in your environment
+    * Modify `FLUO_CONN_PROPS` if you don't want use the default.
+
+3. Configure [fluo-yarn.properties] to set how your application will be launched in YARN:
+
+    * YARN resource manager hostname
+    * Number of oracle and worker instances
+    * Max memory usage per oracle/worker
+
+   If you are managing multiple Fluo applications in YARN, you can copy this file and configure it for
+   each application.
+
+## Start Fluo application in YARN
+
+Follow the instructions below to start a Fluo application which contain an oracle and multiple workers.
+
+1. Configure [fluo-yarn-env.sh] and [fluo-yarn.properties] if you have not already.
+
+2. Run the commands below to start your Fluo application in YARN.
+
+        fluo-yarn start myapp conf/fluo-yarn.properties
+
+   The commands will retrieve your application configuration and observer jars (using your
+   application name) before starting the application in YARN.
+
+## Manage Fluo application in YARN
+
+Except for stopping your application in YARN, the `fluo` script can be used to manage your application using the
+`scan` and `wait` commands.  See Fluo's [install instruction][install] for more information.
+
+When you want you stop your Fluo application, use the the YARN resource manager or the 
+`yarn application -kill <App ID>` to stop the application in YARN.
+
+[Fluo]: https://fluo.apache.org/
+[YARN]: http://hadoop.apache.org/
+[related]: https://fluo.apache.org/related-projects/
+[related]: https://fluo.apache.org/related-projects/
+[install]: https://github.com/apache/incubator-fluo/blob/master/docs/install.md
+[release]: https://fluo.apache.org/download/
+[fluo-yarn-env.sh]: distribution/conf/fluo-yarn-env.sh
+[fluo-yarn.properties]: distribution/conf/fluo-yarn.properties
+[log4j.properties]: distribution/conf/log4j.properties
diff --git a/core/pom.xml b/core/pom.xml
new file mode 100644
index 0000000..7b6f455
--- /dev/null
+++ b/core/pom.xml
@@ -0,0 +1,68 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+  Licensed to the Apache Software Foundation (ASF) under one or more contributor license
+  agreements. See the NOTICE file distributed with this work for additional information regarding
+  copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the
+  "License"); you may not use this file except in compliance with the License. You may obtain a
+  copy of the License at
+
+     http://www.apache.org/licenses/LICENSE-2.0
+
+  Unless required by applicable law or agreed to in writing, software distributed under the License
+  is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
+  or implied. See the License for the specific language governing permissions and limitations under
+  the License.
+-->
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+  <modelVersion>4.0.0</modelVersion>
+  <parent>
+    <groupId>org.apache.fluo</groupId>
+    <artifactId>fluo-yarn-project</artifactId>
+    <version>1.0.0-SNAPSHOT</version>
+    <relativePath>../pom.xml</relativePath>
+  </parent>
+  <artifactId>fluo-yarn-core</artifactId>
+  <name>Apache Fluo YARN Core</name>
+  <description>This module contains all code necessary to run Apache Fluo on a YARN cluster using Apache Twill</description>
+  <dependencies>
+    <dependency>
+      <groupId>com.google.guava</groupId>
+      <artifactId>guava</artifactId>
+    </dependency>
+    <dependency>
+      <groupId>org.apache.fluo</groupId>
+      <artifactId>fluo-api</artifactId>
+    </dependency>
+    <dependency>
+      <groupId>org.apache.fluo</groupId>
+      <artifactId>fluo-core</artifactId>
+    </dependency>
+    <dependency>
+      <groupId>org.apache.hadoop</groupId>
+      <artifactId>hadoop-yarn-client</artifactId>
+      <exclusions>
+        <!-- Excluded so we use asm jar included by twill-yarn.  See FLUO-409 -->
+        <exclusion>
+          <groupId>asm</groupId>
+          <artifactId>asm</artifactId>
+        </exclusion>
+      </exclusions>
+    </dependency>
+    <dependency>
+      <groupId>org.apache.twill</groupId>
+      <artifactId>twill-api</artifactId>
+    </dependency>
+    <dependency>
+      <groupId>org.apache.twill</groupId>
+      <artifactId>twill-ext</artifactId>
+    </dependency>
+    <dependency>
+      <groupId>org.apache.twill</groupId>
+      <artifactId>twill-yarn</artifactId>
+    </dependency>
+    <dependency>
+      <groupId>org.slf4j</groupId>
+      <artifactId>slf4j-api</artifactId>
+    </dependency>
+  </dependencies>
+</project>
diff --git a/core/src/main/java/org/apache/fluo/yarn/core/FluoYarnEnv.java b/core/src/main/java/org/apache/fluo/yarn/core/FluoYarnEnv.java
new file mode 100644
index 0000000..dcf2041
--- /dev/null
+++ b/core/src/main/java/org/apache/fluo/yarn/core/FluoYarnEnv.java
@@ -0,0 +1,132 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more contributor license
+ * agreements. See the NOTICE file distributed with this work for additional information regarding
+ * copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License. You may obtain a
+ * copy of the License at
+ * 
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * 
+ * Unless required by applicable law or agreed to in writing, software distributed under the License
+ * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
+ * or implied. See the License for the specific language governing permissions and limitations under
+ * the License.
+ */
+
+package org.apache.fluo.yarn.core;
+
+import java.io.File;
+import java.nio.file.Paths;
+import java.util.Objects;
+import java.util.Properties;
+
+import com.google.common.base.Preconditions;
+import org.apache.hadoop.yarn.conf.YarnConfiguration;
+
+public class FluoYarnEnv {
+
+  private Properties props;
+  private String applicationName;
+  private String connPropsPath;
+  private String logPropsPath;
+  private String bundledJarPath = null;
+  private YarnConfiguration yarnConfig = null;
+
+  public FluoYarnEnv(String yarnProps, String connProps, String logProps, String appName,
+      String jarPath) {
+    Objects.requireNonNull(yarnProps);
+    Objects.requireNonNull(connProps);
+    Objects.requireNonNull(logProps);
+    Objects.requireNonNull(appName);
+    Objects.requireNonNull(jarPath);
+    verifyPath(connProps);
+    verifyPath(yarnProps);
+    verifyPath(logProps);
+    verifyPath(jarPath);
+    props = FluoYarnProperties.loadFromFile(yarnProps);
+    connPropsPath = connProps;
+    logPropsPath = logProps;
+    applicationName = appName;
+    bundledJarPath = jarPath;
+
+  }
+
+  public String getBundledJarPath() {
+    return bundledJarPath;
+  }
+
+  public String getBundledJarName() {
+    return Paths.get(getBundledJarPath()).getFileName().toString();
+  }
+
+  public String getLogPropsPath() {
+    return logPropsPath;
+  }
+
+  public String getConnPropsPath() {
+    return connPropsPath;
+  }
+
+  public String getApplicationName() {
+    return applicationName;
+  }
+
+  private static void verifyPath(String path) {
+    File f = new File(path);
+    Preconditions.checkState(f.exists());
+    Preconditions.checkState(f.canRead());
+  }
+
+  public YarnConfiguration getYarnConfiguration() {
+    if (yarnConfig == null) {
+      yarnConfig = new YarnConfiguration();
+      yarnConfig.set("mapreduce.framework.name", "yarn");
+      yarnConfig.set("yarn.resourcemanager.hostname", getYarnResourceManager());
+    }
+    return yarnConfig;
+  }
+
+  public String getYarnResourceManager() {
+    return props.getProperty(FluoYarnProperties.YARN_RESOURCE_MANAGER_PROP,
+        FluoYarnProperties.YARN_RESOURCE_MANAGER_DEFAULT);
+  }
+
+  public String getZookeepers() {
+    return props.getProperty(FluoYarnProperties.ZOOKEEPERS_PROP,
+        FluoYarnProperties.ZOOKEEPERS_DEFAULT);
+  }
+
+  public int getWorkerCores() {
+    return Integer.valueOf(props.getProperty(FluoYarnProperties.WORKER_NUM_CORES_PROP,
+        FluoYarnProperties.WORKER_NUM_CORES_DEFAULT));
+  }
+
+  public int getWorkerInstances() {
+    return Integer.valueOf(props.getProperty(FluoYarnProperties.WORKER_INSTANCES_PROP,
+        FluoYarnProperties.WORKER_INSTANCES_DEFAULT));
+  }
+
+  public int getWorkerMaxMemory() {
+    return Integer.valueOf(props.getProperty(FluoYarnProperties.WORKER_MAX_MEMORY_MB_PROP,
+        FluoYarnProperties.WORKER_MAX_MEMORY_MB_DEFAULT));
+  }
+
+  public int getOracleCores() {
+    return Integer.valueOf(props.getProperty(FluoYarnProperties.ORACLE_NUM_CORES_PROP,
+        FluoYarnProperties.ORACLE_NUM_CORES_DEFAULT));
+  }
+
+  public int getOracleInstances() {
+    return Integer.valueOf(props.getProperty(FluoYarnProperties.ORACLE_INSTANCES_PROP,
+        FluoYarnProperties.ORACLE_INSTANCES_DEFAULT));
+  }
+
+  public int getOracleMaxMemory() {
+    return Integer.valueOf(props.getProperty(FluoYarnProperties.ORACLE_MAX_MEMORY_MB_PROP,
+        FluoYarnProperties.ORACLE_MAX_MEMORY_MB_DEFAULT));
+  }
+
+  public int getTotalInstances() {
+    return getOracleInstances() + getWorkerInstances();
+  }
+}
diff --git a/core/src/main/java/org/apache/fluo/yarn/core/FluoYarnLauncher.java b/core/src/main/java/org/apache/fluo/yarn/core/FluoYarnLauncher.java
new file mode 100644
index 0000000..7da612e
--- /dev/null
+++ b/core/src/main/java/org/apache/fluo/yarn/core/FluoYarnLauncher.java
@@ -0,0 +1,149 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more contributor license
+ * agreements. See the NOTICE file distributed with this work for additional information regarding
+ * copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License. You may obtain a
+ * copy of the License at
+ * 
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * 
+ * Unless required by applicable law or agreed to in writing, software distributed under the License
+ * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
+ * or implied. See the License for the specific language governing permissions and limitations under
+ * the License.
+ */
+
+package org.apache.fluo.yarn.core;
+
+import java.io.File;
+import java.util.Collection;
+
+import org.apache.twill.api.ResourceReport;
+import org.apache.twill.api.ResourceSpecification;
+import org.apache.twill.api.TwillApplication;
+import org.apache.twill.api.TwillController;
+import org.apache.twill.api.TwillPreparer;
+import org.apache.twill.api.TwillRunResources;
+import org.apache.twill.api.TwillRunnerService;
+import org.apache.twill.api.TwillSpecification;
+import org.apache.twill.ext.BundledJarRunnable;
+import org.apache.twill.ext.BundledJarRunner;
+import org.apache.twill.yarn.YarnTwillRunnerService;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+public class FluoYarnLauncher {
+
+  private static final Logger log = LoggerFactory.getLogger(FluoYarnLauncher.class);
+
+  private static final String ORACLE_ID = "Oracle";
+  private static final String WORKER_ID = "Worker";
+  private static final String CONN_PROPS = "fluo-conn.properties";
+  private static final String LOG4J_PROPS = "log4j.properties";
+
+  private static class FluoYarnApp implements TwillApplication {
+
+    private FluoYarnEnv env;
+
+    FluoYarnApp(FluoYarnEnv env) {
+      this.env = env;
+    }
+
+    @Override
+    public TwillSpecification configure() {
+      ResourceSpecification oracleResources =
+          ResourceSpecification.Builder.with().setVirtualCores(env.getOracleCores())
+              .setMemory(env.getOracleMaxMemory(), ResourceSpecification.SizeUnit.MEGA)
+              .setInstances(env.getOracleInstances()).build();
+
+      ResourceSpecification workerResources =
+          ResourceSpecification.Builder.with().setVirtualCores(env.getWorkerCores())
+              .setMemory(env.getWorkerMaxMemory(), ResourceSpecification.SizeUnit.MEGA)
+              .setInstances(env.getWorkerInstances()).build();
+
+      return TwillSpecification.Builder.with().setName("fluo-app-" + env.getApplicationName())
+          .withRunnable().add(ORACLE_ID, new BundledJarRunnable(), oracleResources)
+          .withLocalFiles().add(env.getBundledJarName(), new File(env.getBundledJarPath()), false)
+          .add(CONN_PROPS, new File(env.getConnPropsPath()), false)
+          .add(LOG4J_PROPS, new File(env.getLogPropsPath()), false).apply()
+          .add(WORKER_ID, new BundledJarRunnable(), workerResources).withLocalFiles()
+          .add(env.getBundledJarName(), new File(env.getBundledJarPath()), false)
+          .add(CONN_PROPS, new File(env.getConnPropsPath()), false)
+          .add(LOG4J_PROPS, new File(env.getLogPropsPath()), false).apply().anyOrder().build();
+    }
+  }
+
+  private static int getNumRunning(TwillController controller) {
+    ResourceReport report = controller.getResourceReport();
+    if (report == null) {
+      return 0;
+    }
+    int total = 0;
+    Collection<TwillRunResources> resources = report.getRunnableResources(ORACLE_ID);
+    if (resources != null) {
+      total += resources.size();
+    }
+    resources = report.getRunnableResources(WORKER_ID);
+    if (resources != null) {
+      total += resources.size();
+    }
+    return total;
+  }
+
+  public static void main(String[] args) throws Exception {
+
+    if (args.length != 5) {
+      System.err.println("Invalid arguments");
+      System.exit(-1);
+    }
+
+    String connProps = args[0];
+    String yarnProps = args[1];
+    String logProps = args[2];
+    String appName = args[3];
+    String jarPath = args[4];
+
+    FluoYarnEnv env = new FluoYarnEnv(yarnProps, connProps, logProps, appName, jarPath);
+
+    BundledJarRunner.Arguments oracleArgs =
+        new BundledJarRunner.Arguments.Builder().setJarFileName(env.getBundledJarName())
+            .setLibFolder("lib").setMainClassName("org.apache.fluo.command.FluoOracle")
+            .setMainArgs(new String[] {CONN_PROPS, appName}).createArguments();
+
+    BundledJarRunner.Arguments workerArgs =
+        new BundledJarRunner.Arguments.Builder().setJarFileName(env.getBundledJarName())
+            .setLibFolder("lib").setMainClassName("org.apache.fluo.command.FluoWorker")
+            .setMainArgs(new String[] {CONN_PROPS, appName}).createArguments();
+
+    TwillRunnerService twillRunner =
+        new YarnTwillRunnerService(env.getYarnConfiguration(), env.getZookeepers());
+    twillRunner.start();
+
+    TwillPreparer preparer =
+        twillRunner.prepare(new FluoYarnApp(env))
+            .addJVMOptions("-Dlog4j.configuration=file:$PWD/" + LOG4J_PROPS)
+            .withArguments(ORACLE_ID, oracleArgs.toArray())
+            .withArguments(WORKER_ID, workerArgs.toArray());
+
+    TwillController controller = preparer.start();
+
+    ResourceReport report = controller.getResourceReport();
+    log.info("Waiting for Fluo application '{}' to start in YARN...", appName);
+    while (report == null) {
+      Thread.sleep(500);
+      report = controller.getResourceReport();
+    }
+    String appID = report.getApplicationId();
+    log.info("Fluo application '{}' has started in YARN with ID '{}'", appName, appID);
+
+    log.info("Waiting for all containers of Fluo application '{}' to start in YARN...", appName);
+    int numRunning = getNumRunning(controller);
+    while (numRunning != env.getTotalInstances()) {
+      log.info("{} of {} containers have started in YARN", numRunning, env.getTotalInstances());
+      Thread.sleep(2000);
+      numRunning = getNumRunning(controller);
+    }
+    log.info("{} of {} containers have started in YARN", numRunning, env.getTotalInstances());
+    log.info("Fluo application '{}' has successfully started in YARN with ID '{}'", appName, appID);
+  }
+}
diff --git a/core/src/main/java/org/apache/fluo/yarn/core/FluoYarnProperties.java b/core/src/main/java/org/apache/fluo/yarn/core/FluoYarnProperties.java
new file mode 100644
index 0000000..4d34647
--- /dev/null
+++ b/core/src/main/java/org/apache/fluo/yarn/core/FluoYarnProperties.java
@@ -0,0 +1,66 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more contributor license
+ * agreements. See the NOTICE file distributed with this work for additional information regarding
+ * copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License. You may obtain a
+ * copy of the License at
+ * 
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * 
+ * Unless required by applicable law or agreed to in writing, software distributed under the License
+ * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
+ * or implied. See the License for the specific language governing permissions and limitations under
+ * the License.
+ */
+
+package org.apache.fluo.yarn.core;
+
+import java.io.FileInputStream;
+import java.io.IOException;
+import java.util.Properties;
+
+public class FluoYarnProperties {
+
+  private static final String YARN_PREFIX = "fluo.yarn";
+
+  // General properties
+  public static final String YARN_RESOURCE_MANAGER_PROP = YARN_PREFIX + ".resource.manager";
+  public static final String ZOOKEEPERS_PROP = YARN_PREFIX + ".zookeepers";
+  public static final String YARN_RESOURCE_MANAGER_DEFAULT = "localhost";
+  public static final String ZOOKEEPERS_DEFAULT = "localhost/fluo-yarn";
+
+  // Worker properties
+  public static final String WORKER_INSTANCES_PROP = YARN_PREFIX + ".worker.instances";
+  public static final String WORKER_MAX_MEMORY_MB_PROP = YARN_PREFIX + ".worker.max.memory.mb";
+  public static final String WORKER_NUM_CORES_PROP = YARN_PREFIX + ".worker.num.cores";
+  public static final String WORKER_INSTANCES_DEFAULT = "1";
+  public static final String WORKER_MAX_MEMORY_MB_DEFAULT = "1024";
+  public static final String WORKER_NUM_CORES_DEFAULT = "1";
+
+  // Oracle properties
+  public static final String ORACLE_INSTANCES_PROP = YARN_PREFIX + ".oracle.instances";
+  public static final String ORACLE_MAX_MEMORY_MB_PROP = YARN_PREFIX + ".oracle.max.memory.mb";
+  public static final String ORACLE_NUM_CORES_PROP = YARN_PREFIX + ".oracle.num.cores";
+  public static final String ORACLE_INSTANCES_DEFAULT = "1";
+  public static final String ORACLE_MAX_MEMORY_MB_DEFAULT = "512";
+  public static final String ORACLE_NUM_CORES_DEFAULT = "1";
+
+  public static Properties loadFromFile(String propsFilePath) {
+    try {
+      return loadFromStream(new FileInputStream(propsFilePath));
+    } catch (IOException e) {
+      throw new IllegalStateException(e);
+    }
+  }
+
+  public static Properties loadFromStream(FileInputStream fis) {
+    Properties props = new Properties();
+    try {
+      props.load(fis);
+      fis.close();
+    } catch (IOException e) {
+      throw new IllegalStateException(e);
+    }
+    return props;
+  }
+}
diff --git a/distribution/bin/fluo-yarn b/distribution/bin/fluo-yarn
new file mode 100755
index 0000000..367dec4
--- /dev/null
+++ b/distribution/bin/fluo-yarn
@@ -0,0 +1,102 @@
+#! /usr/bin/env bash
+
+# Licensed to the Apache Software Foundation (ASF) under one or more contributor license
+# agreements. See the NOTICE file distributed with this work for additional information regarding
+# copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance with the License. You may obtain a
+# copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software distributed under the License
+# is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
+# or implied. See the License for the specific language governing permissions and limitations under
+# the License.
+
+SOURCE="${BASH_SOURCE[0]}"
+while [ -h "$SOURCE" ]; do
+   bin="$( cd -P "$( dirname "$SOURCE" )" && pwd )"
+   SOURCE="$(readlink "$SOURCE")"
+   [[ $SOURCE != /* ]] && SOURCE="$bin/$SOURCE"
+done
+# Set up variables needed by fluo-env.sh
+export bin="$( cd -P "$( dirname "$SOURCE" )" && pwd )"
+export basedir="$( cd -P ${bin}/.. && pwd )"
+export conf="$basedir/conf"
+export lib="$basedir/lib"
+export cmd="$1"
+case "$cmd" in
+  start) export app="$2" ;;
+esac
+export FLUO_YARN_VERSION=${project.version}
+
+if [ ! -f "$conf/fluo-yarn-env.sh" ]; then
+  echo "fluo-yarn-env.sh must exist in $conf"
+  exit 1
+fi
+source "$conf/fluo-yarn-env.sh"
+export CLASSPATH=$LAUNCHER_CLASSPATH
+
+# stop if any command fails
+set -e
+
+function print_usage {
+  echo -e "Usage: fluo-yarn <command> (<argument> ...)\n"
+  echo -e "Possible commands:\n"
+  echo "  classpath                   Prints the classpath setup by fluo-yarn-env.sh"
+  echo "  start <app> <yarnProps>     Starts Fluo <app> in YARN using <yarnProps>"
+  echo "  version                     Prints the version of Fluo YARN launcher"
+  echo " "
+  exit 1
+}
+
+function build_bundled_jar {
+  app_dir=$lib/apps/$app
+  mkdir -p "$app_dir/lib"
+  $FLUO_HOME/bin/fluo get-jars "$app" "$app_dir/lib"
+
+  fluo_classpath=$FLUO_CLASSPATH
+
+  IFS=':' read -ra ADDR <<< "$fluo_classpath"
+  for cpentry in "${ADDR[@]}"; do
+    if [[ $cpentry = *\* ]]; then
+      for f in $cpentry; do
+        if [[ -f $f ]]; then
+          cp "$f" "$app_dir/lib/"
+        fi
+      done
+    else
+      cp "$cpentry" "$app_dir/lib/"
+    fi
+  done
+
+  bundled_jar=fluo-app-${app}.jar
+
+  pushd "$app_dir" > /dev/null
+  jar cf $bundled_jar lib
+  popd > /dev/null
+}
+
+case "$1" in
+start)
+  if [ ! -f "$FLUO_CONN_PROPS" ]; then
+    echo "Fluo connection properties file '$FLUO_CONN_PROPS' does not exist"
+    exit 1
+  fi
+  if [ ! -f "$3" ]; then
+    echo "The file argument given for <yarnProps> does not exist at $3"
+    print_usage
+    exit 1
+  fi
+  build_bundled_jar
+  java org.apache.fluo.yarn.core.FluoYarnLauncher "$FLUO_CONN_PROPS" "$3" "$conf/log4j.properties" "$app" "$app_dir/$bundled_jar"
+  ;;
+classpath)
+  echo "$CLASSPATH"
+  ;;
+version)
+  echo "$FLUO_YARN_VERSION"
+  ;;
+*)
+  print_usage
+esac
diff --git a/distribution/conf/fluo-yarn-env.sh b/distribution/conf/fluo-yarn-env.sh
new file mode 100755
index 0000000..6b2f4f3
--- /dev/null
+++ b/distribution/conf/fluo-yarn-env.sh
@@ -0,0 +1,73 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more contributor license
+# agreements.  See the NOTICE file distributed with this work for additional information regarding
+# copyright ownership.  The ASF licenses this file to you under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with the License.  You may
+# obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software distributed under the License
+# is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
+# or implied. See the License for the specific language governing permissions and limitations under
+# the License.
+
+## Before fluo-yarn-env.sh is loaded, these environment variables are set and can be used in this file:
+
+# cmd - Command that is being called such as oracle, worker, etc.
+# app - Fluo application name 
+# basedir - Root of Fluo YARN launcher installation
+# conf - Directory containing Fluo YARN launcher configuration
+# lib - Directory containing Fluo YARN launcher libraries
+
+####################################
+# General variables that must be set
+####################################
+
+## Fluo installation
+export FLUO_HOME="${FLUO_HOME:-/path/to/fluo}"
+## Hadoop installation
+export HADOOP_PREFIX="${HADOOP_PREFIX:-/path/to/hadoop}"
+## Zookeeper installation
+export ZOOKEEPER_HOME="${ZOOKEEPER_HOME:-/path/to/zookeeper}"
+## File that contains properties need to connect to Fluo
+export FLUO_CONN_PROPS=${FLUO_CONN_PROPS:-$FLUO_HOME/conf/fluo-conn.properties}
+
+###########################
+# Build classpath variables
+###########################
+
+## A classpath needs to built for the YARN launcher and the launched Fluo application
+
+## Classpath for launched Fluo application
+export FLUO_CLASSPATH=$($FLUO_HOME/bin/fluo classpath)
+
+## Classpath of Fluo YARN Launcher
+addToClasspath()
+{
+  local dir=$1
+  local filterRegex=$2
+
+  if [ ! -d "$dir" ]; then
+    echo "ERROR $dir does not exist or not a directory"
+    exit 1
+  fi
+
+  for jar in $dir/*.jar; do
+    if ! [[ $jar =~ $filterRegex ]]; then
+       LAUNCHER_CLASSPATH="$LAUNCHER_CLASSPATH:$jar"
+    fi
+  done
+}
+
+# Any jars matching this pattern is excluded from classpath
+EXCLUDE_RE="(.*log4j.*)|(.*asm.*)|(.*guava.*)|(.*gson.*)"
+LAUNCHER_CLASSPATH="$conf:$lib/*"
+addToClasspath "$ZOOKEEPER_HOME" $EXCLUDE_RE
+addToClasspath "$ZOOKEEPER_HOME/lib" $EXCLUDE_RE
+addToClasspath "$HADOOP_PREFIX/share/hadoop/common" $EXCLUDE_RE
+addToClasspath "$HADOOP_PREFIX/share/hadoop/common/lib" $EXCLUDE_RE
+addToClasspath "$HADOOP_PREFIX/share/hadoop/hdfs" $EXCLUDE_RE
+addToClasspath "$HADOOP_PREFIX/share/hadoop/hdfs/lib" $EXCLUDE_RE
+addToClasspath "$HADOOP_PREFIX/share/hadoop/yarn" $EXCLUDE_RE
+addToClasspath "$HADOOP_PREFIX/share/hadoop/yarn/lib" $EXCLUDE_RE
+export LAUNCHER_CLASSPATH
diff --git a/distribution/conf/fluo-yarn.properties b/distribution/conf/fluo-yarn.properties
new file mode 100644
index 0000000..996b132
--- /dev/null
+++ b/distribution/conf/fluo-yarn.properties
@@ -0,0 +1,51 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more contributor license
+# agreements.  See the NOTICE file distributed with this work for additional information regarding
+# copyright ownership.  The ASF licenses this file to you under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with the License.  You may
+# obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software distributed under the License
+# is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
+# or implied. See the License for the specific language governing permissions and limitations under
+# the License.
+
+########################
+# Fluo YARN properties
+########################
+
+# NOTE - All properties that have a default are set with it. Uncomment
+# a property if you want to use a value different than the default. 
+# Properties that have no default are uncommented and must be set by
+# the user.
+
+## General properties
+## ------------------
+## Zookeeper connection string specifying host and chroot where Fluo YARN launcher stores data.
+## A chroot directory suffix (with max depth of 1) must be specified but doesn't need to be named '/fluo-yarn'.
+#fluo.yarn.zookeepers=localhost/fluo-yarn
+## YARN resource manager hostname. Should match 'yarn.resourcemanager.hostname' property in Hadoop's yarn-site.xml
+#fluo.yarn.resource.manager=localhost
+
+## Oracle properties
+## -----------------
+## Number of oracle yarn instances
+#fluo.yarn.oracle.instances=1
+## Max memory of Oracle yarn containers (in MB)
+#fluo.yarn.oracle.max.memory.mb=512
+## Number of Oracle virtual cores
+#fluo.yarn.oracle.num.cores=1
+
+## Worker properties
+## -----------------
+## Number of Worker instances
+#fluo.yarn.worker.instances=1
+## Max memory of worker YARN containers (in MB). If YARN is killing worker processes consider
+## increasing twill.java.reserved.memory.mb (which defaults to 200 and is set in yarn-site.xml).
+## The twill.java.reserved.memory.mb config determines the gap between the YARN memory limit set
+## below and the java -Xmx setting.  For example, if max memory is 1024 and twill reserved memory
+## is 200, the java -Xmx setting will be 1024-200 = 824 MB.
+#fluo.yarn.worker.max.memory.mb=1024
+## Number of worker virtual cores
+#fluo.yarn.worker.num.cores=1
diff --git a/distribution/conf/log4j.properties b/distribution/conf/log4j.properties
new file mode 100644
index 0000000..e140fbf
--- /dev/null
+++ b/distribution/conf/log4j.properties
@@ -0,0 +1,25 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+## Log4j 1.2 file that configures logging for Fluo processes in YARN
+
+log4j.appender.console=org.apache.log4j.ConsoleAppender
+log4j.appender.console.layout=org.apache.log4j.PatternLayout
+log4j.appender.console.layout.ConversionPattern=%d{ISO8601} [%-8c{2}] %-5p: %m%n
+
+log4j.logger.org.apache.curator=ERROR
+log4j.logger.org.apache.zookeeper=ERROR
+
+log4j.rootLogger=INFO, console
diff --git a/distribution/conf/logback.xml b/distribution/conf/logback.xml
new file mode 100644
index 0000000..55d3ccf
--- /dev/null
+++ b/distribution/conf/logback.xml
@@ -0,0 +1,35 @@
+<!--
+  Licensed to the Apache Software Foundation (ASF) under one or more contributor license
+  agreements. See the NOTICE file distributed with this work for additional information regarding
+  copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the
+  "License"); you may not use this file except in compliance with the License. You may obtain a
+  copy of the License at
+
+     http://www.apache.org/licenses/LICENSE-2.0
+
+  Unless required by applicable law or agreed to in writing, software distributed under the License
+  is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
+  or implied. See the License for the specific language governing permissions and limitations under
+  the License.
+-->
+
+<!-- Configures logging for fluo-yarn command -->
+<configuration>
+  <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender"> 
+    <encoder>
+      <pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n</pattern>
+    </encoder>
+  </appender>
+
+  <logger name="org.apache.fluo" level="debug"/>
+  <logger name="org.apache.curator" level="error"/>
+  <logger name="org.apache.hadoop" level="info"/>
+  <logger name="org.apache.hadoop.yarn.client.RMProxy" level="error"/>
+  <logger name="org.apache.twill" level="info"/>
+  <logger name="org.apache.twill.yarn.YarnTwillRunnerService" level="error"/>
+  <logger name="org.apache.zookeeper" level="error"/>
+
+  <root level="debug">
+    <appender-ref ref="STDOUT" />
+  </root>
+</configuration>
diff --git a/distribution/lib/fetch.sh b/distribution/lib/fetch.sh
new file mode 100755
index 0000000..0881187
--- /dev/null
+++ b/distribution/lib/fetch.sh
@@ -0,0 +1,83 @@
+#! /usr/bin/env bash
+
+# Licensed to the Apache Software Foundation (ASF) under one or more contributor license
+# agreements. See the NOTICE file distributed with this work for additional information regarding
+# copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance with the License. You may obtain a
+# copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software distributed under the License
+# is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
+# or implied. See the License for the specific language governing permissions and limitations under
+# the License.
+
+lib_dir=$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )
+maven_prefix=https://repo1.maven.org/maven2
+
+function download {
+  IFS=':' read -ra DEP <<< "$1" 
+  dir=$lib_dir/
+  if [ -n "$2" ]; then
+    dir=$lib_dir/$2
+    if [ ! -d $dir ]; then
+      mkdir $dir
+    fi
+  fi
+  group=${DEP[0]}
+  artifact=${DEP[1]}
+  ftype=${DEP[2]}
+  version=${DEP[3]}
+  fn=$artifact-$version.$ftype
+  path="${group//.//}/$artifact/$version/$fn"
+  download_url=$maven_prefix/$path
+
+  if [ -f $dir/$fn ]; then
+    echo "SUCCESS: Dependency exists - $dir/$fn"
+  else 
+    wget -q $download_url -P $dir
+    if [ $? == 0 ]; then
+      echo "SUCCESS: Dependency downloaded from $download_url"
+    else
+      echo "ERROR: Dependency download failed - $download_url"
+    fi
+  fi
+}
+
+echo "Fetching Fluo YARN launcher dependencies"
+download aopalliance:aopalliance:jar:1.0
+download ch.qos.logback:logback-classic:jar:1.1.3
+download ch.qos.logback:logback-core:jar:1.1.3
+download com.101tec:zkclient:jar:0.3
+download com.google.code.findbugs:jsr305:jar:2.0.1
+download com.google.code.gson:gson:jar:2.8.0
+download com.google.guava:guava:jar:13.0.1
+download com.yammer.metrics:metrics-annotation:jar:2.2.0
+download com.yammer.metrics:metrics-core:jar:2.2.0
+download net.sf.jopt-simple:jopt-simple:jar:3.2
+download org.apache.kafka:kafka_2.10:jar:0.8.0
+download org.apache.twill:twill-api:jar:0.11.0
+download org.apache.twill:twill-common:jar:0.11.0
+download org.apache.twill:twill-core:jar:0.11.0
+download org.apache.twill:twill-discovery-api:jar:0.11.0
+download org.apache.twill:twill-discovery-core:jar:0.11.0
+download org.apache.twill:twill-ext:jar:0.11.0
+download org.apache.twill:twill-yarn:jar:0.11.0
+download org.apache.twill:twill-zookeeper:jar:0.11.0
+download org.ow2.asm:asm-all:jar:5.0.2
+download org.scala-lang:scala-compiler:jar:2.10.1
+download org.scala-lang:scala-library:jar:2.10.1
+download org.scala-lang:scala-reflect:jar:2.10.1
+download org.slf4j:jcl-over-slf4j:jar:1.7.2
+download org.slf4j:log4j-over-slf4j:jar:1.7.12
+download org.slf4j:slf4j-api:jar:1.7.12
+download org.xerial.snappy:snappy-java:jar:1.0.5
+# See https://github.com/apache/fluo/issues/820
+download io.netty:netty:jar:3.9.9.Final
+
+echo -e "Done!\n"
+echo "NOTE - The dependencies downloaded have been tested with some versions of Fluo, Hadoop, Zookeeper, and Accumulo."
+echo "There is no guarantee they will work with all versions. The Fluo YARN launcher chose to defer dependency resolution to as"
+echo "late as possible in order to make it easier to resolve dependency conflicts.  If you run into a dependency"
+echo "conflict in your environment, please consider bringing it up on the Fluo dev list."
diff --git a/distribution/pom.xml b/distribution/pom.xml
new file mode 100644
index 0000000..68e11e9
--- /dev/null
+++ b/distribution/pom.xml
@@ -0,0 +1,90 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+  Licensed to the Apache Software Foundation (ASF) under one or more contributor license
+  agreements. See the NOTICE file distributed with this work for additional information regarding
+  copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the
+  "License"); you may not use this file except in compliance with the License. You may obtain a
+  copy of the License at
+
+     http://www.apache.org/licenses/LICENSE-2.0
+
+  Unless required by applicable law or agreed to in writing, software distributed under the License
+  is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
+  or implied. See the License for the specific language governing permissions and limitations under
+  the License.
+-->
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+  <modelVersion>4.0.0</modelVersion>
+  <parent>
+    <groupId>org.apache.fluo</groupId>
+    <artifactId>fluo-yarn-project</artifactId>
+    <version>1.0.0-SNAPSHOT</version>
+    <relativePath>../pom.xml</relativePath>
+  </parent>
+  <artifactId>fluo-yarn</artifactId>
+  <packaging>pom</packaging>
+  <name>Apache Fluo YARN distribution</name>
+  <description>This module produces a tarball distribution of the YARN launcher for Apache Fluo. It contains all of the
+    default configuration and scripts required to run Fluo applications in YARN.</description>
+  <!-- NOTE: These dependencies are bundled in this assembly -->
+  <dependencies>
+    <dependency>
+      <groupId>org.apache.fluo</groupId>
+      <artifactId>fluo-yarn-core</artifactId>
+      <optional>true</optional>
+    </dependency>
+  </dependencies>
+  <build>
+    <plugins>
+      <plugin>
+        <artifactId>maven-assembly-plugin</artifactId>
+        <executions>
+          <execution>
+            <id>bin-assembly</id>
+            <goals>
+              <goal>single</goal>
+            </goals>
+            <phase>package</phase>
+            <configuration>
+              <escapeString>\</escapeString>
+              <descriptors>
+                <descriptor>src/main/assembly/bin.xml</descriptor>
+              </descriptors>
+            </configuration>
+          </execution>
+        </executions>
+      </plugin>
+    </plugins>
+  </build>
+  <profiles>
+    <profile>
+      <!-- attach source release when it is created by the apache-release profile -->
+      <id>apache-release</id>
+      <build>
+        <plugins>
+          <plugin>
+            <groupId>org.codehaus.mojo</groupId>
+            <artifactId>build-helper-maven-plugin</artifactId>
+            <executions>
+              <execution>
+                <id>attach-source-release-assembly</id>
+                <goals>
+                  <goal>attach-artifact</goal>
+                </goals>
+                <configuration>
+                  <artifacts>
+                    <artifact>
+                      <file>${project.parent.build.directory}/${project.artifactId}-${project.version}-source-release.tar.gz</file>
+                      <type>tar.gz</type>
+                      <classifier>source-release</classifier>
+                    </artifact>
+                  </artifacts>
+                </configuration>
+              </execution>
+            </executions>
+          </plugin>
+        </plugins>
+      </build>
+    </profile>
+  </profiles>
+</project>
diff --git a/distribution/src/main/assembly/bin.xml b/distribution/src/main/assembly/bin.xml
new file mode 100644
index 0000000..68e27b8
--- /dev/null
+++ b/distribution/src/main/assembly/bin.xml
@@ -0,0 +1,73 @@
+<!--
+  Licensed to the Apache Software Foundation (ASF) under one or more contributor license
+  agreements. See the NOTICE file distributed with this work for additional information regarding
+  copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the
+  "License"); you may not use this file except in compliance with the License. You may obtain a
+  copy of the License at
+
+     http://www.apache.org/licenses/LICENSE-2.0
+
+  Unless required by applicable law or agreed to in writing, software distributed under the License
+  is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
+  or implied. See the License for the specific language governing permissions and limitations under
+  the License.
+-->
+<assembly>
+  <id>bin</id>
+  <formats>
+    <format>tar.gz</format>
+  </formats>
+
+  <dependencySets>
+    <dependencySet>
+      <outputDirectory>lib</outputDirectory>
+      <directoryMode>0755</directoryMode>
+      <fileMode>0644</fileMode>
+      <useProjectArtifact>false</useProjectArtifact>
+      <useTransitiveDependencies>false</useTransitiveDependencies>
+      <includes>
+        <include>${groupId}:fluo-*</include>
+      </includes>
+    </dependencySet>
+  </dependencySets>
+
+  <fileSets>
+    <fileSet>
+      <directory>../</directory>
+      <outputDirectory>.</outputDirectory>
+      <includes>
+        <include>DISCLAIMER</include>
+        <include>LICENSE</include>
+        <include>NOTICE</include>
+        <include>README.md</include>
+      </includes>
+    </fileSet>
+    <fileSet>
+      <directory>bin</directory>
+      <outputDirectory>bin</outputDirectory>
+      <fileMode>755</fileMode>
+      <excludes>
+        <exclude>bin/fluo-yarn</exclude>
+      </excludes>
+    </fileSet>
+    <fileSet>
+      <directory>conf</directory>
+      <outputDirectory>conf</outputDirectory>
+    </fileSet>
+    <fileSet>
+      <directory>lib</directory>
+      <outputDirectory>lib</outputDirectory>
+      <fileMode>755</fileMode>
+      <includes>
+        <include>fetch.sh</include>
+      </includes>
+    </fileSet>
+  </fileSets>
+  <files>
+    <file>
+      <source>bin/fluo-yarn</source>
+      <outputDirectory>bin</outputDirectory>
+      <filtered>true</filtered>
+    </file>
+  </files>
+</assembly>
diff --git a/pom.xml b/pom.xml
new file mode 100644
index 0000000..952957a
--- /dev/null
+++ b/pom.xml
@@ -0,0 +1,241 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+  Licensed to the Apache Software Foundation (ASF) under one or more contributor license
+  agreements. See the NOTICE file distributed with this work for additional information regarding
+  copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the
+  "License"); you may not use this file except in compliance with the License. You may obtain a
+  copy of the License at
+   
+     http://www.apache.org/licenses/LICENSE-2.0
+  
+  Unless required by applicable law or agreed to in writing, software distributed under the License
+  is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
+  or implied. See the License for the specific language governing permissions and limitations under
+  the License.
+-->
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+  <modelVersion>4.0.0</modelVersion>
+  <parent>
+    <groupId>org.apache.fluo</groupId>
+    <artifactId>fluo-parent</artifactId>
+    <version>1-incubating</version>
+  </parent>
+  <artifactId>fluo-yarn-project</artifactId>
+  <version>1.0.0-SNAPSHOT</version>
+  <packaging>pom</packaging>
+  <name>Apache Fluo YARN launcher</name>
+  <description>Tool that launches Apache Fluo application in Hadoop YARN</description>
+  <licenses>
+    <license>
+      <name>Apache License, Version 2.0</name>
+      <url>http://www.apache.org/licenses/LICENSE-2.0</url>
+      <distribution>repo</distribution>
+    </license>
+  </licenses>
+  <modules>
+    <module>distribution</module>
+    <module>core</module>
+  </modules>
+  <properties>
+    <accumulo.version>1.6.5</accumulo.version>
+    <findbugs.maxRank>9</findbugs.maxRank>
+    <fluo.version>1.2.0-SNAPSHOT</fluo.version>
+    <hadoop.version>2.6.3</hadoop.version>
+    <logback.version>1.1.3</logback.version>
+    <slf4j.version>1.7.12</slf4j.version>
+    <twill.version>0.11.0</twill.version>
+  </properties>
+  <dependencyManagement>
+    <dependencies>
+      <dependency>
+        <groupId>ch.qos.logback</groupId>
+        <artifactId>logback-classic</artifactId>
+        <version>${logback.version}</version>
+      </dependency>
+      <dependency>
+        <groupId>ch.qos.logback</groupId>
+        <artifactId>logback-core</artifactId>
+        <version>${logback.version}</version>
+      </dependency>
+      <dependency>
+        <groupId>com.google.code.gson</groupId>
+        <artifactId>gson</artifactId>
+        <version>2.8.0</version>
+      </dependency>
+      <dependency>
+        <!-- Guava 13.0.1 is required by Twill (due to beta method usage) -->
+        <groupId>com.google.guava</groupId>
+        <artifactId>guava</artifactId>
+        <version>13.0.1</version>
+      </dependency>
+      <dependency>
+        <!-- Required for Twill.  See https://github.com/apache/fluo/issues/820 -->
+        <groupId>io.netty</groupId>
+        <artifactId>netty</artifactId>
+        <version>3.9.9.Final</version>
+      </dependency>
+      <dependency>
+        <groupId>junit</groupId>
+        <artifactId>junit</artifactId>
+        <version>4.12</version>
+      </dependency>
+      <dependency>
+        <groupId>org.apache.fluo</groupId>
+        <artifactId>fluo-api</artifactId>
+        <version>${fluo.version}</version>
+      </dependency>
+      <dependency>
+        <groupId>org.apache.fluo</groupId>
+        <artifactId>fluo-command</artifactId>
+        <version>${fluo.version}</version>
+      </dependency>
+      <dependency>
+        <groupId>org.apache.fluo</groupId>
+        <artifactId>fluo-core</artifactId>
+        <version>${fluo.version}</version>
+      </dependency>
+      <dependency>
+        <groupId>org.apache.fluo</groupId>
+        <artifactId>fluo-yarn-core</artifactId>
+        <version>${project.version}</version>
+      </dependency>
+      <dependency>
+        <groupId>org.apache.hadoop</groupId>
+        <artifactId>hadoop-yarn-client</artifactId>
+        <version>${hadoop.version}</version>
+      </dependency>
+      <dependency>
+        <groupId>org.apache.twill</groupId>
+        <artifactId>twill-api</artifactId>
+        <version>${twill.version}</version>
+      </dependency>
+      <dependency>
+        <groupId>org.apache.twill</groupId>
+        <artifactId>twill-ext</artifactId>
+        <version>${twill.version}</version>
+      </dependency>
+      <dependency>
+        <groupId>org.apache.twill</groupId>
+        <artifactId>twill-yarn</artifactId>
+        <version>${twill.version}</version>
+      </dependency>
+      <dependency>
+        <groupId>org.slf4j</groupId>
+        <artifactId>slf4j-api</artifactId>
+        <version>${slf4j.version}</version>
+      </dependency>
+      <dependency>
+        <groupId>org.slf4j</groupId>
+        <artifactId>slf4j-log4j12</artifactId>
+        <version>${slf4j.version}</version>
+      </dependency>
+    </dependencies>
+  </dependencyManagement>
+  <build>
+    <pluginManagement>
+      <plugins>
+        <plugin>
+          <groupId>org.apache.accumulo</groupId>
+          <artifactId>accumulo-maven-plugin</artifactId>
+          <version>${accumulo.version}</version>
+          <configuration>
+            <instanceName>it-instance-maven</instanceName>
+            <rootPassword>ITSecret</rootPassword>
+          </configuration>
+          <executions>
+            <execution>
+              <id>run-plugin</id>
+              <goals>
+                <goal>start</goal>
+                <goal>stop</goal>
+              </goals>
+            </execution>
+          </executions>
+        </plugin>
+        <plugin>
+          <groupId>org.apache.rat</groupId>
+          <artifactId>apache-rat-plugin</artifactId>
+          <configuration />
+        </plugin>
+        <plugin>
+          <groupId>org.apache.maven.plugins</groupId>
+          <artifactId>maven-failsafe-plugin</artifactId>
+          <configuration>
+            <systemPropertyVariables>
+              <fluo.it.instance.name>it-instance-maven</fluo.it.instance.name>
+              <fluo.it.instance.clear>false</fluo.it.instance.clear>
+            </systemPropertyVariables>
+          </configuration>
+        </plugin>
+        <plugin>
+          <groupId>org.apache.maven.plugins</groupId>
+          <artifactId>maven-release-plugin</artifactId>
+          <configuration>
+            <!-- must put this here, due to MRELEASE-963 -->
+            <tagNameFormat>rel/fluo-@{project.version}</tagNameFormat>
+          </configuration>
+        </plugin>
+      </plugins>
+    </pluginManagement>
+    <plugins>
+      <plugin>
+        <groupId>org.apache.maven.plugins</groupId>
+        <artifactId>maven-dependency-plugin</artifactId>
+        <executions>
+          <execution>
+            <id>analyze</id>
+            <goals>
+              <goal>analyze-only</goal>
+            </goals>
+            <configuration>
+              <failOnWarning>true</failOnWarning>
+              <ignoredDependencies>
+                <ignoredDependency>log4j:log4j:jar:*</ignoredDependency>
+                <ignoredDependency>org.apache.fluo:fluo-core:jar:${fluo.version}</ignoredDependency>
+                <ignoredDependency>org.apache.hadoop:hadoop-common:jar:${hadoop.version}</ignoredDependency>
+                <ignoredDependency>org.apache.hadoop:hadoop-client:jar:${hadoop.version}</ignoredDependency>
+                <ignoredDependency>org.apache.hadoop:hadoop-mapreduce-client-core:jar:${hadoop.version}</ignoredDependency>
+                <ignoredDependency>org.apache.hadoop:hadoop-yarn-api:jar:${hadoop.version}</ignoredDependency>
+                <ignoredDependency>org.apache.hadoop:hadoop-yarn-client:jar:${hadoop.version}</ignoredDependency>
+                <ignoredDependency>org.slf4j:slf4j-log4j12:jar:${slf4j.version}</ignoredDependency>
+              </ignoredDependencies>
+            </configuration>
+          </execution>
+        </executions>
+      </plugin>
+    </plugins>
+  </build>
+  <profiles>
+    <profile>
+      <id>fluo-release</id>
+      <!-- some properties to make the release build a bit faster -->
+      <properties>
+        <checkstyle.skip>true</checkstyle.skip>
+        <findbugs.skip>true</findbugs.skip>
+        <modernizer.skip>true</modernizer.skip>
+        <skipITs>true</skipITs>
+        <skipTests>true</skipTests>
+      </properties>
+    </profile>
+    <profile>
+      <!-- set proper source assembly name with apache-release and don't attach here -->
+      <id>apache-release</id>
+      <build>
+        <pluginManagement>
+          <plugins>
+            <plugin>
+              <groupId>org.apache.maven.plugins</groupId>
+              <artifactId>maven-assembly-plugin</artifactId>
+              <inherited>false</inherited>
+              <configuration>
+                <!-- source assembly gets attached in the assemble module -->
+                <attach>false</attach>
+                <finalName>fluo-${project.version}</finalName>
+              </configuration>
+            </plugin>
+          </plugins>
+        </pluginManagement>
+      </build>
+    </profile>
+  </profiles>
+</project>

-- 
To stop receiving notification emails like this one, please contact
['"commits@fluo.apache.org" <commits@fluo.apache.org>'].

Mime
View raw message