fluo-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From mwa...@apache.org
Subject [fluo-docker] branch master updated: Initial implementation of Fluo Docker image (#1)
Date Mon, 11 Sep 2017 15:05:09 GMT
This is an automated email from the ASF dual-hosted git repository.

mwalch pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/fluo-docker.git


The following commit(s) were added to refs/heads/master by this push:
     new fdeecf8  Initial implementation of Fluo Docker image (#1)
fdeecf8 is described below

commit fdeecf81f5512bffff09f4c48e62ea138b01fe7c
Author: Mike Walch <mwalch@apache.org>
AuthorDate: Mon Sep 11 11:05:07 2017 -0400

    Initial implementation of Fluo Docker image (#1)
    
    * Depends on local fluo-1.2.0-SNAPSHOT tarball
---
 .gitignore |   1 +
 Dockerfile |  48 +++++++++++++++++++++++
 README.md  | 127 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-
 3 files changed, 175 insertions(+), 1 deletion(-)

diff --git a/.gitignore b/.gitignore
new file mode 100644
index 0000000..335ec95
--- /dev/null
+++ b/.gitignore
@@ -0,0 +1 @@
+*.tar.gz
diff --git a/Dockerfile b/Dockerfile
new file mode 100644
index 0000000..ab65e0e
--- /dev/null
+++ b/Dockerfile
@@ -0,0 +1,48 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+FROM openjdk:8
+
+ARG HADOOP_VERSION
+ARG ZOOKEEPER_VERSION
+ARG ACCUMULO_VERSION
+
+ENV HADOOP_VERSION ${HADOOP_VERSION:-2.7.3}
+ENV ZOOKEEPER_VERSION ${ZOOKEEPER_VERSION:-3.4.9}
+ENV ACCUMULO_VERSION ${ACCUMULO_VERSION:-1.8.1}
+ENV FLUO_VERSION 1.2.0-SNAPSHOT
+
+RUN curl -sL http://archive.apache.org/dist/hadoop/core/hadoop-$HADOOP_VERSION/hadoop-$HADOOP_VERSION.tar.gz
| tar -xzC /tmp
+RUN curl -sL http://archive.apache.org/dist/zookeeper/zookeeper-$ZOOKEEPER_VERSION/zookeeper-$ZOOKEEPER_VERSION.tar.gz
| tar -xzC /tmp
+RUN curl -sL http://archive.apache.org/dist/accumulo/$ACCUMULO_VERSION/accumulo-$ACCUMULO_VERSION-bin.tar.gz
| tar -xzC /tmp
+#RUN curl -sL http://archive.apache.org/dist/fluo/fluo/$FLUO_VERSION/fluo-$FLUO_VERSION-bin.tar.gz
| tar -xzC /tmp
+# Comment out line above and remove line below in 1.2.0 is released
+ADD ./fluo-$FLUO_VERSION-bin.tar.gz /tmp/
+
+RUN mv /tmp/hadoop-$HADOOP_VERSION /opt/hadoop
+RUN mv /tmp/zookeeper-$ZOOKEEPER_VERSION /opt/zookeeper
+RUN mv /tmp/accumulo-$ACCUMULO_VERSION /opt/accumulo
+RUN mv /tmp/fluo-$FLUO_VERSION /opt/fluo
+
+ENV HADOOP_PREFIX /opt/hadoop
+ENV ZOOKEEPER_HOME /opt/zookeeper
+ENV ACCUMULO_HOME /opt/accumulo
+ENV FLUO_HOME /opt/fluo
+ENV PATH "$PATH:$FLUO_HOME/bin"
+
+RUN /opt/fluo/lib/fetch.sh extra
+
+ENTRYPOINT ["fluo"]
+CMD ["help"]
diff --git a/README.md b/README.md
index 9b3e163..d4c4319 100644
--- a/README.md
+++ b/README.md
@@ -1 +1,126 @@
-# Apache Fluo Docker Image
+# Fluo Docker Image
+
+## Obtain the Docker image
+
+To obtain the docker image created by this project, you can either pull it from DockerHub
at
+`mikewalch/fluo` or build it yourself. To pull the image from DockerHub, run the command
below:
+
+    sudo docker pull mikewalch/fluo
+
+While it is easier to pull from DockerHub, the image will default to the software versions
below:
+
+| Software    | Version        |
+|-------------|----------------|
+| [Fluo]      | 1.2.0-SNAPSHOT |
+| [Accumulo]  | 1.8.1          |
+| [Hadoop]    | 2.7.3          |
+| [Zookeeper] | 3.4.9          |
+
+If these versions do not match what is running on your cluster, you should consider building
+your own image with matching versions. However, Fluo must be 1.2+. Below are instructions
for
+building an image:
+
+1. Clone the Fluo docker repo
+
+        git clone git@github.com:mikewalch/fluo-docker.git
+
+2. Until Fluo 1.2 is released, build a Fluo tarball distribution and copy it to the root
+   directory of the repo.
+
+        git clone git@github.com:apache/fluo.git
+        cd fluo/
+        mvn clean package
+        cp modules/distribution/target/fluo-1.2.0-SNAPSHOT-bin.tar.gz /path/to/fluo-docker/
+
+3. Build the default Fluo docker image using the command below.
+
+        cd /path/to/fluo-docker
+        docker build -t fluo .
+
+   Or build the Fluo docker image with specific versions of Hadoop, Zookeeper, etc using
the command below:
+
+        docker build --build-arg ZOOKEEPER_VERSION=3.4.8 --build-arg ACCUMULO_VERSION=1.7.3
--build-arg HADOOP_VERSION=2.7.0 -t fluo .
+
+## Image basics
+
+The entrypoint for the Fluo docker image is the `fluo` script. While the primary use
+case for this image is to start an oracle or worker, you can run other commands in the
+`fluo` script to test out the image:
+
+```bash
+# No arguments prints Fluo command usage
+docker run mikewalch/fluo
+# Print Fluo version
+docker run mikewalch/fluo version
+# Print Fluo classpath
+docker run mikewalch/fluo classpath
+```
+
+# Run Fluo Applications using Docker
+
+Before starting a Fluo oracle and worker using the Fluo Docker image, [initialize your Fluo
application][application]. 
+Next, choose a method below to run the oracle and worker(s) of your Fluo application. In
the examples below, the Fluo
+application is named `myapp` and was initialized using a Zookeeper node on `zkhost`.
+
+## Docker engine
+
+Use the `docker` command to start local docker containers.
+
+1. Start a Fluo oracle
+
+        docker run -d --network="host" mikewalch/fluo oracle -a myapp -o fluo.connection.zookeepers=zkhost/fluo
+
+2. Start Fluo worker(s). Execute this command multiple times to start multiple workers.
+
+        docker run -d --network="host" mikewalch/fluo worker -a myapp -o fluo.connection.zookeepers=zkhost/fluo
+
+## Marathon
+
+Using the [Marathon] UI, you can create applications using JSON configuration.
+
+The JSON below can be used to start a Fluo oracle.
+
+```json
+{
+  "id": "myapp-fluo-oracle",
+  "cmd": "fluo oracle -a myapp -o fluo.connection.zookeepers=zkhost/fluo",
+  "cpus": 1,
+  "mem": 256,
+  "disk": 0,
+  "instances": 1,
+  "container": {
+    "docker": {
+      "image": "mikewalch/fluo",
+      "network": "HOST"
+    },
+    "type": "DOCKER"
+  }
+}
+```
+
+The JSON below can be used to start Fluo worker(s). Modify instances to start multiple workers.
+
+```json
+{
+  "id": "myapp-fluo-worker",
+  "cmd": "fluo worker -a myapp -o fluo.connection.zookeepers=zkhost/fluo",
+  "cpus": 1,
+  "mem": 512,
+  "disk": 0,
+  "instances": 1,
+  "container": {
+    "docker": {
+      "image": "mikewalch/fluo",
+      "network": "HOST"
+    },
+    "type": "DOCKER"
+  }
+}
+```
+
+[Fluo]: https://fluo.apache.org/
+[Accumulo]: https://accumulo.apache.org/
+[Hadoop]: https://hadoop.apache.org/
+[Zookeeper]: https://zookeeper.apache.org/
+[application]: https://github.com/apache/fluo/blob/master/docs/applications.md
+[Marathon]: https://mesosphere.github.io/marathon/

-- 
To stop receiving notification emails like this one, please contact
['"commits@fluo.apache.org" <commits@fluo.apache.org>'].

Mime
View raw message