Mailing-List: contact notifications-help@accumulo.apache.org; run by ezmlm
Precedence: bulk
Reply-To: jira@apache.org
Date: Mon, 30 Oct 2017 18:24:00 +0000 (UTC)
From: "ASF GitHub Bot (JIRA)" <jira@apache.org>
To: notifications@accumulo.apache.org
Message-ID: <JIRA.13102054.1505334368000.102641.1509387840520@Atlassian.JIRA>
In-Reply-To: <JIRA.13102054.1505334368000@Atlassian.JIRA>
References: <JIRA.13102054.1505334368000@Atlassian.JIRA> <JIRA.13102054.1505334368176@jira-lw-us.apache.org>
Subject: [jira] [Commented] (ACCUMULO-4706) Create official Accumulo Docker
 image
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Mon, 30 Oct 2017 18:24:06 -0000


    [ https://issues.apache.org/jira/browse/ACCUMULO-4706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16225484#comment-16225484 ] 

ASF GitHub Bot commented on ACCUMULO-4706:
------------------------------------------

mikewalch commented on a change in pull request #1: ACCUMULO-4706 Initial implementation of Accumulo docker image
URL: https://github.com/apache/accumulo-docker/pull/1#discussion_r147791831
 
 
 ##########
 File path: README.md
 ##########
 @@ -1 +1,138 @@
-# Accumulo Docker Image
+# Apache Accumulo Docker Image
+
+**This is currently a work in progress that depends on unreleased features of Accumulo and will not be ready
+for use until after Accumulo 2.0.0 is released.**  Sometime after Accumulo 2.0.0 is released this project
+will make its first release. Eventually, this will project will create a `apache/accumulo` image at DockerHub.
+Until then, you will need to build your own image.
+
+## Obtain the Docker image
+
+To obtain the docker image created by this project, you can either pull it from DockerHub at
+`apache/accumulo` or build it yourself. To pull the image from DockerHub, run the command below:
+
+    docker pull apache/accumulo
+
+While it is easier to pull from DockerHub, the image will default to the software versions below:
+
+| Software    | Version        |
+|-------------|----------------|
+| [Accumulo]  | 2.0.0-SNAPSHOT |
+| [Hadoop]    | 2.7.3          |
+| [Zookeeper] | 3.4.9          |
+
+If these versions do not match what is running on your cluster, you should consider building
+your own image with matching versions. However, Accumulo must be 2.0.0+. Below are instructions for
+building an image:
+
+1. Clone the Accumulo docker repo
+
+        git clone git@github.com:apache/accumulo-docker.git
+
+2. Until Accumulo 2.0.0 is released, build a Accumulo tarball distribution and copy it to the root
+   directory of the repo.
+
+        git clone git@github.com:apache/accumulo.git
+        cd accumulo/
+        mvn clean package
+        cp assemble/target/accumulo-2.0.0-SNAPSHOT-bin.tar.gz /path/to/accumulo-docker/
+
+3. Build the default Accumulo docker image using the command below.
+
+        cd /path/to/accumulo-docker
+        docker build -t accumulo .
+
+   Or build the Accumulo docker image with specific versions of Hadoop, Zookeeper, etc using the command below:
+
+        docker build --build-arg ZOOKEEPER_VERSION=3.4.8 --build-arg HADOOP_VERSION=2.7.0 -t accumulo .
+
+## Image basics
+
+The entrypoint for the Accumulo docker image is the `accumulo` script. While the primary use
+case for this image is to start Accumulo processes (i.e tserver, master, etc), you can run other
+commands in the `accumulo` script to test out the image:
+
+```bash
+# No arguments prints Accumulo command usage
+docker run accumulo
+# Print Accumulo version
+docker run accumulo version
+# Print Accumulo classpath
+docker run accumulo classpath
+```
+
+# Run Accumulo using Docker
+
+Before you can run Accumulo services in Docker, you will need to install Accumulo, configure `accumulo-site.xml`,
+and initialize your instance with `--upload-accumulo-site`. This will upload configuration to Zookeeper and limit
+how much configuration needs to be set on the command line.
+
+```bash
+$ accumulo init --upload-accumulo-site
+...
+Uploading properties in accumulo-site.xml to Zookeeper. Properties that cannot be set in Zookeeper will be skipped:
+Skipped - instance.secret = <hidden>
+Skipped - instance.volumes = hdfs://localhost:8020/accumulo
+Skipped - instance.zookeeper.host = localhost:2181
+Uploaded - table.durability = flush
+Uploaded - tserver.memory.maps.native.enabled = false
+Uploaded - tserver.readahead.concurrent.max = 64
+Uploaded - tserver.server.threads.minimum = 64
+Uploaded - tserver.walog.max.size = 512M
+```
+
+Any configuration that is skipped above will need to be passed in as a command line option to Accumulo services running
+in Docker containers. These options can be set in an environment variable which is used in later commands.
+
+```
+export ACCUMULO_CL_OPTS="-o instance.secret=mysecret -o instance.volumes=hdfs://localhost:8020/accumulo -o instance.zookeeper.host=localhost:2181"
+```
+
+The Accumulo docker image expects that the HDFS path set by `instance.volumes` is owned by the `accumulo` user. This
+can be accomplished by running the command below (replace the HDFS path with yours):
+
+```bash
+hdfs dfs -chown -R accumulo hdfs://localhost:8020/accumulo
+```
+
+## Docker engine
+
+Use the `docker` command to start local docker containers. The commands below will start a local Accumulo cluster
+with two tablet servers.
+
+```
+docker run -d --network="host" accumulo monitor $ACCUMULO_CL_OPTS
+docker run -d --network="host" accumulo tserver $ACCUMULO_CL_OPTS
+docker run -d --network="host" accumulo tserver $ACCUMULO_CL_OPTS
+docker run -d --network="host" accumulo master $ACCUMULO_CL_OPTS
+docker run -d --network="host" accumulo gc $ACCUMULO_CL_OPTS
+```
+
+## Marathon
+
+Using the Marathon UI, you can start Accumulo services using the following
+JSON configuration template.  The template is configured to start an Accumulo
+monitor but it can be modified to start other Accumulo services such as
+`master`, `tserver` and `gc`. For tablet servers, set `instances` to the number
+of tablet servers that you want to run.
+
+```
+{
+  "id": "accumulo-monitor",
+  "cmd": "accumulo monitor -o instance.secret=mysecret -o instance.volumes=hdfs://localhost:8020/accumulo -o instance.zookeeper.host=localhost:2181",
+  "cpus": 1,
+  "mem": 512,
 
 Review comment:
   I think this can be done by settting ACCUMULO_JAVA_OPTS before you run docker command.
   
   https://stackoverflow.com/questions/29923531/how-to-set-java-heap-size-xms-xmx-inside-docker-container
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


> Create official Accumulo Docker image
> -------------------------------------
>
>                 Key: ACCUMULO-4706
>                 URL: https://issues.apache.org/jira/browse/ACCUMULO-4706
>             Project: Accumulo
>          Issue Type: New Feature
>          Components: scripts, start
>    Affects Versions: 2.0.0
>            Reporter: Mike Walch
>            Assignee: Mike Walch
>              Labels: pull-request-available
>          Time Spent: 10m
>  Remaining Estimate: 0h
>
> While there are some [Accumulo images|https://hub.docker.com/search/?isAutomated=0&isOfficial=0&page=1&pullCount=0&q=accumulo&starCount=0] on DockerHub, it looks the majority of the them are designed to run a single-node Accumulo instance in a Docker container for development and testing.
> It would be great if Accumulo had an official image for running Accumulo processes in containers on a production cluster.  The image could be be published as an official image 'apache/accumulo' to DockerHub.  
> In order to make this possible, I think work needs to be done to allow configuration to be passed to the Accumulo process in the docker container without using configuration files (as passing files to a running container is hard in Docker).  One way to do this is to add an option called {{--upload-accumulo-site}} to 'accumulo init' command which is called outside of Docker by the user.  This would set properties in accumulo-site.xml as system properties in Zookeeper during Accumulo initialization.  Accumulo processes in Docker containers could be started with minimal configuration by updating 'accumulo <service>' commands to have a {{-o key=value}} option to override configuration.  These changes to Accumulo would enable the following commands to start an Accumulo cluster in Docker:
> {noformat}
> accumulo init --upload-accumulo-site
> docker pull apache/accumulo
> docker run apache/accumulo master -o instance.zookeeper.host=zkhost:2181
> docker run apache/accumulo tserver -o instance.zookeeper.host=zkhost:2181
> docker run apache/accumulo monitor -o instance.zookeeper.host=zkhost:2181
> docker run apache/accumulo tracer -o instance.zookeeper.host=zkhost:2181
> {noformat}


--
This message was sent by Atlassian JIRA
(v6.4.14#64029)