bookkeeper-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From si...@apache.org
Subject [bookkeeper] 01/04: ISSUE #338: add first draft Docker image including community suggestions
Date Sat, 12 Aug 2017 01:42:35 GMT
This is an automated email from the ASF dual-hosted git repository.

sijie pushed a commit to tag release-4.5.0
in repository https://gitbox.apache.org/repos/asf/bookkeeper.git

commit ddc7104430698309c7ca82801242bf2868f0f18b
Author: zhaijack <zhaijia03@gmail.com>
AuthorDate: Sun Aug 6 21:08:11 2017 +0800

    ISSUE #338: add first draft Docker image including community suggestions
    
    This is the first part of #335. And it is based on #197
    Main changes:
     327: Docker image: Drop versions and Alpine support.
     260: Docker image: provide a way to pass any desired configuration property via ENV vars.
    
    ---
    Be sure to do all of the following to help us incorporate your contribution
    quickly and easily:
    
    - [X] Make sure the PR title is formatted like:
        `<Issue # or BOOKKEEPER-#>: Description of pull request`
        `e.g. Issue 123: Description ...`
        `e.g. BOOKKEEPER-1234: Description ...`
    - [ ] Make sure tests pass via `mvn clean apache-rat:check install findbugs:check`.
    - [X] Replace `<Issue # or BOOKKEEPER-#>` in the title with the actual Issue/JIRA
number.
    
    ---
    
    Author: zhaijack <zhaijia03@gmail.com>
    
    Reviewers: Matteo Merli <None>, Sijie Guo <None>
    
    This closes #342 from zhaijack/issue_338, closes #338
---
 docker/Dockerfile                       |  58 ++++++++++
 docker/Makefile                         | 194 ++++++++++++++++++++++++++++++++
 docker/README.md                        | 174 ++++++++++++++++++++++++++++
 docker/scripts/apply-config-from-env.py |  85 ++++++++++++++
 docker/scripts/entrypoint.sh            |  72 ++++++++++++
 docker/scripts/healthcheck.sh           |  28 +++++
 6 files changed, 611 insertions(+)

diff --git a/docker/Dockerfile b/docker/Dockerfile
new file mode 100644
index 0000000..45d422e
--- /dev/null
+++ b/docker/Dockerfile
@@ -0,0 +1,58 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+FROM centos:7
+MAINTAINER Apache BookKeeper <dev@bookkeeper.apache.org>
+
+ARG BK_VERSION=4.4.0
+ARG DISTRO_NAME=bookkeeper-server-${BK_VERSION}-bin
+ARG GPG_KEY=B3D56514
+
+ENV BOOKIE_PORT=3181
+EXPOSE $BOOKIE_PORT
+ENV BK_USER=bookkeeper
+
+# Download Apache Bookkeeper, untar and clean up
+RUN set -x \
+    && adduser "${BK_USER}" \
+    && yum install -y java-1.8.0-openjdk-headless wget bash python md5sum sha1sum
\
+    && mkdir -pv /opt \
+    && cd /opt \
+    && wget -q "https://archive.apache.org/dist/bookkeeper/bookkeeper-${BK_VERSION}/${DISTRO_NAME}.tar.gz"
\
+    && wget -q "https://archive.apache.org/dist/bookkeeper/bookkeeper-${BK_VERSION}/${DISTRO_NAME}.tar.gz.asc"
\
+    && wget -q "https://archive.apache.org/dist/bookkeeper/bookkeeper-${BK_VERSION}/${DISTRO_NAME}.tar.gz.md5"
\
+    && wget -q "https://archive.apache.org/dist/bookkeeper/bookkeeper-${BK_VERSION}/${DISTRO_NAME}.tar.gz.sha1"
\
+    && md5sum -c ${DISTRO_NAME}.tar.gz.md5 \
+    && sha1sum -c ${DISTRO_NAME}.tar.gz.sha1 \
+    && gpg --keyserver ha.pool.sks-keyservers.net --recv-key "$GPG_KEY" \
+    && gpg --batch --verify "$DISTRO_NAME.tar.gz.asc" "$DISTRO_NAME.tar.gz" \
+    && tar -xzf "$DISTRO_NAME.tar.gz" \
+    && mv bookkeeper-server-${BK_VERSION}/ /opt/bookkeeper/ \
+    && rm -rf "$DISTRO_NAME.tar.gz" "$DISTRO_NAME.tar.gz.asc" "$DISTRO_NAME.tar.gz.md5"
"$DISTRO_NAME.tar.gz.sha1" \
+    && yum remove -y wget \
+    && yum clean all
+
+WORKDIR /opt/bookkeeper
+
+COPY scripts/apply-config-from-env.py scripts/entrypoint.sh scripts/healthcheck.sh /opt/bookkeeper/
+
+ENTRYPOINT [ "/bin/bash", "/opt/bookkeeper/entrypoint.sh" ]
+CMD ["/opt/bookkeeper/bin/bookkeeper", "bookie"]
+
+HEALTHCHECK --interval=10s --timeout=60s CMD /bin/bash /opt/bookkeeper/healthcheck.sh
diff --git a/docker/Makefile b/docker/Makefile
new file mode 100644
index 0000000..27c8ce6
--- /dev/null
+++ b/docker/Makefile
@@ -0,0 +1,194 @@
+#!/bin/bash
+#
+#/**
+# * Copyright 2007 The Apache Software Foundation
+# *
+# * Licensed to the Apache Software Foundation (ASF) under one
+# * or more contributor license agreements.  See the NOTICE file
+# * distributed with this work for additional information
+# * regarding copyright ownership.  The ASF licenses this file
+# * to you under the Apache License, Version 2.0 (the
+# * "License"); you may not use this file except in compliance
+# * with the License.  You may obtain a copy of the License at
+# *
+# *     http://www.apache.org/licenses/LICENSE-2.0
+# *
+# * Unless required by applicable law or agreed to in writing, software
+# * distributed under the License is distributed on an "AS IS" BASIS,
+# * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# * See the License for the specific language governing permissions and
+# * limitations under the License.
+# */
+
+
+VERSION ?= centos
+IMAGE ?= bookkeeper/bookie:$(VERSION)
+BOOKIE ?= 1
+DOCKER_NETWORK ?= bk_network
+
+BUILD_DIR ?= $(VERSION)
+
+CONTAINER_NAME = bookkeeper-$(BOOKIE)
+DOCKER_HOSTNAME = $(shell hostname)
+BK_LOCAL_DATA_DIR = /tmp/test_bk
+BK_LOCAL_CONTAINER_DATA_DIR = $(BK_LOCAL_DATA_DIR)/$(CONTAINER_NAME)
+BK_DIR = /data
+BK_zkLedgersRootPath = /ledgers
+
+ZK_CONTAINER_NAME=test_zookeeper
+ZK_LOCAL_DATA_DIR=$(BK_LOCAL_DATA_DIR)/zookkeeper
+
+
+CONTAINER_IP=$(shell docker inspect --format '{{ .NetworkSettings.IPAddress }}' $(CONTAINER_NAME))
+
+# -------------------------------- #
+
+.PHONY: all build run create start stop shell exec root-shell root-exec info ip clean-files
clean
+
+# -------------------------------- #
+
+all:
+	make info
+
+# -------------------------------- #
+
+# Build the bookkeeper image.
+#   make build
+build:
+	cd $(BUILD_DIR) ; \
+	time docker build \
+	    -t $(IMAGE) .
+
+# -------------------------------- #
+
+# Create and run a bookkeeper container with data persisted on local filesystem. It needs
the zookkeeper container.
+# In order to launch several bookies, the command need the bookie number
+#   make run-bk BOOKIE=4
+
+run-bk:
+	mkdir -p $(BK_LOCAL_DATA_DIR) \
+			$(BK_LOCAL_CONTAINER_DATA_DIR) \
+			$(BK_LOCAL_CONTAINER_DATA_DIR)/journal \
+			$(BK_LOCAL_CONTAINER_DATA_DIR)/ledger \
+			$(BK_LOCAL_CONTAINER_DATA_DIR)/index
+	
+	-docker rm -f $(CONTAINER_NAME)
+	docker run -it\
+		--network $(DOCKER_NETWORK) \
+	    --volume $(BK_LOCAL_CONTAINER_DATA_DIR)/journal:$(BK_DIR)/journal \
+	    --volume $(BK_LOCAL_CONTAINER_DATA_DIR)/ledger:$(BK_DIR)/ledger \
+	    --volume $(BK_LOCAL_CONTAINER_DATA_DIR)/index:$(BK_DIR)/index \
+	    --name "$(CONTAINER_NAME)" \
+	    --hostname "$(CONTAINER_NAME)" \
+	    --env BK_zkServers=$(ZK_CONTAINER_NAME):2181 \
+	    --env BK_zkLedgersRootPath=$(BK_zkLedgersRootPath) \
+	    $(IMAGE)
+
+# -------------------------------- #
+
+# Create run and destroy a container that will format zookkeeper metadata
+#   make run-format
+
+run-format:
+	docker run -it --rm \
+		--network $(DOCKER_NETWORK) \
+		--env BK_zkServers=$(ZK_CONTAINER_NAME):2181 \
+		$(IMAGE) \
+		bookkeeper shell metaformat $(FORMAT_OPTS)
+
+# -------------------------------- #
+
+# Create and run the zookkeeper container needed by the ensemble
+#   make run-zk
+
+run-zk:
+	-docker network create $(DOCKER_NETWORK)
+	mkdir -pv $(BK_LOCAL_DATA_DIR) $(ZK_LOCAL_DATA_DIR) $(ZK_LOCAL_DATA_DIR)/data $(ZK_LOCAL_DATA_DIR)/datalog
+	-docker rm -f $(ZK_CONTAINER_NAME)
+	docker run -it --rm \
+		--network $(DOCKER_NETWORK) \
+		--name "$(ZK_CONTAINER_NAME)" \
+		--hostname "$(ZK_CONTAINER_NAME)" \
+		-v $(ZK_LOCAL_DATA_DIR)/data:/data \
+		-v $(ZK_LOCAL_DATA_DIR)/datalog:/datalog \
+		-p 2181:2181 \
+		zookeeper
+
+# -------------------------------- #
+
+# Create and run a container running the bookkeeper tutorial application (a simple dice rolling
application).
+# It's possible to run several dice applications in order to simulate a real life concurrent
scenario.
+#   make run-dice
+run-dice:
+	docker run -it --rm \
+		--network $(DOCKER_NETWORK) \
+		--env ZOOKEEPER_SERVERS=$(ZK_CONTAINER_NAME):2181 \
+		caiok/bookkeeper-tutorial
+
+# -------------------------------- #
+
+# This is an example of a full bookkeeper ensemble of 3 bookies, a zookkeeper server and
2 client dice applications.
+# On MacOS please run these command manually in several terminals
+#   make run-demo
+run-demo:
+	$(eval WAIT_CMD := read -p 'Press Enter to close...')
+	$(TERMINAL_EMULATOR) -e "bash -l -c \"make run-zk ; $(WAIT_CMD)"\"
+	sleep 3
+	$(TERMINAL_EMULATOR) -e "bash -l -c \"make run-bk BOOKIE=1 TRY_METAFORMAT=true; $(WAIT_CMD)\""
+	$(TERMINAL_EMULATOR) -e "bash -l -c \"make run-bk BOOKIE=2 TRY_METAFORMAT=true; $(WAIT_CMD)\""
+	$(TERMINAL_EMULATOR) -e "bash -l -c \"make run-bk BOOKIE=3 TRY_METAFORMAT=true; $(WAIT_CMD)\""
+	sleep 6
+	$(TERMINAL_EMULATOR) -e "bash -l -c \"make run-dice ; $(WAIT_CMD)\""
+	sleep 2
+	$(TERMINAL_EMULATOR) -e "bash -l -c \"make run-dice ; $(WAIT_CMD)\""
+
+	@echo
+	@echo "If you want to restart from scratch the application, remove all its data:"
+	@echo "  sudo rm -rf $(BK_LOCAL_DATA_DIR)"
+	@echo
+
+# -------------------------------- #
+# Other undocumented utilities     #
+# -------------------------------- #
+
+start:
+	docker start "$(CONTAINER_NAME)"
+
+# -------------------------------- #
+
+stop:
+	docker stop "$(CONTAINER_NAME)"
+
+# -------------------------------- #
+
+shell exec:
+	docker exec -it \
+	    "$(CONTAINER_NAME)" \
+	    /bin/bash -il
+
+# -------------------------------- #
+
+root-shell root-exec:
+	docker exec -it "$(CONTAINER_NAME)" /bin/bash -il
+
+# -------------------------------- #
+
+info ip:
+	@echo 
+	@echo "Image: $(IMAGE)"
+	@echo "Container name: $(CONTAINER_NAME)"
+	@echo
+	-@echo "Actual Image: $(shell docker inspect --format '{{ .RepoTags }} (created {{.Created
}})' $(IMAGE))"
+	-@echo "Actual Container: $(shell docker inspect --format '{{ .Name }} (created {{.Created
}})' $(CONTAINER_NAME))"
+	-@echo "Actual Container IP: $(shell docker inspect --format '{{ .NetworkSettings.IPAddress
}}' $(CONTAINER_NAME))"
+	@echo
+
+# -------------------------------- #
+
+clean-files:
+
+clean:
+	-docker stop $(CONTAINER_NAME)
+	-docker rm $(CONTAINER_NAME)
+	-docker rmi $(IMAGE)
+	make clean-files
diff --git a/docker/README.md b/docker/README.md
new file mode 100644
index 0000000..7b7cb36
--- /dev/null
+++ b/docker/README.md
@@ -0,0 +1,174 @@
+
+# What is Apache Bookkeeper?
+
+Apache ZooKeeper is a software project of the Apache Software Foundation, providing a replicated
log service which can be used to build replicated state machines. A log contains a sequence
of events which can be applied to a state machine. BookKeeper guarantees that each replica
state machine will see all the same entries, in the same order.
+
+> [Apache Bookkeeper](http://bookkeeper.apache.org/)
+
+
+# How to use this image
+
+Bookkeeper needs [Zookeeper](https://zookeeper.apache.org/) in order to preserve its state
and publish its bookies (bookkepeer servers). The client only need to connect to a Zookkeeper
server in the ensamble in order to obtain the list of Bookkeeper servers.
+
+## TL;DR
+
+If you just want to see things working, you can play with Makefile hosted in this project
and check its targets for a fairly complex set up example:
+```
+git clone https://github.com/apache/bookkeeper
+cd bookkeeper/docker
+make run-demo
+```
+While, if you don't have access to a X environment, e.g. on default MacOS, It has to run
the last command manually in 6 terminals respectively.
+```
+make run-zk
+make run-bk BOOKIE=1
+make run-bk BOOKIE=2
+make run-bk BOOKIE=3
+make run-dice
+make run-dice
+```
+This will do all the following steps and start up a working ensemble with two dice applications.
+
+## Step by step
+
+The simplest way to let Bookkeeper servers publish themselves with a name, which could be
resolved consistently across container runs, is through creation of a [docker network](https://docs.docker.com/engine/reference/commandline/network_create/):
+```
+docker network create "my-bookkeeper-network"
+```
+Then we can start a Zookeeper (from [Zookeeper official image](https://hub.docker.com/_/zookeeper/))
server in standalone mode on that network:
+```
+docker run -d \
+    --network "my-bookkeeper-network" \
+    --name "my-zookeeper" \
+    --hostname "my-zookeeper" \
+    zookeeper
+```
+And initialize the metadata store that bookies will use to store information:
+```
+docker run -it --rm \
+    --network "my-bookkeeper-network" \
+    --env ZK_URL=my-zookeeper:2181 \
+    bookkeeper \
+    bookkeeper shell metaformat
+```
+Now we can start our Bookkeeper ensemble (e.g. with three bookies):
+```
+docker run -it\
+    --network "my-bookkeeper-network" \
+    --env ZK_URL=my-zookeeper:2181 \
+    --name "bookie1" \
+    --hostname "bookie1" \
+    bookkeeper
+```
+And so on for "bookie2" and "bookie3". We have now our fully functional ensemble, ready to
accept clients.
+
+In order to play with our freshly created ensemble, you can use the simple application taken
from [Bookkeeper Tutorial](http://bookkeeper.apache.org/docs/master/bookkeeperTutorial.html)
and packaged in a [docker image](https://github.com/caiok/bookkeeper-tutorial) for convenience.
+
+This application check if it can be leader, if yes start to roll a dice and book this rolls
on bookkeeper, otherwise it will start to follow the leader rolls. If leader stops, follower
will try to become leader and so on.
+
+Start a dice application (you can run it several times to view the behavior in a concurrent
environment):
+```
+docker run -it --rm \
+    --network "my-bookkeeper-network" \
+    --env ZK_URL=my-zookkeeper:2181 \
+    caiok/bookkeeper-tutorial
+```
+
+## Configuration
+
+Bookkeeper configuration is located in `/opt/bookkeeper/conf` in the docker container, it
is a copy of [these files](https://github.com/apache/bookkeeper/tree/master/bookkeeper-server/conf)
in bookkeeper repo.
+
+There are 2 ways to set bookkeeper configuration:
+
+1, Apply setted (e.g. docker -e kk=vv) environment variables into configuration files. Environment
variable names is in format "BK_originalName", in which "originalName" is the key in config
files.
+
+2, If you are able to handle your local volumes, use `docker --volume` command to bind-mount
your local configure volumes to `/opt/bookkeeper/conf`.
+
+Example showing how to use your own configuration files:
+```
+$ docker run --name bookie1 -d \
+    -v $(local_configure_dir):/opt/bookkeeper/conf/ \   < == use 2nd approach, mount dir
contains config_files
+    -e BK_bookiePort=3181 \                             < == use 1st approach, set bookiePort
+    -e BK_zkServers=zk-server1:2181,zk-server2:2181 \   < == use 1st approach, set zookeeper
servers
+    -e BK_journalPreAllocSizeMB=32 \                    < == use 1st approach, set journalPreAllocSizeMB
in [bk_server.conf](https://github.com/apache/bookkeeper/blob/master/bookkeeper-server/conf/bk_server.conf)
+    bookkeeper
+```
+
+### Override rules for bookkeeper configuration
+If you have applied several ways to set the same config target, e.g. the environment variable
names contained in [these files](https://github.com/apache/bookkeeper/tree/master/bookkeeper-server/conf)
and conf_file in /opt/bookkeeper/conf/.
+
+Then the override rules is as this:
+
+Environment variable names contained in [these files](https://github.com/apache/bookkeeper/tree/master/bookkeeper-server/conf),
e.g. `zkServers`
+
+    Override
+
+Values in /opt/bookkeeper/conf/conf_files.
+
+Take above example, if in docker instance you have bind-mount your config file as /opt/bookkeeper/conf/bk_server.conf,
and in it contains key-value pair: `zkServers=zk-server3:2181`, then the value that take effect
finally is `zkServers=zk-server1:2181,zk-server2:2181`
+
+Because
+
+`-e BK_zkServers=zk-server1:2181,zk-server2:2181` will override key-value pair: `zkServers=zk-server3:2181`,
which contained in /opt/bookkeeper/conf/bk_server.conf.
+
+
+### Environment variable names that mostly used for your configuration.
+
+#### `BK_bookiePort`
+
+This variable allows you to specify the port on which Bookkeeper should listen for incoming
connections.
+
+This will override `bookiePort` in [bk_server.conf](https://github.com/apache/bookkeeper/blob/master/bookkeeper-server/conf/bk_server.conf).
+
+Default value is "3181".
+
+#### `BK_zkServers`
+
+This variable allows you to specify a list of machines of the Zookeeper ensemble. Each entry
has the form of `host:port`. Entries are separated with a comma.
+
+This will override `zkServers` in [bk_server.conf](https://github.com/apache/bookkeeper/blob/master/bookkeeper-server/conf/bk_server.conf).
+
+Default value is "127.0.0.1:2181"
+
+#### `BK_zkLedgersRootPath`
+
+This variable allows you to specify the root directory bookkeeper will use on Zookeeper to
store ledgers metadata.
+
+This will override `zkLedgersRootPath ` in [bk_server.conf](https://github.com/apache/bookkeeper/blob/master/bookkeeper-server/conf/bk_server.conf).
+
+Default value is "/bookkeeper/ledgers"
+
+#### `BK_CLUSTER_ROOT_PATH`
+
+This variable allows you to specify the root directory bookkeeper will use on Zookeeper.
+
+Default value is empty - " ". so ledgers dir in zookeeper will be at "/ledgers" by default.
You could set it as that you want, e.g. "/bookkeeper"
+
+#### `BK_DATA_DIR`
+This variable allows you to specify where to store data in docker instance.
+
+This could be override by env vars "BK_journalDirectory", "BK_ledgerDirectories", "BK_indexDirectories"
 and also `journalDirectory`, `ledgerDirectories`, `indexDirectories` in [bk_server.conf](https://github.com/apache/bookkeeper/blob/master/bookkeeper-server/conf/bk_server.conf).
+
+Default value is "/data/bookkeeper", which contains volumes `/data/bookkeeper/journal`, `/data/bookkeeper/ledger`
and `/data/bookkeeper/index` to hold Bookkeeper data in docker.
+
+
+### Configure files under /opt/bookkeeper/conf
+These files is originally un-tared from the bookkeeper building binary, such as [bookkeeper-server-4.4.0-bin.tar.tgz](https://archive.apache.org/dist/bookkeeper/bookkeeper-4.4.0/bookkeeper-4.4.0-src.tar.gz),
and it comes from [these files](https://github.com/apache/bookkeeper/tree/master/bookkeeper-server/conf)
in bookkeeper repo.
+
+Usually we could config files bk_server.conf, bkenv.sh, log4j.properties, and log4j.shell.properties.
Please read and understand them before you do the configuration.
+
+
+### Caveats
+
+Be careful where you put the transaction log (journal). A dedicated transaction log device
is key to consistent good performance. Putting the log on a busy device will adversely effect
performance.
+
+Here is some useful and graceful command the could be used to replace the default command,
once you want to delete the cookeis and do auto recovery:
+```
+/bookkeeper/bookkeeper-server/bin/bookkeeper shell bookieformat -nonInteractive -force -deleteCookie
+/bookkeeper/bookkeeper-server/bin/bookkeeper autorecovery
+```
+Use them, and replace the default [CMD] when you wanted to do things other than start a bookie.
+
+# License
+
+View [license information](https://github.com/apache/bookkeeper/blob/master/LICENSE) for
the software contained in this image.
diff --git a/docker/scripts/apply-config-from-env.py b/docker/scripts/apply-config-from-env.py
new file mode 100755
index 0000000..78e6945
--- /dev/null
+++ b/docker/scripts/apply-config-from-env.py
@@ -0,0 +1,85 @@
+#!/usr/bin/env python
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+##
+## Edit properties config files under config_dir and replace values
+## based on the ENV variables
+## export my-key=new-value
+##
+## ./apply-config-from-env config_dir
+##
+
+import os, sys
+
+if len(sys.argv) != 2:
+    print 'Usage: %s ' + 'config_dir' % (sys.argv[0])
+    sys.exit(1)
+
+def mylistdir(dir):
+    return [os.path.join(dir, filename) for filename in os.listdir(dir)]
+
+# Always apply env config to all the files under conf
+conf_dir = sys.argv[1]
+conf_files = mylistdir(conf_dir)
+print 'conf files: '
+print conf_files
+
+bk_env_prefix = 'BK_'
+
+for conf_filename in conf_files:
+    lines = []  # List of config file lines
+    keys = {}   # Map a key to its line number in the file
+
+    # Load conf file
+    for line in open(conf_filename):
+        lines.append(line)
+        line = line.strip()
+        #if not line or line.startswith('#'):
+        if not line or '=' not in line:
+            continue
+
+        if line.startswith('#'):
+            line = line.replace('#', '')
+
+        # Remove spaces around key,
+        line = line.replace(' ', '')
+        k,v = line.split('=', 1)
+
+        # Only replace first appearance
+        if k not in keys:
+            keys[k] = len(lines) - 1
+        else:
+           lines.pop()
+
+    # Update values from Env
+    for k in sorted(os.environ.keys()):
+        v = os.environ[k]
+        if k.startswith(bk_env_prefix):
+            search_key = k[len(bk_env_prefix):]
+            if search_key in keys:
+                print '[%s] Applying config %s = %s' % (conf_filename, search_key, v)
+                idx = keys[search_key]
+                lines[idx] = '%s=%s\n' % (search_key, v)
+
+    # Store back the updated config in the same file
+    f = open(conf_filename, 'w')
+    for line in lines:
+        f.write(line)
+    f.close()
diff --git a/docker/scripts/entrypoint.sh b/docker/scripts/entrypoint.sh
new file mode 100755
index 0000000..7610361
--- /dev/null
+++ b/docker/scripts/entrypoint.sh
@@ -0,0 +1,72 @@
+#!/bin/bash
+#
+#/**
+# * Copyright 2007 The Apache Software Foundation
+# *
+# * Licensed to the Apache Software Foundation (ASF) under one
+# * or more contributor license agreements.  See the NOTICE file
+# * distributed with this work for additional information
+# * regarding copyright ownership.  The ASF licenses this file
+# * to you under the Apache License, Version 2.0 (the
+# * "License"); you may not use this file except in compliance
+# * with the License.  You may obtain a copy of the License at
+# *
+# *     http://www.apache.org/licenses/LICENSE-2.0
+# *
+# * Unless required by applicable law or agreed to in writing, software
+# * distributed under the License is distributed on an "AS IS" BASIS,
+# * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# * See the License for the specific language governing permissions and
+# * limitations under the License.
+# */
+
+export PATH=$PATH:/opt/bookkeeper/bin
+export JAVA_HOME=/usr
+
+# env var used often
+PORT0=${PORT0:-${BOOKIE_PORT}}
+PORT0=${PORT0:-3181}
+BK_DATA_DIR=${BK_DATA_DIR:-"/data/bookkeeper"}
+BK_CLUSTER_ROOT_PATH=${BK_CLUSTER_ROOT_PATH:-" "}
+
+# env vars to replace values in config files
+export BK_bookiePort=${BK_bookiePort:-${PORT0}}
+export BK_zkServers=${BK_zkServers}
+export BK_zkLedgersRootPath=${BK_zkLedgersRootPath:-"${BK_CLUSTER_ROOT_PATH}/ledgers"}
+export BK_journalDirectory=${BK_journalDirectory:-${BK_DATA_DIR}/journal}
+export BK_ledgerDirectories=${BK_ledgerDirectories:-${BK_DATA_DIR}/ledgers}
+export BK_indexDirectories=${BK_indexDirectories:-${BK_DATA_DIR}/index}
+
+echo "BK_bookiePort bookie service port is $BK_bookiePort"
+echo "BK_zkServers is $BK_zkServers"
+echo "BK_DATA_DIR is $BK_DATA_DIR"
+echo "BK_CLUSTER_ROOT_PATH is $BK_CLUSTER_ROOT_PATH"
+
+
+mkdir -p "${BK_journalDirectory}" "${BK_ledgerDirectories}" "${BK_indexDirectories}"
+# -------------- #
+# Allow the container to be started with `--user`
+if [ "$1" = 'bookkeeper' -a "$(id -u)" = '0' ]; then
+    chown -R "$BK_USER:$BK_USER" "/opt/bookkeeper/" "${BK_journalDirectory}" "${BK_ledgerDirectories}"
"${BK_indexDirectories}"
+    sudo -s -E -u "$BK_USER" /bin/bash "$0" "$@"
+    exit
+fi
+# -------------- #
+
+python apply-config-from-env.py /opt/bookkeeper/conf
+
+echo "wait for zookeeper"
+until /opt/bookkeeper/bin/bookkeeper org.apache.zookeeper.ZooKeeperMain -server ${BK_zkServers}
ls /; do sleep 5; done
+
+echo "create the zk root dir for bookkeeper"
+/opt/zk/bin/zkCli.sh -server ${BK_zkServers} create ${BK_CLUSTER_ROOT_PATH}
+
+echo "format zk metadata"
+echo "please ignore the failure, if it has already been formatted, "
+export BOOKIE_CONF=/opt/bookkeeper/conf/bk_server.conf
+export SERVICE_PORT=$PORT0
+/opt/bookkeeper/bin/bookkeeper shell metaformat -n || true
+
+echo "run command by exec"
+exec "$@"
+
diff --git a/docker/scripts/healthcheck.sh b/docker/scripts/healthcheck.sh
new file mode 100755
index 0000000..87ce09d
--- /dev/null
+++ b/docker/scripts/healthcheck.sh
@@ -0,0 +1,28 @@
+#!/bin/bash
+#
+#/**
+# * Copyright 2007 The Apache Software Foundation
+# *
+# * Licensed to the Apache Software Foundation (ASF) under one
+# * or more contributor license agreements.  See the NOTICE file
+# * distributed with this work for additional information
+# * regarding copyright ownership.  The ASF licenses this file
+# * to you under the Apache License, Version 2.0 (the
+# * "License"); you may not use this file except in compliance
+# * with the License.  You may obtain a copy of the License at
+# *
+# *     http://www.apache.org/licenses/LICENSE-2.0
+# *
+# * Unless required by applicable law or agreed to in writing, software
+# * distributed under the License is distributed on an "AS IS" BASIS,
+# * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# * See the License for the specific language governing permissions and
+# * limitations under the License.
+# */
+
+#!/bin/bash
+
+set -x -e -u
+
+# Sanity check that creates a ledger, writes a few entries, reads them and deletes the ledger.
+bookkeeper shell bookiesanity

-- 
To stop receiving notification emails like this one, please contact
"commits@bookkeeper.apache.org" <commits@bookkeeper.apache.org>.

Mime
View raw message