Return-Path: X-Original-To: apmail-karaf-commits-archive@minotaur.apache.org Delivered-To: apmail-karaf-commits-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id AA075193CE for ; Thu, 28 Apr 2016 15:26:34 +0000 (UTC) Received: (qmail 47029 invoked by uid 500); 28 Apr 2016 15:26:34 -0000 Delivered-To: apmail-karaf-commits-archive@karaf.apache.org Received: (qmail 47004 invoked by uid 500); 28 Apr 2016 15:26:34 -0000 Mailing-List: contact commits-help@karaf.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@karaf.apache.org Delivered-To: mailing list commits@karaf.apache.org Received: (qmail 46995 invoked by uid 99); 28 Apr 2016 15:26:34 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 28 Apr 2016 15:26:34 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id 7073AC769F for ; Thu, 28 Apr 2016 15:26:33 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 0.804 X-Spam-Level: X-Spam-Status: No, score=0.804 tagged_above=-999 required=6.31 tests=[KAM_ASCII_DIVIDERS=0.8, KAM_LAZY_DOMAIN_SECURITY=1, RP_MATCHES_RCVD=-0.996] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id euf8ZwBD31aa for ; Thu, 28 Apr 2016 15:26:28 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTP id A8EE760DD4 for ; Thu, 28 Apr 2016 15:26:28 +0000 (UTC) Received: from svn01-us-west.apache.org (svn.apache.org [10.41.0.6]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id F2057E027F for ; Thu, 28 Apr 2016 15:27:13 +0000 (UTC) Received: from svn01-us-west.apache.org (localhost [127.0.0.1]) by svn01-us-west.apache.org (ASF Mail Server at svn01-us-west.apache.org) with ESMTP id B271F3A0051 for ; Thu, 28 Apr 2016 15:26:27 +0000 (UTC) Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: svn commit: r1741452 [1/9] - in /karaf/site/production/manual/cellar/latest-3: ./ architecture-guide/ images/ user-guide/ Date: Thu, 28 Apr 2016 15:26:26 -0000 To: commits@karaf.apache.org From: jbonofre@apache.org X-Mailer: svnmailer-1.0.9 Message-Id: <20160428152627.B271F3A0051@svn01-us-west.apache.org> Author: jbonofre Date: Thu Apr 28 15:26:24 2016 New Revision: 1741452 URL: http://svn.apache.org/viewvc?rev=1741452&view=rev Log: [scm-publish] Updating main site with Karaf Cellar manual Added: karaf/site/production/manual/cellar/latest-3/architecture-guide/ karaf/site/production/manual/cellar/latest-3/broadcasting_commands.html karaf/site/production/manual/cellar/latest-3/cloud.html karaf/site/production/manual/cellar/latest-3/deploy.html karaf/site/production/manual/cellar/latest-3/design.html karaf/site/production/manual/cellar/latest-3/event.html karaf/site/production/manual/cellar/latest-3/groups.html karaf/site/production/manual/cellar/latest-3/hazelcast.html karaf/site/production/manual/cellar/latest-3/images/ karaf/site/production/manual/cellar/latest-3/images/architecture.jpg (with props) karaf/site/production/manual/cellar/latest-3/images/cross_topology.jpg (with props) karaf/site/production/manual/cellar/latest-3/images/event_flow.jpg (with props) karaf/site/production/manual/cellar/latest-3/images/shared_architecture.jpg (with props) karaf/site/production/manual/cellar/latest-3/images/star_topology.jpg (with props) karaf/site/production/manual/cellar/latest-3/index.html karaf/site/production/manual/cellar/latest-3/installation.html karaf/site/production/manual/cellar/latest-3/introduction.html karaf/site/production/manual/cellar/latest-3/nodes.html karaf/site/production/manual/cellar/latest-3/obr.html karaf/site/production/manual/cellar/latest-3/overview.html karaf/site/production/manual/cellar/latest-3/supported_events.html karaf/site/production/manual/cellar/latest-3/transport.html karaf/site/production/manual/cellar/latest-3/user-guide/ Added: karaf/site/production/manual/cellar/latest-3/broadcasting_commands.html URL: http://svn.apache.org/viewvc/karaf/site/production/manual/cellar/latest-3/broadcasting_commands.html?rev=1741452&view=auto ============================================================================== --- karaf/site/production/manual/cellar/latest-3/broadcasting_commands.html (added) +++ karaf/site/production/manual/cellar/latest-3/broadcasting_commands.html Thu Apr 28 15:26:24 2016 @@ -0,0 +1,556 @@ + + + + + + + +Broadcasting commands + + + + + + +
+
+

Broadcasting commands

+
+

Commands are a special kind of event. They imply that when they are handled, a Result event will be fired +containing the outcome of the command. For each command, we have one result per recipient. Each command +contains an unique id (unique for all cluster nodes, created from Hazelcast). This id is used to correlate +the request with the result. For each result successfully correlated the result is added to list of results +on the command object. If the list gets full of if 10 seconds from the command execution have elapsed, the list +is moved to a blocking queue from which the result can be retrieved.

+
+
+

The following code snippet shows what happens when a command is sent for execution:

+
+
+
+
public Map<node,result> execute(Command command) throws Exception {
+   if (command == null) {
+      throw new Exception("Command store not found");
+   } else {
+      //store the command to correlate it with the result.
+      commandStore.getPending().put(command.getId(), command);
+      //I create a timeout task and schedule it
+      TimeoutTask timeoutTask = new TimeoutTask(command, commandStore);
+      ScheduledFuture timeoutFuture = timeoutScheduler.schedule(timeoutTask, command.getTimeout(), TimeUnit.MILLISECONDS);
+   }
+   if (producer != null) {
+      //send the command to the topic
+      producer.produce(command);
+      //retrieve the result list from the blocking queue.
+      return command.getResult();
+   }
+   throw new Exception("Command producer not found");
+}
+
+
+
+
+ + + \ No newline at end of file Added: karaf/site/production/manual/cellar/latest-3/cloud.html URL: http://svn.apache.org/viewvc/karaf/site/production/manual/cellar/latest-3/cloud.html?rev=1741452&view=auto ============================================================================== --- karaf/site/production/manual/cellar/latest-3/cloud.html (added) +++ karaf/site/production/manual/cellar/latest-3/cloud.html Thu Apr 28 15:26:24 2016 @@ -0,0 +1,678 @@ + + + + + + + +Discovery Services + + + + + + +
+
+

Discovery Services

+
+

The Discovery Services allow you to use third party libraries to discover the nodes member of the Cellar cluster.

+
+
+

jClouds

+
+

Cellar relies on Hazelcast in order to discover cluster nodes. This can happen either by using unicast, multicast or specifying the ip address of each node. +See the Core Configuration section for details.

+
+
+

Unfortunately multicast is not allowed in most IaaS providers and the alternative of specifying all IP addresses creates maintenance difficulties, especially since in most cases the addresses are not known in advance.

+
+
+

Cellar solves this problem using a cloud discovery service powered by jClouds.

+
+
+
Cloud discovery service
+
+

Most cloud providers provide cloud storage among other services. Cellar uses the cloud storage via jclouds, in order to determine the IP addresses of each node so that Hazelcast can find them.

+
+
+

This approach is also called blackboard and refers to the process where each node registers itself in a common storage are so that other nodes know its existence.

+
+
+
+
Installing Cellar cloud discovery service
+
+

To install the cloud discovery service simply install the appropriate jclouds provider and then install cellar-cloud feature. +Amazon S3 is being used here for this example, but the below applies to any provider supported by jclouds.

+
+
+
+
karaf@root()> feature:install jclouds-aws-s3
+karaf@root()> feature:install cellar-cloud
+
+
+
+

Once the feature is installed, you’re required to create a configuration that contains credentials and the type of the cloud storage (aka blobstore). +To do that add a configuration file under the etc folder with the name org.apache.karaf.cellar.cloud-<provider>.cfg and place the following information there:

+
+
+
+
provider=aws-s3 (this varies according to the blobstore provider)
+identity=<the identity of the blobstore account>
+credential=<the credential/password of the blobstore account)
+container=<the name of the bucket>
+validity=<the amount of time an entry is considered valid, after that time the entry is removed>
+
+
+
+

For instance, you can create etc/org.apache.karaf.cellar.cloud-mycloud.cfg containing:

+
+
+
+
provider=aws-s3
+identity=username
+credential=password
+container=cellar
+validity=360000
+
+
+
+ + + + + +
+
Note
+		 +
+

You can find the cloud providers supported by jclouds here http://repo1.maven.org/maven2/org/apache/jclouds/provider/. +You have to install the corresponding jclouds feature for the provider.

+
+
+
+
+

After creating the file the service will check for new nodes. If new nodes are found the Hazelcast instance configuration will be updated and the instance restarted.

+
+
+
+
Kubernetes & docker.io
+
+

Kubernetes is an open source orchestration system for docker.io containers. +It handles scheduling onto nodes in a compute cluster and actively manages workloads to ensure that their state matches +the users declared intentions. +Using the concepts of "labels", "pods", "replicationControllers" and "services", it groups the containers which make up +an application into logical units for easy management and discovery. +Following the aforementioned concept will most likely change how you package and provision your Karaf based applications. +For instance, you will eventually have to provide a Docker image with a pre-configured Karaf, KAR files in deployment +folder, etc. so that your Kubernetes container may bootstrap everything on boot.

+
+
+

The Cellar Kubernetes discovery service is a great complement to the Karaf docker.io feature (allowing you to easily +create and manage docker.io images in and for Karaf).

+
+
+
+
Kubernetes discovery service
+
+

In order to determine the IP address of each node, so that Hazelcast can connect to them, the Kubernetes discovery service queries +the Kubernetes API for containers labeled with the pod.label.key and pod.label.key specified in etc/org.apache.karaf.cellar.kubernetes-name.cfg. +The name in etc/org.apache.karaf.cellar.kubernetes-name.cfg is a name of the choice. It allows you to create multiple Kubernetes discovery services. +Thanks to that, the Cellar nodes can be discovered on different Kubernetes.

+
+
+

So, you must be sure to label your containers (pods) accordingly.

+
+
+

After a Cellar node starts up, Kubernetes discovery service will configure Hazelcast with currently running Cellar nodes. +Since Hazelcast follows a peer-to-peer all-shared topology, whenever nodes come up and down, the cluster will remain up-to-date.

+
+
+
+
Installing Kubernetes discovery service
+
+

To install the Kubernetes discovery service, simply install cellar-kubernetes feature.

+
+
+
+
karaf@root()> feature:install cellar-kubernetes
+
+
+
+

Once the cellar-kubernetes feature is installed, you have to create the Kubernetes provider configuration file. +If you have multiple Kubernetes instances, you create one configuration file per instance.

+
+
+

For instance, you can create etc/org.apache.karaf.cellar.kubernetes-myfirstcluster.cfg containing:

+
+
+
+
host=localhost
+port=8080
+pod.label.key=name
+pod.label.value=cellar
+
+
+
+

and another one etc/org.apache.karaf.cellar.kubernetes-mysecondcluster.cfg containing:

+
+
+
+
host=192.168.134.2
+port=8080
+pod.label.key=name
+pod.label.value=cellar
+
+
+
+

In case you change the file, the discovery service will check again for new nodes. If new nodes are found, Hazelcast configuration will be +updated and the instance restarted.

+
+
+
+
+
+ + + \ No newline at end of file