Return-Path: X-Original-To: apmail-brooklyn-dev-archive@minotaur.apache.org Delivered-To: apmail-brooklyn-dev-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id AD9FA17938 for ; Wed, 24 Jun 2015 13:21:27 +0000 (UTC) Received: (qmail 52644 invoked by uid 500); 24 Jun 2015 13:21:27 -0000 Delivered-To: apmail-brooklyn-dev-archive@brooklyn.apache.org Received: (qmail 52613 invoked by uid 500); 24 Jun 2015 13:21:27 -0000 Mailing-List: contact dev-help@brooklyn.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@brooklyn.incubator.apache.org Delivered-To: mailing list dev@brooklyn.incubator.apache.org Received: (qmail 52601 invoked by uid 99); 24 Jun 2015 13:21:27 -0000 Received: from Unknown (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 24 Jun 2015 13:21:27 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id BE6A4CFE9B for ; Wed, 24 Jun 2015 13:21:26 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.575 X-Spam-Level: * X-Spam-Status: No, score=1.575 tagged_above=-999 required=6.31 tests=[KAM_BADIPHTTP=2, KAM_LAZY_DOMAIN_SECURITY=1, NORMAL_HTTP_TO_IP=0.001, RP_MATCHES_RCVD=-1.428, URIBL_BLOCKED=0.001, WEIRD_PORT=0.001] autolearn=disabled Received: from mx1-eu-west.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id xJj4HjhCOjtF for ; Wed, 24 Jun 2015 13:21:14 +0000 (UTC) Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx1-eu-west.apache.org (ASF Mail Server at mx1-eu-west.apache.org) with SMTP id 2CE6024F69 for ; Wed, 24 Jun 2015 13:21:13 +0000 (UTC) Received: (qmail 52556 invoked by uid 99); 24 Jun 2015 13:21:12 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 24 Jun 2015 13:21:12 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id 3D3A0DFD9C; Wed, 24 Jun 2015 13:21:12 +0000 (UTC) From: neykov To: dev@brooklyn.incubator.apache.org Reply-To: dev@brooklyn.incubator.apache.org References: In-Reply-To: Subject: [GitHub] incubator-brooklyn pull request: Catalog default niceness Content-Type: text/plain Message-Id: <20150624132112.3D3A0DFD9C@git1-us-west.apache.org> Date: Wed, 24 Jun 2015 13:21:12 +0000 (UTC) Github user neykov commented on a diff in the pull request: https://github.com/apache/incubator-brooklyn/pull/705#discussion_r33146037 --- Diff: usage/cli/src/main/resources/brooklyn/default.catalog.bom --- @@ -1,23 +1,359 @@ -# Licensed to the Apache Software Foundation (ASF) under one -# or more contributor license agreements. See the NOTICE file -# distributed with this work for additional information -# regarding copyright ownership. The ASF licenses this file -# to you under the Apache License, Version 2.0 (the -# "License"); you may not use this file except in compliance -# with the License. You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, -# software distributed under the License is distributed on an -# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -# KIND, either express or implied. See the License for the -# specific language governing permissions and limitations -# under the License. -# - -# this catalog bom simply scans. -# in a dist a default.catalog.bom from the conf/ dir takes precedence. + +# this catalog bom is an illustration supplying a few useful sample items +# and templates to get started using Brooklyn brooklyn.catalog: - scanJavaAnnotations: true + version: 0.7.0-SNAPSHOT # BROOKLYN_VERSION + items: + + # load everything in the classpath with a @Catalog annotation + - scanJavaAnnotations: true + + - id: server + description: | + Provision a server, with customizable provisioning.properties and credentials installed, + but no other special software process or scripts executed. + item: + type: brooklyn.entity.basic.EmptySoftwareProcess + name: Server + + - id: vanilla-bash-server + description: | + Provision a server, with customizable provisioning.properties and credentials installed, + but no other special software process or scripts executed. + The script should be supplied in "launch.command" as per docs on + brooklyn.entity.basic.VanillaSoftwareProcess. + item: + type: brooklyn.entity.basic.VanillaSoftwareProcess + name: Server with Launch Script (bash) + + - id: load-balancer + description: | + Create a load balancer which will point at members in the group entity + referred to by the config key "serverPool". + The sensor advertising the port can be configured with the "member.sensor.portNumber" config key, + defaulting to `http.port`; all member entities which have published "service.up" will then be picked up. + item: + type: brooklyn.entity.proxy.nginx.NginxController + name: Load Balancer (nginx) + + - id: cluster + description: | + Create a cluster of entities, resizable, with starting size "initialSize", + and using a spec supplied in the "memberSpec" key. + item: + type: brooklyn.entity.group.DynamicCluster + + - id: 1-server-template + itemType: template + name: "Template 1: Server" + description: | + Sample YAML to provision a server in a cloud with illustrative VM properties + item: + name: Server (Brooklyn Example) + + # this basic example shows how Brooklyn can provision a single raw VM + # in the cloud or location of your choice + + services: + - type: server + name: My VM + + # location can be e.g. `softlayer` or `jclouds:openstack-nova:https://9.9.9.9:9999/v2.0/`, + # or `localhost` or `byon: { nodes: [ 10.0.0.1, 10.0.0.2, 10.0.1.{1,2} ] }` + location: + jclouds:aws-ec2: + # edit these to use your credential (or delete if credentials specified in brooklyn.properties) + identity: + credential: + + region: eu-central-1 + + # we want Ubuntu, with a lot of RAM + osFamily: ubuntu + minRam: 8gb + + # set up this user and password (default is to authorize a public key) + user: sample + password: s4mpl3 + + - id: 2-bash-web-server-template + itemType: template + name: "Template 2: Bash Web Server" + description: | + Sample YAML building on Template 1, + adding bash commands to launch a Python-based web server + on port 8020 + item: + name: Python Web Server (Brooklyn Example) + + # this example builds on the previous one, + # adding some scripts to initialize the VM + + services: + - type: vanilla-bash-server + name: My Bash Web Server VM + brooklyn.config: + install.command: | + # install python if not present + which python || \ + { apt-get update && apt-get install python ; } || \ + { yum update && yum install python ; } || \ + { echo WARNING: cannot install python && exit 1 ; } + + customize.command: | + # create the web page to serve + cat > index.html << EOF + + Hello world. +

+ I am ${ENTITY_INFO}, ${MESSAGE:-a Brooklyn sample}. +

+ Created at: `date` +

+ I am running at ${HOSTNAME}, with on-box IP configuration: + 

    +            `ifconfig | grep inet`
    +
+ + EOF + + launch.command: | + # launch in background (ensuring no streams open), and record PID to file + nohup python -m SimpleHTTPServer ${PORT:-8020} < /dev/null > output.txt 2>&1 & + echo $! > ${PID_FILE:-pid.txt} + sleep 5 + ps -p `cat ${PID_FILE:-pid.txt}` + if [ $? -ne 0 ] ; then + cat output.txt + echo WARNING: python web server not running + exit 1 + fi + + shell.env: + HOSTNAME: $brooklyn:attributeWhenReady("host.name") + PORT: $brooklyn:config("my.app.port") + ENTITY_INFO: $brooklyn:component("this", "") + MESSAGE: $brooklyn:config("my.message") + + # custom + my.app.port: 8020 + my.message: "good to meet you" + + brooklyn.enrichers: + # publish the URL as a sensor; the GUI will pick this up (main.uri) + - type: brooklyn.enricher.basic.Transformer + brooklyn.config: + uniqueTag: url-generator + enricher.sourceSensor: host.subnet.hostname + # use the definition from Attributes class, as it has a RendererHint so GUI makes it a link + enricher.targetSensor: $brooklyn:sensor("brooklyn.entity.basic.Attributes", "main.uri") + enricher.targetValue: + $brooklyn:formatString: + - "http://%s:%s/" + - $brooklyn:attributeWhenReady("host.subnet.hostname") + - $brooklyn:config("my.app.port") + + location: + jclouds:aws-ec2: + region: eu-central-1 + # edit these (or delete if credentials specified in brooklyn.properties) + identity: + credential: + + - id: 3-bash-web-and-riak-template + itemType: template + name: "Template 3: Bash Web Server and Scaling Riak Cluster" + description: | + Sample YAML building on Template 2, + composing that blueprint with a Riak cluster and injecting the URL + item: + name: Bash Web Server and Riak Cluster (Brooklyn Example) + + # this example *references* the previous one, + # combining it with a stock blueprint for a Riak cluster, + # and shows how a sensor from the latter can be injected + + services: + + # reference template 2, overriding message to point at riak + - type: 2-bash-web-server-template + brooklyn.config: + my.message: $brooklyn:formatString("connected to Riak at %s", + $brooklyn:entity("riak-cluster").attributeWhenReady("main.uri")) + + # use the off-the-shelf Riak cluster + - type: brooklyn.entity.nosql.riak.RiakCluster + id: riak-cluster + initialSize: 3 + # and add a policy to scale based on ops per minute + brooklyn.policies: + - type: brooklyn.policy.autoscaling.AutoScalerPolicy + brooklyn.config: + metric: riak.node.ops.1m.perNode + # more than 100 ops per second (6k/min) scales out, less than 50 scales back + # up to a max of 8 riak nodes here (can be changed in GUI / REST API afterwards) + metricLowerBound: 3000 + metricUpperBound: 6000 + minPoolSize: 3 + maxPoolSize: 8 + resizeUpStabilizationDelay: 30s + resizeDownStabilizationDelay: 5m + + location: + jclouds:aws-ec2: + region: eu-central-1 + # edit these (or delete if credentials specified in brooklyn.properties) + identity: + credential: + + - id: 4-resilient-bash-web-cluster-template + itemType: template + name: "Template 4: Resilient Load-Balanced Bash Web Cluster with Sensors" + description: | + Sample YAML to provision a cluster of the bash/python web server nodes, + with sensors configured, and a load balancer pointing at them, + and resilience policies for node replacement and scaling + item: + name: Resilient Load-Balanced Bash Web Cluster (Brooklyn Example) + + # this final example shows some of the advanced functionality: + # defining custom sensors, and a cluster with a "spec", + # policies for resilience and scaling based on that sensor, + # and wiring a load balancer in front of the cluster + + # combining this with the riak cluster in the previous example + # is left as a suggested exercise for the user + + services: + + # define a cluster of the web nodes + - type: cluster + name: Cluster of Bash Web Nodes + id: my-web-cluster + brooklyn.config: + initialSize: 1 + memberSpec: + $brooklyn:entitySpec: + # template 2 is used as the spec for items in this cluster + # with a new message overwriting the previous, + # and a lot of sensors defined + type: 2-bash-web-server-template + name: My Bash Web Server VM with Sensors + + brooklyn.config: + my.message: "part of the cluster" + + brooklyn.initializers: + # make a simple request-count sensor, by counting the number of 200 responses in output.txt + - type: brooklyn.entity.software.ssh.SshCommandSensor + brooklyn.config: + name: reqs.count + targetType: int + period: 5s + command: "cat output.txt | grep HTTP | grep 200 | wc | awk '{print $1}'" + # and publish the port as a sensor so the load-balancer can pick it up + - type: brooklyn.entity.software.StaticSensor + brooklyn.config: + name: app.port + targetType: int + static.value: $brooklyn:config("my.app.port") + + brooklyn.enrichers: + # derive reqs.per_sec from reqs.count + - type: brooklyn.enricher.basic.YamlTimeWeightedDeltaEnricher + brooklyn.config: + enricher.sourceSensor: reqs.count + enricher.targetSensor: reqs.per_sec + enricher.delta.period: 1s + # and take an average over 30s for reqs.per_sec into reqs.per_sec.windowed_30s + - type: brooklyn.enricher.basic.YamlRollingTimeWindowMeanEnricher + brooklyn.config: + enricher.sourceSensor: reqs.per_sec + enricher.targetSensor: reqs.per_sec.windowed_30s + enricher.window.duration: 30s + + # emit failure sensor if a failure connecting to the service is sustained for 30s + - type: brooklyn.policy.ha.ServiceFailureDetector + brooklyn.config: + entityFailed.stabilizationDelay: 30s + + brooklyn.policies: + # restart if a failure is detected (with a max of one restart in 2m, sensor will propagate otherwise) + - type: brooklyn.policy.ha.ServiceRestarter + brooklyn.config: + failOnRecurringFailuresInThisDuration: 2m + + # back at the cluster, create a total per-sec and some per-node average + brooklyn.enrichers: + - type: brooklyn.enricher.basic.Aggregator + brooklyn.config: + enricher.sourceSensor: reqs.per_sec + enricher.targetSensor: reqs.per_sec + transformation: sum + - type: brooklyn.enricher.basic.Aggregator + brooklyn.config: + enricher.sourceSensor: reqs.per_sec + enricher.targetSensor: reqs.per_sec.per_node + transformation: average + - type: brooklyn.enricher.basic.Aggregator + brooklyn.config: + enricher.sourceSensor: reqs.per_sec.windowed_30s + enricher.targetSensor: reqs.per_sec.windowed_30s.per_node + transformation: average + + brooklyn.policies: + # resilience: if a per-node restart policy fails, + # just throw that node away and create a new one + - type: brooklyn.policy.ha.ServiceReplacer + + # and scale based on reqs/sec + - type: brooklyn.policy.autoscaling.AutoScalerPolicy + brooklyn.config: + # scale based on reqs/sec (though in a real-world situation, + # reqs.per_sec.windowed_30s.per_node might be a better choice) + metric: reqs.per_sec.per_node + + # really low numbers, so you can trigger a scale-out just by hitting reload a lot + metricUpperBound: 3 + metricLowerBound: 1 + + # sustain 3 reqs/sec for 2s and it will scale out + resizeUpStabilizationDelay: 2s + # only scale down when sustained for 1m + resizeDownStabilizationDelay: 1m + + maxPoolSize: 10 + + # and add a load-balancer pointing at the cluster + - type: load-balancer + id: load-bal + brooklyn.config: + # point this load balancer at the cluster, specifying port to forward to + loadbalancer.serverpool: $brooklyn:entity("my-web-cluster") + member.sensor.portNumber: app.port + + brooklyn.enrichers: + # publish a few useful info sensors and KPI's to the root of the app + - type: brooklyn.enricher.basic.Propagator + brooklyn.config: + uniqueTag: propagate-load-balancer-url + producer: $brooklyn:entity("load-bal") + propagating: + - main.uri + - type: brooklyn.enricher.basic.Propagator + brooklyn.config: + uniqueTag: propagate-reqs-per-sec + producer: $brooklyn:entity("my-web-cluster") + propagating: + - reqs.per_sec + - reqs.per_sec.windowed_30s.per_node + + location: + jclouds:aws-ec2: + # edit these (or delete if credentials specified in brooklyn.properties) + identity: + credential: + + region: eu-central-1 + minRam: 2gb --- End diff -- Excellent examples! A comprehensive showcase of the power of Brooklyn. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastructure@apache.org or file a JIRA ticket with INFRA. ---