Architecture

Return-Path: X-Original-To: apmail-karaf-commits-archive@minotaur.apache.org Delivered-To: apmail-karaf-commits-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 4FA1D1945A for ; Thu, 28 Apr 2016 15:33:10 +0000 (UTC) Received: (qmail 82640 invoked by uid 500); 28 Apr 2016 15:33:10 -0000 Delivered-To: apmail-karaf-commits-archive@karaf.apache.org Received: (qmail 82614 invoked by uid 500); 28 Apr 2016 15:33:10 -0000 Mailing-List: contact commits-help@karaf.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@karaf.apache.org Delivered-To: mailing list commits@karaf.apache.org Received: (qmail 82605 invoked by uid 99); 28 Apr 2016 15:33:10 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 28 Apr 2016 15:33:10 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id 8F58DC0257 for ; Thu, 28 Apr 2016 15:33:09 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.799 X-Spam-Level: * X-Spam-Status: No, score=1.799 tagged_above=-999 required=6.31 tests=[KAM_ASCII_DIVIDERS=0.8, KAM_LAZY_DOMAIN_SECURITY=1, RP_MATCHES_RCVD=-0.001] autolearn=disabled Received: from mx2-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id REUQBOe88hwQ for ; Thu, 28 Apr 2016 15:32:58 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx2-lw-eu.apache.org (ASF Mail Server at mx2-lw-eu.apache.org) with ESMTP id 4796C5F298 for ; Thu, 28 Apr 2016 15:32:57 +0000 (UTC) Received: from svn01-us-west.apache.org (svn.apache.org [10.41.0.6]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 9E379E0500 for ; Thu, 28 Apr 2016 15:33:42 +0000 (UTC) Received: from svn01-us-west.apache.org (localhost [127.0.0.1]) by svn01-us-west.apache.org (ASF Mail Server at svn01-us-west.apache.org) with ESMTP id 6DB653A0A1D for ; Thu, 28 Apr 2016 15:32:56 +0000 (UTC) Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Subject: svn commit: r1741456 [2/6] - in /karaf/site/production/manual/decanter/latest-1: appenders.html architecture.html collectors.html custom-alerter.html custom-appender.html custom-collector.html dev-guide/ index.html introduction.html sla.html user-guide/ Date: Thu, 28 Apr 2016 15:32:56 -0000 To: commits@karaf.apache.org From: jbonofre@apache.org X-Mailer: svnmailer-1.0.9 Message-Id: <20160428153256.6DB653A0A1D@svn01-us-west.apache.org> Added: karaf/site/production/manual/decanter/latest-1/architecture.html URL: http://svn.apache.org/viewvc/karaf/site/production/manual/decanter/latest-1/architecture.html?rev=1741456&view=auto ============================================================================== --- karaf/site/production/manual/decanter/latest-1/architecture.html (added) +++ karaf/site/production/manual/decanter/latest-1/architecture.html Thu Apr 28 15:32:56 2016 @@ -0,0 +1,548 @@ + + + + + + + +Architecture + + + + + + +

Architecture

Apache Karaf Decanter uses OSGi EventAdmin to dispatch the harvested data between the collectors and the appenders, +and also to throw the alerts to the alerters:

+
decanter/collect/* EventAdmin topics are used by the collectors to send the harvested data. The appenders consume +from these topics and insert the data in a backend.
+
+
decanter/alert/* EventAdmin topics are used by the SLA checker to send the alerts. The SLA alerters consume from +these topics.
+

Decanter uses EventAdmin topics as monitoring events dispatcher.

Collectors, appenders, and alerters are simple OSGi services exposed by different bundles.

It means that you can easily extend Decanter adding your own collectors, appenders, or alerters.

+ + + \ No newline at end of file Added: karaf/site/production/manual/decanter/latest-1/collectors.html URL: http://svn.apache.org/viewvc/karaf/site/production/manual/decanter/latest-1/collectors.html?rev=1741456&view=auto ============================================================================== --- karaf/site/production/manual/decanter/latest-1/collectors.html (added) +++ karaf/site/production/manual/decanter/latest-1/collectors.html Thu Apr 28 15:32:56 2016 @@ -0,0 +1,1061 @@ + + + + + + + +Collectors + + + + + + +

Collectors

Decanter collectors harvest the monitoring data, and send this data to the Decanter appenders.

Two kinds of collector are available:

+
Event Driven Collectors react to events and "broadcast" the data to the appenders.
+
+
Polled Collectors are periodically executed by the Decanter Scheduler. When executed, the collectors harvest the +data and send to the appenders.
+

Log

The Decanter Log Collector is an event driven collector. It automatically reacts when a log occurs, and +send the log details (level, logger name, message, etc) to the appenders.

The decanter-collector-log feature installs the log collector:

karaf@root()> feature:install decanter-collector-log

The log collector doesn’t need any configuration, the installation of the decanter-collector-log feature is enough.

+ + + + + +

Note

The Decanter log collector is using osgi:DecanterLogCollectorAppender appender. +In order to work, your Apache Karaf Pax Logging configuration should contain this appender.

The default Apache Karaf etc/org.ops4j.pax.logging.cfg configuration file is already fine:

log4j.rootLogger = DEBUG, out, osgi:*

If you want, you can explicitly specify the DecanterLogCollectorAppender appender:

log4j.rootLogger = DEBUG, out, osgi:DecanterLogCollectorAppender, osgi:VmLogAppender

Log Socket

The Decanter Log Socket Collector is an event driven collector. It creates a socket, waiting for incoming event. The expected +events are log4j LoggingEvent. The log4j LoggingEvent is transformed as a Map containing the log details (level, logger name, message, …). +This Map is sent to the appenders.

The collector allows you to remotely use Decanter. For instance, you can have an application running on a different platform (spring-boot, +application servers, …). This application can use a log4j socket appender that send the logging events to the Decanter +log socket collector.

The decanter-collector-log-socket feature install the log socket collector:

karaf@root()> feature:install decanter-collector-log-socket

This feature installs the collector and a default etc/org.apache.karaf.decanter.collector.log.socket.cfg configuration file +containing:

#
+# Decanter Log/Log4j Socket collector configuration
+#
+
+#port=4560

+
the portÂ property defines the port number where the collector is bound and listen for incoming logging event. Default is 4560.
+

File

The Decanter File Collector is an event driven collector. It automatically reacts when new lines are appended into +a file (especially a log file). It acts like the tail Unix command. Basically, it’s an alternative to the log collector. +The log collector reacts for local Karaf log messages, whereas the file collector can react to any files, included log +file from other system than Karaf. It means that you can monitor and send collected data for any system (even not Java +base, or whatever).

The file collector deals with file rotation, file not found.

The decanter-collector-file feature installs the file collector:

karaf@root()> feature:install decanter-collector-file

Now, you have to create a configuration file for each file that you want to monitor. In the etc folder, you have to +create a file with the following format name etc/org.apache.karaf.decanter.collector.file-ID.cfg where ID is an ID +of your choice.

This file contains:

type=my
+path=/path/to/file
+any=value

+
type is an ID (mandatory) that allows you to easily identify the monitored file
+
+
path is the location of the file that you want to monitore
+
+
all other values (like any) will be part of the collected data. It means that you can add your own custom data, and +easily create queries bases on this data.
+

For instance, instead of the log collector, you can create the following etc/org.apache.karaf.decanter.collector.file-karaf.cfg +file collector configuration file:

type=karaf-log-file
+path=/path/to/karaf/data/log/karaf.log
+my=stuff

The file collector will tail on karaf.log file, and send any new line in this log file as collected data.

EventAdmin

The Decanter EventAdmin Collector is an event-driven collector, listening for all internal events happening in +the Apache Karaf Container.

+ + + + + +

Note

It’s the perfect way to audit all actions performed on resources (features, bundles, configurations, …) by users +(via local shell console, SSH, or JMX).

We recommend to use this collector to implement users and actions auditing.

The decanter-collector-eventadmin feature installs the eventadmin collector:

karaf@root()> feature:install decanter-collector-eventadmin

By default, the eventadmin collector is listening for all OSGi framework and Karaf internal events.

You can specify additional events to trap by providing a `etc/org.apache.karaf.decanter.collector.eventadmin-my.cfg' configuration +file, containing the EventAdmin topics you want to listen:

event.topics=my/*

+ + + + + +

Note

By default, the events contain timestamp and subject. +You can disable this by modifying etc/org.apache.felix.eventadmin.impl.EventAdmin configuration file:

org.apache.felix.eventadmin.AddTimestamp=true
+org.apache.felix.eventadmin.AddSubject=true

JMX

The Decanter JMX Collector is a polled collector, executed periodically by the Decanter Scheduler.

The JMX collector connects to a JMX MBeanServer (local or remote), and retrieves all attributes of each available MBeans. +The JMX metrics (attribute values) are send to the appenders.

The decanter-collector-jmx feature installs the JMX collector, and a default configuration file:

karaf@root()> feature:install decanter-collector-jmx

This feature brings a etc/org.apache.karaf.decanter.collector.jmx-local.cfg configuration file containing:

#
+# Decanter Local JMX collector configuration
+#
+
+# Name/type of the JMX collection
+type=jmx-local
+
+# URL of the JMX MBeanServer.
+# local keyword means the local platform MBeanServer or you can specify to full JMX URL
+# like service:jmx:rmi:///jndi/rmi://hostname:port/karaf-instance
+url=local
+
+# Username to connect to the JMX MBeanServer
+#username=karaf
+
+# Password to connect to the JMX MBeanServer
+#password=karaf
+
+# Object name filter to use. Instead of harvesting all MBeans, you can select only
+# some MBeans matching the object name filter
+#object.name=org.apache.camel:context=*,type=routes,name=*

This file harvests the data of the local MBeanServer:

+
the type property is a name (of your choice) allowing you to easily identify the harvested data
+
+
the url property is the MBeanServer to connect. "local" is reserved keyword to specify the local MBeanServer. +Instead of "local", you can use the JMX service URL. For instance, for Karaf version 3.0.0, 3.0.1, 3.0.2, and 3.0.3, +as the local MBeanServer is secured, you can specify service:jmx:rmi:///jndi/rmi://localhost:1099/karaf-root. You +can also polled any remote MBean server (Karaf based or not) providing the service URL.
+
+
the username property contains the username to connect to the MBean server. It’s only required if the MBean server +is secured.
+
+
the password property contains the password to connect to the MBean server. It’s only required if the MBean server +is secured.
+
+
the object.name property is optional. If this property is not specified, the collector will retrieve the attributes +of all MBeans. You can filter to consider only some MBeans. This property contains the ObjectName filter to retrieve +the attributes only to some MBeans.
+
+
any other values will be part of the collected data. It means that you can add your own property if you want to add +additional data, and create queries based on this data.
+

You can retrieve multiple MBean servers. For that, you just create a new configuration file using the file name format +etc/org.apache.karaf.decanter.collector.jmx-[ANYNAME].cfg.

ActiveMQ (JMX)

The ActiveMQ JMX collector is just a special configuration of the JMX collector.

The decanter-collector-activemq feature installs the default JMX collector, with the specific ActiveMQ JMX configuration:

karaf@root()> feature:install decanter-collector-activemq

This feature installs the same collector as the decanter-collector-jmx, but also add the +etc/org.apache.karaf.decanter.collector.jmx-activemq.cfg configuration file.

This file contains:

#
+# Decanter Local ActiveMQ JMX collector configuration
+#
+
+# Name/type of the JMX collection
+type=jmx-activemq
+
+# URL of the JMX MBeanServer.
+# local keyword means the local platform MBeanServer or you can specify to full JMX URL
+# like service:jmx:rmi:///jndi/rmi://hostname:port/karaf-instance
+url=local
+
+# Username to connect to the JMX MBeanServer
+#username=karaf
+
+# Password to connect to the JMX MBeanServer
+#password=karaf
+
+# Object name filter to use. Instead of harvesting all MBeans, you can select only
+# some MBeans matching the object name filter
+object.name=org.apache.activemq:*

This configuration actually contains a filter to retrieve only the ActiveMQ JMX MBeans.

Camel (JMX)

The Camel JMX collector is just a special configuration of the JMX collector.

The decanter-collector-camel feature installs the default JMX collector, with the specific Camel JMX configuration:

karaf@root()> feature:install decanter-collector-camel

This feature installs the same collector as the decanter-collector-jmx, but also add the +etc/org.apache.karaf.decanter.collector.jmx-camel.cfg configuration file.

This file contains:

#
+# Decanter Local Camel JMX collector configuration
+#
+
+# Name/type of the JMX collection
+type=jmx-camel
+
+# URL of the JMX MBeanServer.
+# local keyword means the local platform MBeanServer or you can specify to full JMX URL
+# like service:jmx:rmi:///jndi/rmi://hostname:port/karaf-instance
+url=local
+
+# Username to connect to the JMX MBeanServer
+#username=karaf
+
+# Password to connect to the JMX MBeanServer
+#password=karaf
+
+# Object name filter to use. Instead of harvesting all MBeans, you can select only
+# some MBeans matching the object name filter
+object.name=org.apache.camel:context=*,type=routes,name=*

This configuration actually contains a filter to retrieve only the Camel Routes JMX MBeans.

Camel Tracer

The Camel Tracer provides a Camel Tracer Handler that you can set on a Camel Tracer.

If you enable the tracer on a Camel route, all tracer events (exchanges on each step of the route) are send to the +appenders.

The decanter-collector-camel-tracer feature provides the Camel Tracer Handler:

karaf@root()> feature:install decanter-collector-camel-tracer

Now, you can use the Decanter Camel Tracer Handler in a tracer that you can use in routes.

For instance, the following route definition shows how to enable tracer on a route, and use the Decanter Tracer Handler +in the Camel Tracer:

<?xml version="1.0" encoding="UTF-8"?>
+<blueprint xmlns="http://www.osgi.org/xmlns/blueprint/v1.0.0">
+
+    <reference id="eventAdmin" interface="org.osgi.service.event.EventAdmin"/>
+
+    <bean id="traceHandler" class="org.apache.karaf.decanter.collector.camel.DecanterTraceEventHandler">
+        <property name="eventAdmin" ref="eventAdmin"/>
+    </bean>
+
+    <bean id="tracer" class="org.apache.camel.processor.interceptor.Tracer">
+        <property name="traceHandler" ref="traceHandler"/>
+        <property name="enabled" value="true"/>
+        <property name="traceOutExchanges" value="true"/>
+        <property name="logLevel" value="OFF"/>
+    </bean>
+
+    <camelContext trace="true" xmlns="http://camel.apache.org/schema/blueprint">
+        <route id="test">
+            <from uri="timer:fire?period=10000"/>
+            <setBody><constant>Hello World</constant></setBody>
+            <to uri="log:test"/>
+        </route>
+    </camelContext>
+
+</blueprint>

System

The system collector is a polled collector (periodically executed by the Decanter Scheduler).

This collector executes operating system commands (or scripts) and send the execution output to the appenders.

The decanter-collector-system feature installs the system collector:

karaf@root()> feature:install decanter-collector-system

This feature installs a default etc/org.apache.karaf.decanter.collector.system.cfg configuration file containing:

#
+# Decanter OperationSystem Collector configuration
+#
+
+# This collector executes system commands, retrieve the exec output/err
+#Â sent to the appenders
+#
+# The format is key=command
+# for instance:
+# df=df -h
+# free=free
+# You can also create a script containing command like:
+#
+#   df -k / | awk -F " |%" '/dev/{print $8}'
+#
+# This script will get the available space on the / filesystem for instance.
+# and call the script:
+# df=/bin/script
+#
+# Another example of script to get the temperature:
+#
+#   sensors|grep temp1|awk '{print $2}'|cut -b2,3,4,5
+#

You can add the commands that you want to execute using the format:

name=command

The collector will execute each command described in this file, and send the execution output to the appenders.

For instance, if you want to periodically send the free space available on the / filesystem, you can add:

df=df -k / | awk -F " |%" '/dev/{print $8}'

+ + + \ No newline at end of file