metron-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ma...@apache.org
Subject svn commit: r20216 [11/18] - in /dev/metron/0.4.0-RC4: ./ site-book/ site-book/css/ site-book/images/ site-book/images/logos/ site-book/images/profiles/ site-book/img/ site-book/js/ site-book/metron-analytics/ site-book/metron-analytics/metron-maas-ser...
Date Tue, 27 Jun 2017 18:15:56 GMT
Added: dev/metron/0.4.0-RC4/site-book/metron-docker/index.html
==============================================================================
--- dev/metron/0.4.0-RC4/site-book/metron-docker/index.html (added)
+++ dev/metron/0.4.0-RC4/site-book/metron-docker/index.html Tue Jun 27 18:15:56 2017
@@ -0,0 +1,486 @@
+<!DOCTYPE html>
+<!--
+ | Generated by Apache Maven Doxia at 2017-06-27
+ | Rendered using Apache Maven Fluido Skin 1.3.0
+-->
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+  <head>
+    <meta charset="UTF-8" />
+    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+    <meta name="Date-Revision-yyyymmdd" content="20170627" />
+    <meta http-equiv="Content-Language" content="en" />
+    <title>Metron &#x2013; Metron Docker</title>
+    <link rel="stylesheet" href="../css/apache-maven-fluido-1.3.0.min.css" />
+    <link rel="stylesheet" href="../css/site.css" />
+    <link rel="stylesheet" href="../css/print.css" media="print" />
+
+      
+    <script type="text/javascript" src="../js/apache-maven-fluido-1.3.0.min.js"></script>
+
+                          
+        
+<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
+          
+            </head>
+        <body class="topBarDisabled">
+          
+                
+                    
+    
+        <div class="container-fluid">
+          <div id="banner">
+        <div class="pull-left">
+                                    <a href="http://metron.apache.org/" id="bannerLeft">
+                                                                                                <img src="../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/>
+                </a>
+                      </div>
+        <div class="pull-right">  </div>
+        <div class="clear"><hr/></div>
+      </div>
+
+      <div id="breadcrumbs">
+        <ul class="breadcrumb">
+                
+                    
+                              <li class="">
+                    <a href="http://www.apache.org" class="externalLink" title="Apache">
+        Apache</a>
+        </li>
+      <li class="divider ">/</li>
+            <li class="">
+                    <a href="http://metron.apache.org/" class="externalLink" title="Metron">
+        Metron</a>
+        </li>
+      <li class="divider ">/</li>
+            <li class="">
+                    <a href="../index.html" title="Documentation">
+        Documentation</a>
+        </li>
+      <li class="divider ">/</li>
+        <li class="">Metron Docker</li>
+        
+                
+                    
+                  <li id="publishDate" class="pull-right">Last Published: 2017-06-27</li> <li class="divider pull-right">|</li>
+              <li id="projectVersion" class="pull-right">Version: 0.4.0</li>
+            
+                            </ul>
+      </div>
+
+            
+      <div class="row-fluid">
+        <div id="leftColumn" class="span3">
+          <div class="well sidebar-nav">
+                
+                    
+                <ul class="nav nav-list">
+                    <li class="nav-header">User Documentation</li>
+                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          
+      <li>
+    
+                          <a href="../index.html" title="Metron">
+          <i class="icon-chevron-down"></i>
+        Metron</a>
+                    <ul class="nav nav-list">
+                      
+      <li>
+    
+                          <a href="../Upgrading.html" title="Upgrading">
+          <i class="none"></i>
+        Upgrading</a>
+            </li>
+                                                                                                                                                      
+      <li>
+    
+                          <a href="../metron-analytics/index.html" title="Analytics">
+          <i class="icon-chevron-right"></i>
+        Analytics</a>
+                  </li>
+                                                                                                                                                                                                                                                                                                                                                                                    
+      <li>
+    
+                          <a href="../metron-deployment/index.html" title="Deployment">
+          <i class="icon-chevron-right"></i>
+        Deployment</a>
+                  </li>
+                      
+      <li class="active">
+    
+            <a href="#"><i class="none"></i>Docker</a>
+          </li>
+                      
+      <li>
+    
+                          <a href="../metron-interface/metron-config/index.html" title="Config">
+          <i class="none"></i>
+        Config</a>
+            </li>
+                      
+      <li>
+    
+                          <a href="../metron-interface/metron-rest/index.html" title="Rest">
+          <i class="none"></i>
+        Rest</a>
+            </li>
+                                                                                                                                                                                                                                                
+      <li>
+    
+                          <a href="../metron-platform/index.html" title="Platform">
+          <i class="icon-chevron-right"></i>
+        Platform</a>
+                  </li>
+                                                                                                            
+      <li>
+    
+                          <a href="../metron-sensors/index.html" title="Sensors">
+          <i class="icon-chevron-right"></i>
+        Sensors</a>
+                  </li>
+              </ul>
+        </li>
+            </ul>
+                
+                    
+                
+          <hr class="divider" />
+
+           <div id="poweredBy">
+                            <div class="clear"></div>
+                            <div class="clear"></div>
+                            <div class="clear"></div>
+                             <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
+        <img class="builtBy" alt="Built by Maven" src="../images/logos/maven-feather.png" />
+      </a>
+                  </div>
+          </div>
+        </div>
+        
+                
+        <div id="bodyColumn"  class="span9" >
+                                  
+            <h1>Metron Docker</h1>
+<p><a name="Metron_Docker"></a></p>
+<p>Metron Docker is a <a class="externalLink" href="https://docs.docker.com/compose/overview/">Docker Compose</a> application that is intended for development and integration testing of Metron. Use this instead of Vagrant when:</p>
+
+<ul>
+  
+<li>You want an environment that can be built and spun up quickly</li>
+  
+<li>You need to frequently rebuild and restart services</li>
+  
+<li>You only need to test, troubleshoot or develop against a subset of services</li>
+</ul>
+<p>Metron Docker includes these images that have been customized for Metron:</p>
+
+<ul>
+  
+<li>Kafka (with Zookeeper)</li>
+  
+<li>HBase</li>
+  
+<li>Storm (with all topologies deployed)</li>
+  
+<li>Elasticsearch</li>
+  
+<li>Kibana</li>
+  
+<li>HDFS</li>
+</ul>
+<div class="section">
+<h2><a name="Setup"></a>Setup</h2>
+<p>Install <a class="externalLink" href="https://docs.docker.com/docker-for-mac/">Docker for Mac</a> or <a class="externalLink" href="https://docs.docker.com/docker-for-windows/">Docker for Windows</a>. The following versions have been tested:</p>
+
+<ul>
+  
+<li>Docker version 1.12.0</li>
+  
+<li>docker-machine version 0.8.0</li>
+  
+<li>docker-compose version 1.8.0</li>
+</ul>
+<p>Build Metron from the top level directory with:</p>
+
+<div class="source">
+<div class="source">
+<pre>$ cd $METRON_HOME
+$ mvn clean install -DskipTests
+</pre></div></div>
+<p>You are welcome to use an existing Docker host but we prefer one with more resources. You can create one of those with this script:</p>
+
+<div class="source">
+<div class="source">
+<pre>$ export METRON_DOCKER_HOME=$METRON_HOME/metron-docker
+$ cd $METRON_DOCKER_HOME &amp;&amp; ./scripts/create-docker-machine.sh
+</pre></div></div>
+<p>This will create a host called &#x201c;metron-machine&#x201d;. Anytime you want to run Docker commands against this host, make sure you run this first to set the Docker environment variables:</p>
+
+<div class="source">
+<div class="source">
+<pre>$ eval &quot;$(docker-machine env metron-machine)&quot;
+</pre></div></div>
+<p>If you wish to use a local docker-engine install, please set an environment variable BROKER_IP_ADDR to the IP address of your host machine. This cannot be the loopback address.</p></div>
+<div class="section">
+<h2><a name="Usage"></a>Usage</h2>
+<p>Navigate to the compose application root:</p>
+
+<div class="source">
+<div class="source">
+<pre>$ cd $METRON_DOCKER_HOME/compose/
+</pre></div></div>
+<p>The Metron Docker environment lifecycle is controlled by the <a class="externalLink" href="https://docs.docker.com/compose/reference/overview/">docker-compose</a> command. The service names can be found in the docker-compose.yml file. For example, to build and start the environment run this command:</p>
+
+<div class="source">
+<div class="source">
+<pre>$ eval &quot;$(docker-machine env metron-machine)&quot;
+$ docker-compose up -d
+</pre></div></div>
+<p>After all services have started list the containers and ensure their status is &#x2018;Up&#x2019;:</p>
+
+<div class="source">
+<div class="source">
+<pre>$ docker ps --format 'table {{.Names}}\t{{.Status}}'
+NAMES                    STATUS
+metron_storm_1           Up 5 minutes
+metron_hbase_1           Up 5 minutes
+metron_kibana_1          Up 5 minutes
+metron_kafkazk_1         Up 5 minutes
+metron_elasticsearch_1   Up 5 minutes
+</pre></div></div>
+<p>Various services are exposed through http on the Docker host. Get the host ip from the URL property:</p>
+
+<div class="source">
+<div class="source">
+<pre>$ docker-machine ls
+NAME             ACTIVE   DRIVER       STATE     URL                         SWARM   DOCKER    ERRORS
+metron-machine   *        virtualbox   Running   tcp://192.168.99.100:2376           v1.12.5
+</pre></div></div>
+<p>Then, assuming a host ip of <tt>192.168.99.100</tt>, the UIs and APIs are available at:</p>
+
+<ul>
+  
+<li>Storm - <a class="externalLink" href="http://192.168.99.100:8080/">http://192.168.99.100:8080/</a></li>
+  
+<li>HBase - <a class="externalLink" href="http://192.168.99.100:16010/">http://192.168.99.100:16010/</a></li>
+  
+<li>Elasticsearch - <a class="externalLink" href="http://192.168.99.100:9200/_plugin/head/">http://192.168.99.100:9200/_plugin/head/</a></li>
+  
+<li>Kibana - <a class="externalLink" href="http://192.168.99.100:5601/">http://192.168.99.100:5601/</a></li>
+  
+<li>HDFS (Namenode) - <a class="externalLink" href="http://192.168.99.100:50070/">http://192.168.99.100:50070/</a></li>
+</ul>
+<p>The Storm logs can be useful when troubleshooting topologies. They can be found on the Storm container in <tt>/usr/share/apache-storm/logs</tt>.</p>
+<p>When done using the machine, shut it down with:</p>
+
+<div class="source">
+<div class="source">
+<pre>$ docker-compose down
+</pre></div></div></div>
+<div class="section">
+<h2><a name="Examples"></a>Examples</h2>
+
+<ul>
+  
+<li><a href="#Deploy_a_new_parser_class">Deploy a new parser class</a></li>
+  
+<li><a href="#Connect_to_a_container">Connect to a container</a></li>
+  
+<li><a href="create-a-sensor-from-sample-data/index.html">Create a sensor from sample data</a></li>
+  
+<li><a href="upload-configs-to-zookeeper/index.html">Upload configs to Zookeeper</a></li>
+  
+<li><a href="manage-a-topology/index.html">Manage a topology</a></li>
+  
+<li><a href="run-sensor-data-end-to-end/index.html">Run sensor data end to end</a></li>
+</ul>
+<div class="section">
+<h3><a name="Deploy_a_new_parser_class"></a>Deploy a new parser class</h3>
+<p>After adding a new parser to metron-parsers, build Metron from the top level directory:</p>
+
+<div class="source">
+<div class="source">
+<pre>$ cd $METRON_HOME
+$ mvn clean install -DskipTests
+</pre></div></div>
+<p>Then run these commands to redeploy the parsers to the Storm image:</p>
+
+<div class="source">
+<div class="source">
+<pre>$ cd $METRON_DOCKER_HOME/compose
+$ docker-compose down
+$ docker-compose build storm
+$ docker-compose up -d
+</pre></div></div></div>
+<div class="section">
+<h3><a name="Connect_to_a_container"></a>Connect to a container</h3>
+<p>Suppose there is a problem with Kafka and the logs are needed for further investigation. Run this command to connect and explore the running Kafka container:</p>
+
+<div class="source">
+<div class="source">
+<pre>$ cd $METRON_DOCKER_HOME/compose
+$ docker-compose exec kafkazk bash
+</pre></div></div></div>
+<div class="section">
+<h3><a name="Create_a_sensor_from_sample_data"></a>Create a sensor from sample data</h3>
+<p>A tool for producing test data in Kafka is included with the Kafka/Zookeeper image. It loops through lines in a test data file and outputs them to Kafka at the desired frequency. Create a test data file in <tt>./kafkazk/data/</tt> and rebuild the Kafka/Zookeeper image:</p>
+
+<div class="source">
+<div class="source">
+<pre>$ cd $METRON_DOCKER_HOME/compose
+$ printf 'first test data\nsecond test data\nthird test data\n' &gt; ./kafkazk/data/TestData.txt
+$ docker-compose down
+$ docker-compose build kafkazk
+$ docker-compose up -d
+</pre></div></div>
+<p>This will deploy the test data file to the Kafka/Zookeeper container. Now that data can be streamed to a Kafka topic:</p>
+
+<div class="source">
+<div class="source">
+<pre>$ docker-compose exec kafkazk ./bin/produce-data.sh
+Usage:  produce-data.sh data_path topic [message_delay_in_seconds]
+
+# Stream data in TestData.txt to the 'test' Kafka topic at a frequency of 5 seconds (default is 1 second)
+$ docker-compose exec kafkazk ./bin/produce-data.sh /data/TestData.txt test 5 
+</pre></div></div>
+<p>The Kafka/Zookeeper image comes with sample Bro and Squid data:</p>
+
+<div class="source">
+<div class="source">
+<pre># Stream Bro test data every 1 second
+$ docker-compose exec kafkazk ./bin/produce-data.sh /data/BroExampleOutput.txt bro
+
+# Stream Squid test data every 0.1 seconds
+$ docker-compose exec kafkazk ./bin/produce-data.sh /data/SquidExampleOutput.txt squid 0.1
+</pre></div></div></div>
+<div class="section">
+<h3><a name="Upload_configs_to_Zookeeper"></a>Upload configs to Zookeeper</h3>
+<p>Parser configs and a global config configured for this Docker environment are included with the Kafka/Zookeeper image. Load them with:</p>
+
+<div class="source">
+<div class="source">
+<pre>$ docker-compose exec kafkazk bash
+# $METRON_HOME/bin/zk_load_configs.sh -z localhost:2181 -m PUSH -i $METRON_HOME/config/zookeeper
+# exit
+</pre></div></div>
+<p>Dump out the configs with:</p>
+
+<div class="source">
+<div class="source">
+<pre>$ docker-compose exec kafkazk bash
+# $METRON_HOME/bin/zk_load_configs.sh -z localhost:2181 -m DUMP
+# exit
+</pre></div></div></div>
+<div class="section">
+<h3><a name="Manage_a_topology"></a>Manage a topology</h3>
+<p>The Storm image comes with a script to easily start parser topologies:</p>
+
+<div class="source">
+<div class="source">
+<pre>docker-compose exec storm ./bin/start_docker_parser_topology.sh sensor_name
+</pre></div></div>
+<p>The enrichment topology can be started with:</p>
+
+<div class="source">
+<div class="source">
+<pre>docker-compose exec storm ./bin/start_enrichment_topology.sh
+</pre></div></div>
+<p>The indexing topology can be started with:</p>
+
+<div class="source">
+<div class="source">
+<pre>docker-compose exec storm ./bin/start_elasticsearch_topology.sh
+</pre></div></div>
+<p>Topologies can be stopped using the Storm CLI. For example, stop the enrichment topology with:</p>
+
+<div class="source">
+<div class="source">
+<pre>docker-compose exec storm storm kill enrichments -w 0
+</pre></div></div></div>
+<div class="section">
+<h3><a name="Run_sensor_data_end_to_end"></a>Run sensor data end to end</h3>
+<p>First ensure configs were uploaded as described in the previous example. Then start a sensor and leave it running:</p>
+
+<div class="source">
+<div class="source">
+<pre>$ cd $METRON_DOCKER_HOME/compose
+$ docker-compose exec kafkazk ./bin/produce-data.sh /data/BroExampleOutput.txt bro
+</pre></div></div>
+<p>Open a separate console session and verify the sensor is running by consuming a message from Kafka:</p>
+
+<div class="source">
+<div class="source">
+<pre>$ export METRON_DOCKER_HOME=$METRON_HOME/metron-docker
+$ cd $METRON_DOCKER_HOME/compose
+$ docker-compose exec kafkazk ./bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic bro
+</pre></div></div>
+<p>A new message should be printed every second. Now kill the consumer and start the Bro parser topology:</p>
+
+<div class="source">
+<div class="source">
+<pre>$ docker-compose exec storm ./bin/start_docker_parser_topology.sh bro
+</pre></div></div>
+<p>Bro data should be flowing through the bro parser topology and into the Kafka enrichments topic. The enrichments topic should be created automatically:</p>
+
+<div class="source">
+<div class="source">
+<pre>$ docker-compose exec kafkazk ./bin/kafka-topics.sh --zookeeper localhost:2181 --list
+bro
+enrichments
+indexing
+</pre></div></div>
+<p>Verify parsed Bro data is in the Kafka enrichments topic:</p>
+
+<div class="source">
+<div class="source">
+<pre>docker-compose exec kafkazk ./bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic enrichments
+</pre></div></div>
+<p>Now start the enrichment topology:</p>
+
+<div class="source">
+<div class="source">
+<pre>docker-compose exec storm ./bin/start_enrichment_topology.sh
+</pre></div></div>
+<p>Parsed Bro data should be flowing through the enrichment topology and into the Kafka indexing topic. Verify enriched Bro data is in the Kafka indexing topic:</p>
+
+<div class="source">
+<div class="source">
+<pre>docker-compose exec kafkazk ./bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic indexing
+</pre></div></div>
+<p>Now start the indexing topology:</p>
+
+<div class="source">
+<div class="source">
+<pre>docker-compose exec storm ./bin/start_elasticsearch_topology.sh
+</pre></div></div>
+<p>Enriched Bro data should now be present in the Elasticsearch container:</p>
+
+<div class="source">
+<div class="source">
+<pre>$ docker-machine ls
+NAME             ACTIVE   DRIVER       STATE     URL                         SWARM   DOCKER    ERRORS
+metron-machine   *        virtualbox   Running   tcp://192.168.99.100:2376           v1.12.5
+
+$ curl -XGET http://192.168.99.100:9200/_cat/indices?v
+health status index                   pri rep docs.count docs.deleted store.size pri.store.size
+yellow open   .kibana                   1   1          1            0      3.1kb          3.1kb
+yellow open   bro_index_2016.12.19.18   5   1        180            0      475kb          475kb
+</pre></div></div></div></div>
+                  </div>
+            </div>
+          </div>
+
+    <hr/>
+
+    <footer>
+            <div class="container-fluid">
+              <div class="row span12">Copyright &copy;                    2017
+                        <a href="https://www.apache.org">The Apache Software Foundation</a>.
+            All Rights Reserved.      
+                    
+      </div>
+
+                          
+        
+                </div>
+    </footer>
+  </body>
+</html>

Added: dev/metron/0.4.0-RC4/site-book/metron-interface/metron-config/index.html
==============================================================================
--- dev/metron/0.4.0-RC4/site-book/metron-interface/metron-config/index.html (added)
+++ dev/metron/0.4.0-RC4/site-book/metron-interface/metron-config/index.html Tue Jun 27 18:15:56 2017
@@ -0,0 +1,328 @@
+<!DOCTYPE html>
+<!--
+ | Generated by Apache Maven Doxia at 2017-06-27
+ | Rendered using Apache Maven Fluido Skin 1.3.0
+-->
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+  <head>
+    <meta charset="UTF-8" />
+    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+    <meta name="Date-Revision-yyyymmdd" content="20170627" />
+    <meta http-equiv="Content-Language" content="en" />
+    <title>Metron &#x2013; Metron Management UI</title>
+    <link rel="stylesheet" href="../../css/apache-maven-fluido-1.3.0.min.css" />
+    <link rel="stylesheet" href="../../css/site.css" />
+    <link rel="stylesheet" href="../../css/print.css" media="print" />
+
+      
+    <script type="text/javascript" src="../../js/apache-maven-fluido-1.3.0.min.js"></script>
+
+                          
+        
+<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
+          
+            </head>
+        <body class="topBarDisabled">
+          
+                
+                    
+    
+        <div class="container-fluid">
+          <div id="banner">
+        <div class="pull-left">
+                                    <a href="http://metron.apache.org/" id="bannerLeft">
+                                                                                                <img src="../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/>
+                </a>
+                      </div>
+        <div class="pull-right">  </div>
+        <div class="clear"><hr/></div>
+      </div>
+
+      <div id="breadcrumbs">
+        <ul class="breadcrumb">
+                
+                    
+                              <li class="">
+                    <a href="http://www.apache.org" class="externalLink" title="Apache">
+        Apache</a>
+        </li>
+      <li class="divider ">/</li>
+            <li class="">
+                    <a href="http://metron.apache.org/" class="externalLink" title="Metron">
+        Metron</a>
+        </li>
+      <li class="divider ">/</li>
+            <li class="">
+                    <a href="../../index.html" title="Documentation">
+        Documentation</a>
+        </li>
+      <li class="divider ">/</li>
+        <li class="">Metron Management UI</li>
+        
+                
+                    
+                  <li id="publishDate" class="pull-right">Last Published: 2017-06-27</li> <li class="divider pull-right">|</li>
+              <li id="projectVersion" class="pull-right">Version: 0.4.0</li>
+            
+                            </ul>
+      </div>
+
+            
+      <div class="row-fluid">
+        <div id="leftColumn" class="span3">
+          <div class="well sidebar-nav">
+                
+                    
+                <ul class="nav nav-list">
+                    <li class="nav-header">User Documentation</li>
+                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          
+      <li>
+    
+                          <a href="../../index.html" title="Metron">
+          <i class="icon-chevron-down"></i>
+        Metron</a>
+                    <ul class="nav nav-list">
+                      
+      <li>
+    
+                          <a href="../../Upgrading.html" title="Upgrading">
+          <i class="none"></i>
+        Upgrading</a>
+            </li>
+                                                                                                                                                      
+      <li>
+    
+                          <a href="../../metron-analytics/index.html" title="Analytics">
+          <i class="icon-chevron-right"></i>
+        Analytics</a>
+                  </li>
+                                                                                                                                                                                                                                                                                                                                                                                    
+      <li>
+    
+                          <a href="../../metron-deployment/index.html" title="Deployment">
+          <i class="icon-chevron-right"></i>
+        Deployment</a>
+                  </li>
+                      
+      <li>
+    
+                          <a href="../../metron-docker/index.html" title="Docker">
+          <i class="none"></i>
+        Docker</a>
+            </li>
+                      
+      <li class="active">
+    
+            <a href="#"><i class="none"></i>Config</a>
+          </li>
+                      
+      <li>
+    
+                          <a href="../../metron-interface/metron-rest/index.html" title="Rest">
+          <i class="none"></i>
+        Rest</a>
+            </li>
+                                                                                                                                                                                                                                                
+      <li>
+    
+                          <a href="../../metron-platform/index.html" title="Platform">
+          <i class="icon-chevron-right"></i>
+        Platform</a>
+                  </li>
+                                                                                                            
+      <li>
+    
+                          <a href="../../metron-sensors/index.html" title="Sensors">
+          <i class="icon-chevron-right"></i>
+        Sensors</a>
+                  </li>
+              </ul>
+        </li>
+            </ul>
+                
+                    
+                
+          <hr class="divider" />
+
+           <div id="poweredBy">
+                            <div class="clear"></div>
+                            <div class="clear"></div>
+                            <div class="clear"></div>
+                             <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
+        <img class="builtBy" alt="Built by Maven" src="../../images/logos/maven-feather.png" />
+      </a>
+                  </div>
+          </div>
+        </div>
+        
+                
+        <div id="bodyColumn"  class="span9" >
+                                  
+            <h1>Metron Management UI</h1>
+<p><a name="Metron_Management_UI"></a></p>
+<p>This module provides a user interface for management functions in Metron.</p>
+<div class="section">
+<h2><a name="Prerequisites"></a>Prerequisites</h2>
+
+<ul>
+  
+<li>A network accessible Metron REST application</li>
+  
+<li>nodejs v6.9+ (nodejs can be installed on quick dev with <tt>curl --silent --location https://rpm.nodesource.com/setup_6.x | bash - &amp;&amp; yum install -y nodejs</tt>)</li>
+</ul></div>
+<div class="section">
+<h2><a name="Installation"></a>Installation</h2>
+<div class="section">
+<h3><a name="From_Source"></a>From Source</h3>
+
+<ol style="list-style-type: decimal">
+  
+<li>
+<p>Package the application with Maven:</p>
+  
+<div class="source">
+<div class="source">
+<pre>cd metron-interface/metron-config
+mvn clean package
+</pre></div></div></li>
+  
+<li>
+<p>Untar the archive in the $METRON_HOME directory. The directory structure will look like:</p>
+  
+<div class="source">
+<div class="source">
+<pre>bin
+  metron-management-ui
+web
+  expressjs
+    package.json
+    server.js
+  management-ui
+    web assets (html, css, js, ...)
+</pre></div></div></li>
+  
+<li>
+<p>Copy the <tt>$METRON_HOME/bin/metron-management-ui</tt> script to <tt>/etc/init.d/metron-management-ui</tt></p></li>
+  
+<li>
+<p>Install the <a class="externalLink" href="https://expressjs.com/">Express</a> web framework from the <tt>package.json</tt> file in <tt>$METRON_HOME/web/expressjs</tt>:</p>
+  
+<div class="source">
+<div class="source">
+<pre>npm --prefix $METRON_HOME/web/expressjs/ install
+</pre></div></div></li>
+</ol></div>
+<div class="section">
+<h3><a name="From_Package_Manager"></a>From Package Manager</h3>
+
+<ol style="list-style-type: decimal">
+  
+<li>
+<p>Deploy the RPM at <tt>/metron/metron-deployment/packaging/docker/rpm-docker/target/RPMS/noarch/metron-config-$METRON_VERSION-*.noarch.rpm</tt></p></li>
+  
+<li>
+<p>Install the RPM with:</p>
+  
+<div class="source">
+<div class="source">
+<pre>rpm -ih metron-config-$METRON_VERSION-*.noarch.rpm
+</pre></div></div></li>
+  
+<li>
+<p>Install the <a class="externalLink" href="https://expressjs.com/">Express</a> web framework from the <tt>package.json</tt> file in <tt>$METRON_HOME/web/expressjs</tt>:</p>
+  
+<div class="source">
+<div class="source">
+<pre>npm --prefix $METRON_HOME/web/expressjs/ install
+</pre></div></div></li>
+</ol></div></div>
+<div class="section">
+<h2><a name="Configuration"></a>Configuration</h2>
+<p>The Managment UI is configured in the <tt>$METRON_HOME/config/management_ui.yml</tt> file. Create this file and set the values to match your environment:</p>
+
+<div class="source">
+<div class="source">
+<pre>port: port the managment UI will run on
+
+rest:
+  host: REST application host
+  port: REST applciation port
+</pre></div></div></div>
+<div class="section">
+<h2><a name="Usage"></a>Usage</h2>
+<p>After configuration is complete, the Management UI can be managed as a service:</p>
+
+<div class="source">
+<div class="source">
+<pre>service metron-management-ui start
+</pre></div></div>
+<p>The application will be available at <a class="externalLink" href="http://host:4200">http://host:4200</a> assuming the port is set to <tt>4200</tt>. Logs can be found at <tt>/var/log/metron/metron-management-ui.log</tt>.</p></div>
+<div class="section">
+<h2><a name="Development"></a>Development</h2>
+<p>The Management UI can also be started in development mode. This allows changes to web assets to be seen interactively.</p>
+
+<ol style="list-style-type: decimal">
+  
+<li>
+<p>Install the application with dev dependencies:</p>
+  
+<div class="source">
+<div class="source">
+<pre>cd metron-interface/metron-config
+npm install
+</pre></div></div></li>
+  
+<li>
+<p>Start the application:</p>
+  
+<div class="source">
+<div class="source">
+<pre>./scripts/start_dev.sh
+</pre></div></div></li>
+</ol>
+<p>The application will be available at <a class="externalLink" href="http://localhost:4200/">http://localhost:4200/</a>. The REST application url defaults to <tt>http://localhost:8080</tt> but can be changed in the <tt>proxy.conf.json</tt> file.</p></div>
+<div class="section">
+<h2><a name="Testing"></a>Testing</h2>
+
+<ol style="list-style-type: decimal">
+  
+<li>
+<p>Install the application with dev dependencies:</p>
+  
+<div class="source">
+<div class="source">
+<pre>cd metron-interface/metron-config
+npm install
+</pre></div></div></li>
+  
+<li>
+<p>Unit tests can be run with:</p>
+  
+<div class="source">
+<div class="source">
+<pre>npm test
+</pre></div></div></li>
+</ol></div>
+<div class="section">
+<h2><a name="License"></a>License</h2>
+<p>This projects bundles Font Awesome which is available under the SIL Open Font License. See <a class="externalLink" href="http://fontawesome.io/license/">http://fontawesome.io/license/</a> for more details.</p></div>
+                  </div>
+            </div>
+          </div>
+
+    <hr/>
+
+    <footer>
+            <div class="container-fluid">
+              <div class="row span12">Copyright &copy;                    2017
+                        <a href="https://www.apache.org">The Apache Software Foundation</a>.
+            All Rights Reserved.      
+                    
+      </div>
+
+                          
+        
+                </div>
+    </footer>
+  </body>
+</html>

Added: dev/metron/0.4.0-RC4/site-book/metron-interface/metron-rest/index.html
==============================================================================
--- dev/metron/0.4.0-RC4/site-book/metron-interface/metron-rest/index.html (added)
+++ dev/metron/0.4.0-RC4/site-book/metron-interface/metron-rest/index.html Tue Jun 27 18:15:56 2017
@@ -0,0 +1,1996 @@
+<!DOCTYPE html>
+<!--
+ | Generated by Apache Maven Doxia at 2017-06-27
+ | Rendered using Apache Maven Fluido Skin 1.3.0
+-->
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+  <head>
+    <meta charset="UTF-8" />
+    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+    <meta name="Date-Revision-yyyymmdd" content="20170627" />
+    <meta http-equiv="Content-Language" content="en" />
+    <title>Metron &#x2013; Metron REST</title>
+    <link rel="stylesheet" href="../../css/apache-maven-fluido-1.3.0.min.css" />
+    <link rel="stylesheet" href="../../css/site.css" />
+    <link rel="stylesheet" href="../../css/print.css" media="print" />
+
+      
+    <script type="text/javascript" src="../../js/apache-maven-fluido-1.3.0.min.js"></script>
+
+                          
+        
+<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
+          
+            </head>
+        <body class="topBarDisabled">
+          
+                
+                    
+    
+        <div class="container-fluid">
+          <div id="banner">
+        <div class="pull-left">
+                                    <a href="http://metron.apache.org/" id="bannerLeft">
+                                                                                                <img src="../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/>
+                </a>
+                      </div>
+        <div class="pull-right">  </div>
+        <div class="clear"><hr/></div>
+      </div>
+
+      <div id="breadcrumbs">
+        <ul class="breadcrumb">
+                
+                    
+                              <li class="">
+                    <a href="http://www.apache.org" class="externalLink" title="Apache">
+        Apache</a>
+        </li>
+      <li class="divider ">/</li>
+            <li class="">
+                    <a href="http://metron.apache.org/" class="externalLink" title="Metron">
+        Metron</a>
+        </li>
+      <li class="divider ">/</li>
+            <li class="">
+                    <a href="../../index.html" title="Documentation">
+        Documentation</a>
+        </li>
+      <li class="divider ">/</li>
+        <li class="">Metron REST</li>
+        
+                
+                    
+                  <li id="publishDate" class="pull-right">Last Published: 2017-06-27</li> <li class="divider pull-right">|</li>
+              <li id="projectVersion" class="pull-right">Version: 0.4.0</li>
+            
+                            </ul>
+      </div>
+
+            
+      <div class="row-fluid">
+        <div id="leftColumn" class="span3">
+          <div class="well sidebar-nav">
+                
+                    
+                <ul class="nav nav-list">
+                    <li class="nav-header">User Documentation</li>
+                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          
+      <li>
+    
+                          <a href="../../index.html" title="Metron">
+          <i class="icon-chevron-down"></i>
+        Metron</a>
+                    <ul class="nav nav-list">
+                      
+      <li>
+    
+                          <a href="../../Upgrading.html" title="Upgrading">
+          <i class="none"></i>
+        Upgrading</a>
+            </li>
+                                                                                                                                                      
+      <li>
+    
+                          <a href="../../metron-analytics/index.html" title="Analytics">
+          <i class="icon-chevron-right"></i>
+        Analytics</a>
+                  </li>
+                                                                                                                                                                                                                                                                                                                                                                                    
+      <li>
+    
+                          <a href="../../metron-deployment/index.html" title="Deployment">
+          <i class="icon-chevron-right"></i>
+        Deployment</a>
+                  </li>
+                      
+      <li>
+    
+                          <a href="../../metron-docker/index.html" title="Docker">
+          <i class="none"></i>
+        Docker</a>
+            </li>
+                      
+      <li>
+    
+                          <a href="../../metron-interface/metron-config/index.html" title="Config">
+          <i class="none"></i>
+        Config</a>
+            </li>
+                      
+      <li class="active">
+    
+            <a href="#"><i class="none"></i>Rest</a>
+          </li>
+                                                                                                                                                                                                                                                
+      <li>
+    
+                          <a href="../../metron-platform/index.html" title="Platform">
+          <i class="icon-chevron-right"></i>
+        Platform</a>
+                  </li>
+                                                                                                            
+      <li>
+    
+                          <a href="../../metron-sensors/index.html" title="Sensors">
+          <i class="icon-chevron-right"></i>
+        Sensors</a>
+                  </li>
+              </ul>
+        </li>
+            </ul>
+                
+                    
+                
+          <hr class="divider" />
+
+           <div id="poweredBy">
+                            <div class="clear"></div>
+                            <div class="clear"></div>
+                            <div class="clear"></div>
+                             <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
+        <img class="builtBy" alt="Built by Maven" src="../../images/logos/maven-feather.png" />
+      </a>
+                  </div>
+          </div>
+        </div>
+        
+                
+        <div id="bodyColumn"  class="span9" >
+                                  
+            <h1>Metron REST</h1>
+<p><a name="Metron_REST"></a></p>
+<p>This module provides a RESTful API for interacting with Metron.</p>
+<div class="section">
+<h2><a name="Prerequisites"></a>Prerequisites</h2>
+
+<ul>
+  
+<li>A running Metron cluster</li>
+  
+<li>Java 8 installed</li>
+  
+<li>Storm CLI and Metron topology scripts (start_parser_topology.sh, start_enrichment_topology.sh, start_elasticsearch_topology.sh) installed</li>
+  
+<li>A relational database</li>
+</ul></div>
+<div class="section">
+<h2><a name="Installation"></a>Installation</h2>
+<div class="section">
+<h3><a name="From_Source"></a>From Source</h3>
+
+<ol style="list-style-type: decimal">
+  
+<li>
+<p>Package the application with Maven:</p>
+  
+<div class="source">
+<div class="source">
+<pre>mvn clean package
+</pre></div></div></li>
+  
+<li>
+<p>Untar the archive in the $METRON_HOME directory. The directory structure will look like:</p>
+  
+<div class="source">
+<div class="source">
+<pre>config
+  rest_application.yml
+bin
+  metron-rest
+lib
+  metron-rest-$METRON_VERSION.jar
+</pre></div></div></li>
+  
+<li>
+<p>Copy the <tt>$METRON_HOME/bin/metron-rest</tt> script to <tt>/etc/init.d/metron-rest</tt></p></li>
+</ol></div>
+<div class="section">
+<h3><a name="From_Package_Manager"></a>From Package Manager</h3>
+
+<ol style="list-style-type: decimal">
+  
+<li>
+<p>Deploy the RPM at <tt>/metron/metron-deployment/packaging/docker/rpm-docker/target/RPMS/noarch/metron-rest-$METRON_VERSION-*.noarch.rpm</tt></p></li>
+  
+<li>
+<p>Install the RPM with:</p>
+  
+<div class="source">
+<div class="source">
+<pre>rpm -ih metron-rest-$METRON_VERSION-*.noarch.rpm
+</pre></div></div></li>
+</ol></div></div>
+<div class="section">
+<h2><a name="Configuration"></a>Configuration</h2>
+<p>The REST application depends on several configuration parameters:</p>
+<div class="section">
+<h3><a name="REQUIRED"></a>REQUIRED</h3>
+<p>No optional parameter has a default.</p>
+
+<table border="0" class="table table-striped">
+  <thead>
+    
+<tr class="a">
+      
+<th>Environment Variable </th>
+      
+<th>Description</th>
+    </tr>
+  </thead>
+  <tbody>
+    
+<tr class="b">
+      
+<td>METRON_JDBC_DRIVER </td>
+      
+<td>JDBC driver class</td>
+    </tr>
+    
+<tr class="a">
+      
+<td>METRON_JDBC_URL </td>
+      
+<td>JDBC url</td>
+    </tr>
+    
+<tr class="b">
+      
+<td>METRON_JDBC_USERNAME </td>
+      
+<td>JDBC username</td>
+    </tr>
+    
+<tr class="a">
+      
+<td>METRON_JDBC_PLATFORM </td>
+      
+<td>JDBC platform (one of h2, mysql, postgres, oracle</td>
+    </tr>
+    
+<tr class="b">
+      
+<td>ZOOKEEPER </td>
+      
+<td>Zookeeper quorum (ex. node1:2181,node2:2181)</td>
+    </tr>
+    
+<tr class="a">
+      
+<td>BROKERLIST </td>
+      
+<td>Kafka Broker list (ex. node1:6667,node2:6667)</td>
+    </tr>
+    
+<tr class="b">
+      
+<td>HDFS_URL </td>
+      
+<td>HDFS url or <tt>fs.defaultFS</tt> Hadoop setting (ex. <a class="externalLink" href="hdfs://node1:8020">hdfs://node1:8020</a>)</td>
+    </tr>
+  </tbody>
+</table></div>
+<div class="section">
+<h3><a name="Optional_-_With_Defaults"></a>Optional - With Defaults</h3>
+
+<table border="0" class="table table-striped">
+  <thead>
+    
+<tr class="a">
+      
+<th>Environment Variable </th>
+      
+<th>Description </th>
+      
+<th>Required </th>
+      
+<th>Default</th>
+    </tr>
+  </thead>
+  <tbody>
+    
+<tr class="b">
+      
+<td>METRON_USER </td>
+      
+<td>Run the application as this user </td>
+      
+<td>Optional </td>
+      
+<td>metron</td>
+    </tr>
+    
+<tr class="a">
+      
+<td>METRON_LOG_DIR </td>
+      
+<td>Directory where the log file is written </td>
+      
+<td>Optional </td>
+      
+<td>/var/log/metron/</td>
+    </tr>
+    
+<tr class="b">
+      
+<td>METRON_PID_DIR </td>
+      
+<td>Directory where the pid file is written </td>
+      
+<td>Optional </td>
+      
+<td>/var/run/metron/</td>
+    </tr>
+    
+<tr class="a">
+      
+<td>METRON_REST_PORT </td>
+      
+<td>REST application port </td>
+      
+<td>Optional </td>
+      
+<td>8082</td>
+    </tr>
+    
+<tr class="b">
+      
+<td>METRON_JDBC_CLIENT_PATH </td>
+      
+<td>Path to JDBC client jar </td>
+      
+<td>Optional </td>
+      
+<td>H2 is bundled</td>
+    </tr>
+    
+<tr class="a">
+      
+<td>METRON_TEMP_GROK_PATH </td>
+      
+<td>Temporary directory used to test grok statements </td>
+      
+<td>Optional </td>
+      
+<td>./patterns/temp</td>
+    </tr>
+    
+<tr class="b">
+      
+<td>METRON_DEFAULT_GROK_PATH </td>
+      
+<td>Defaults HDFS directory used to store grok statements </td>
+      
+<td>Optional </td>
+      
+<td>/apps/metron/patterns</td>
+    </tr>
+    
+<tr class="a">
+      
+<td>SECURITY_ENABLED </td>
+      
+<td>Enables Kerberos support </td>
+      
+<td>Optional </td>
+      
+<td>false</td>
+    </tr>
+  </tbody>
+</table></div>
+<div class="section">
+<h3><a name="Optional_-_Blank_Defaults"></a>Optional - Blank Defaults</h3>
+
+<table border="0" class="table table-striped">
+  <thead>
+    
+<tr class="a">
+      
+<th>Environment Variable </th>
+      
+<th>Description </th>
+      
+<th>Required</th>
+    </tr>
+  </thead>
+  <tbody>
+    
+<tr class="b">
+      
+<td>METRON_JVMFLAGS </td>
+      
+<td>JVM flags added to the start command </td>
+      
+<td>Optional</td>
+    </tr>
+    
+<tr class="a">
+      
+<td>METRON_SPRING_PROFILES_ACTIVE </td>
+      
+<td>Active Spring profiles (see <a href="#spring-profiles">below</a>) </td>
+      
+<td>Optional</td>
+    </tr>
+    
+<tr class="b">
+      
+<td>METRON_SPRING_OPTIONS </td>
+      
+<td>Additional Spring input parameters </td>
+      
+<td>Optional</td>
+    </tr>
+    
+<tr class="a">
+      
+<td>METRON_PRINCIPAL_NAME </td>
+      
+<td>Kerberos principal for the metron user </td>
+      
+<td>Optional</td>
+    </tr>
+    
+<tr class="b">
+      
+<td>METRON_SERVICE_KEYTAB </td>
+      
+<td>Path to the Kerberos keytab for the metron user </td>
+      
+<td>Optional</td>
+    </tr>
+  </tbody>
+</table>
+<p>These are set in the <tt>/etc/sysconfig/metron</tt> file.</p></div></div>
+<div class="section">
+<h2><a name="Database_setup"></a>Database setup</h2>
+<p>The REST application persists data in a relational database and requires a dedicated database user and database (see <a class="externalLink" href="https://docs.spring.io/spring-boot/docs/current/reference/html/boot-features-sql.html">https://docs.spring.io/spring-boot/docs/current/reference/html/boot-features-sql.html</a> for more detail).</p>
+<div class="section">
+<h3><a name="Development"></a>Development</h3>
+<p>The REST application comes with embedded database support for development purposes (<a class="externalLink" href="https://docs.spring.io/spring-boot/docs/current/reference/html/boot-features-sql.html#boot-features-embedded-database-support)">https://docs.spring.io/spring-boot/docs/current/reference/html/boot-features-sql.html#boot-features-embedded-database-support)</a>.</p>
+<p>For example, edit these variables in <tt>/etc/sysconfig/metron</tt> before starting the application to configure H2:</p>
+
+<div class="source">
+<div class="source">
+<pre>METRON_JDBC_DRIVER=&quot;org.h2.Driver&quot;
+METRON_JDBC_URL=&quot;jdbc:h2:file:~/metrondb&quot;
+METRON_JDBC_USERNAME=&quot;root&quot;
+METRON_JDBC_PASSWORD='root&quot;
+METRON_JDBC_PLATFORM=&quot;h2&quot;
+</pre></div></div></div>
+<div class="section">
+<h3><a name="Production"></a>Production</h3>
+<p>The REST application should be configured with a production-grade database outside of development.</p>
+<p>For example, the following configures the application for MySQL:</p>
+
+<ol style="list-style-type: decimal">
+  
+<li>
+<p>Install MySQL if not already available (this example uses version 5.7, installation instructions can be found <a class="externalLink" href="https://dev.mysql.com/doc/refman/5.7/en/linux-installation-yum-repo.html">here</a>)</p></li>
+  
+<li>
+<p>Create a metron user and REST database and permission the user for that database:</p>
+  
+<div class="source">
+<div class="source">
+<pre>CREATE USER 'metron'@'node1' IDENTIFIED BY 'Myp@ssw0rd';
+CREATE DATABASE IF NOT EXISTS metronrest;
+GRANT ALL PRIVILEGES ON metronrest.* TO 'metron'@'node1';
+</pre></div></div></li>
+  
+<li>
+<p>Install the MySQL JDBC client onto the REST application host and configurate the METRON_JDBC_CLIENT_PATH variable:</p>
+  
+<div class="source">
+<div class="source">
+<pre>cd $METRON_HOME/lib
+wget https://dev.mysql.com/get/Downloads/Connector-J/mysql-connector-java-5.1.41.tar.gz
+tar xf mysql-connector-java-5.1.41.tar.gz
+</pre></div></div></li>
+  
+<li>
+<p>Edit these variables in <tt>/etc/sysconfig/metron</tt> to configure the REST application for MySQL:</p>
+  
+<div class="source">
+<div class="source">
+<pre>METRON_JDBC_DRIVER=&quot;com.mysql.jdbc.Driver&quot;
+METRON_JDBC_URL=&quot;jdbc:mysql://mysql_host:3306/metronrest&quot;
+METRON_JDBC_USERNAME=&quot;metron&quot;
+METRON_JDBC_PLATFORM=&quot;mysql&quot;
+METRON_JDBC_CLIENT_PATH=$METRON_HOME/lib/mysql-connector-java-5.1.41/mysql-connector-java-5.1.41-bin.jar
+</pre></div></div></li>
+</ol></div></div>
+<div class="section">
+<h2><a name="Usage"></a>Usage</h2>
+<p>After configuration is complete, the REST application can be managed as a service:</p>
+
+<div class="source">
+<div class="source">
+<pre>service metron-rest start
+</pre></div></div>
+<p>If a production database is configured, the JDBC password should be passed in as the first argument on startup:</p>
+
+<div class="source">
+<div class="source">
+<pre>service metron-rest start Myp@ssw0rd
+</pre></div></div>
+<p>The REST application can be accessed with the Swagger UI at <a class="externalLink" href="http://host:port/swagger-ui.html#/">http://host:port/swagger-ui.html#/</a>. The default port is 8082.</p></div>
+<div class="section">
+<h2><a name="Security"></a>Security</h2>
+<div class="section">
+<h3><a name="Authentication"></a>Authentication</h3>
+<p>The metron-rest module uses <a class="externalLink" href="http://projects.spring.io/spring-security/">Spring Security</a> for authentication and stores user credentials in the relational database configured above. The required tables are created automatically the first time the application is started so that should be done first. For example (continuing the MySQL example above), users can be added by connecting to MySQL and running:</p>
+
+<div class="source">
+<div class="source">
+<pre>use metronrest;
+insert into users (username, password, enabled) values ('your_username','your_password',1);
+insert into authorities (username, authority) values ('your_username', 'ROLE_USER');
+</pre></div></div></div>
+<div class="section">
+<h3><a name="Kerberos"></a>Kerberos</h3>
+<p>Metron REST can be configured for a cluster with Kerberos enabled. A client JAAS file is required for Kafka and Zookeeper and a Kerberos keytab for the metron user principal is required for all other services. Configure these settings in the <tt>/etc/sysconfig/metron</tt> file:</p>
+
+<div class="source">
+<div class="source">
+<pre>SECURITY_ENABLED=true
+METRON_JVMFLAGS=&quot;-Djava.security.auth.login.config=$METRON_HOME/client_jaas.conf&quot;
+METRON_PRINCIPAL_NAME=&quot;metron@EXAMPLE.COM&quot;
+METRON_SERVICE_KEYTAB=&quot;/etc/security/keytabs/metron.keytab&quot;
+</pre></div></div></div></div>
+<div class="section">
+<h2><a name="Spring_Profiles"></a>Spring Profiles</h2>
+<p>The REST application comes with a few <a class="externalLink" href="http://docs.spring.io/autorepo/docs/spring-boot/current/reference/html/boot-features-profiles.html">Spring Profiles</a> to aid in testing and development.</p>
+
+<table border="0" class="table table-striped">
+  <thead>
+    
+<tr class="a">
+      
+<th>Profile </th>
+      
+<th>Description </th>
+    </tr>
+  </thead>
+  <tbody>
+    
+<tr class="b">
+      
+<td>test </td>
+      
+<td>sets variables to in-memory services, only used for integration testing </td>
+    </tr>
+    
+<tr class="a">
+      
+<td>dev </td>
+      
+<td>adds a test user to the database with credentials <tt>user/password</tt> </td>
+    </tr>
+    
+<tr class="b">
+      
+<td>vagrant </td>
+      
+<td>sets configuration variables to match the Metron vagrant environment </td>
+    </tr>
+    
+<tr class="a">
+      
+<td>docker </td>
+      
+<td>sets configuration variables to match the Metron dcoker environment </td>
+    </tr>
+  </tbody>
+</table>
+<p>Setting active profiles is done with the METRON_SPRING_PROFILES_ACTIVE variable. For example, set this variable in <tt>/etc/sysconfig/metron</tt> to configure the REST application for the Vagrant environment and add a test user:</p>
+
+<div class="source">
+<div class="source">
+<pre>METRON_SPRING_PROFILES_ACTIVE=&quot;vagrant,dev&quot;
+</pre></div></div></div>
+<div class="section">
+<h2><a name="API"></a>API</h2>
+<p>Request and Response objects are JSON formatted. The JSON schemas are available in the Swagger UI.</p>
+
+<table border="0" class="table table-striped">
+  <thead>
+    
+<tr class="a">
+      
+<th> </th>
+    </tr>
+  </thead>
+  <tbody>
+    
+<tr class="b">
+      
+<td><a href="#GET_apiv1globalconfig"> <tt>GET /api/v1/global/config</tt></a></td>
+    </tr>
+    
+<tr class="a">
+      
+<td><a href="#DELETE_apiv1globalconfig"> <tt>DELETE /api/v1/global/config</tt></a></td>
+    </tr>
+    
+<tr class="b">
+      
+<td><a href="#POST_apiv1globalconfig"> <tt>POST /api/v1/global/config</tt></a></td>
+    </tr>
+    
+<tr class="a">
+      
+<td><a href="#GET_apiv1grokgetstatement"> <tt>GET /api/v1/grok/get/statement</tt></a></td>
+    </tr>
+    
+<tr class="b">
+      
+<td><a href="#GET_apiv1groklist"> <tt>GET /api/v1/grok/list</tt></a></td>
+    </tr>
+    
+<tr class="a">
+      
+<td><a href="#POST_apiv1grokvalidate"> <tt>POST /api/v1/grok/validate</tt></a></td>
+    </tr>
+    
+<tr class="b">
+      
+<td><a href="#POST_apiv1hdfs"> <tt>POST /api/v1/hdfs</tt></a></td>
+    </tr>
+    
+<tr class="a">
+      
+<td><a href="#GET_apiv1hdfs"> <tt>GET /api/v1/hdfs</tt></a></td>
+    </tr>
+    
+<tr class="b">
+      
+<td><a href="#DELETE_apiv1hdfs"> <tt>DELETE /api/v1/hdfs</tt></a></td>
+    </tr>
+    
+<tr class="a">
+      
+<td><a href="#GET_apiv1hdfslist"> <tt>GET /api/v1/hdfs/list</tt></a></td>
+    </tr>
+    
+<tr class="b">
+      
+<td><a href="#GET_apiv1kafkatopic"> <tt>GET /api/v1/kafka/topic</tt></a></td>
+    </tr>
+    
+<tr class="a">
+      
+<td><a href="#POST_apiv1kafkatopic"> <tt>POST /api/v1/kafka/topic</tt></a></td>
+    </tr>
+    
+<tr class="b">
+      
+<td><a href="#GET_apiv1kafkatopicname"> <tt>GET /api/v1/kafka/topic/{name}</tt></a></td>
+    </tr>
+    
+<tr class="a">
+      
+<td><a href="#DELETE_apiv1kafkatopicname"> <tt>DELETE /api/v1/kafka/topic/{name}</tt></a></td>
+    </tr>
+    
+<tr class="b">
+      
+<td><a href="#GET_apiv1kafkatopicnamesample"> <tt>GET /api/v1/kafka/topic/{name}/sample</tt></a></td>
+    </tr>
+    
+<tr class="a">
+      
+<td><a href="#GET_apiv1sensorenrichmentconfig"> <tt>GET /api/v1/sensor/enrichment/config</tt></a></td>
+    </tr>
+    
+<tr class="b">
+      
+<td><a href="#GET_apiv1sensorenrichmentconfiglistavailableenrichments"> <tt>GET /api/v1/sensor/enrichment/config/list/available/enrichments</tt></a></td>
+    </tr>
+    
+<tr class="a">
+      
+<td><a href="#GET_apiv1sensorenrichmentconfiglistavailablethreattriageaggregators"> <tt>GET /api/v1/sensor/enrichment/config/list/available/threat/triage/aggregators</tt></a></td>
+    </tr>
+    
+<tr class="b">
+      
+<td><a href="#DELETE_apiv1sensorenrichmentconfigname"> <tt>DELETE /api/v1/sensor/enrichment/config/{name}</tt></a></td>
+    </tr>
+    
+<tr class="a">
+      
+<td><a href="#POST_apiv1sensorenrichmentconfigname"> <tt>POST /api/v1/sensor/enrichment/config/{name}</tt></a></td>
+    </tr>
+    
+<tr class="b">
+      
+<td><a href="#GET_apiv1sensorenrichmentconfigname"> <tt>GET /api/v1/sensor/enrichment/config/{name}</tt></a></td>
+    </tr>
+    
+<tr class="a">
+      
+<td><a href="#GET_apiv1sensorindexingconfig"> <tt>GET /api/v1/sensor/indexing/config</tt></a></td>
+    </tr>
+    
+<tr class="b">
+      
+<td><a href="#DELETE_apiv1sensorindexingconfigname"> <tt>DELETE /api/v1/sensor/indexing/config/{name}</tt></a></td>
+    </tr>
+    
+<tr class="a">
+      
+<td><a href="#POST_apiv1sensorindexingconfigname"> <tt>POST /api/v1/sensor/indexing/config/{name}</tt></a></td>
+    </tr>
+    
+<tr class="b">
+      
+<td><a href="#GET_apiv1sensorindexingconfigname"> <tt>GET /api/v1/sensor/indexing/config/{name}</tt></a></td>
+    </tr>
+    
+<tr class="a">
+      
+<td><a href="#POST_apiv1sensorparserconfig"> <tt>POST /api/v1/sensor/parser/config</tt></a></td>
+    </tr>
+    
+<tr class="b">
+      
+<td><a href="#GET_apiv1sensorparserconfig"> <tt>GET /api/v1/sensor/parser/config</tt></a></td>
+    </tr>
+    
+<tr class="a">
+      
+<td><a href="#GET_apiv1sensorparserconfiglistavailable"> <tt>GET /api/v1/sensor/parser/config/list/available</tt></a></td>
+    </tr>
+    
+<tr class="b">
+      
+<td><a href="#POST_apiv1sensorparserconfigparseMessage"> <tt>POST /api/v1/sensor/parser/config/parseMessage</tt></a></td>
+    </tr>
+    
+<tr class="a">
+      
+<td><a href="#GET_apiv1sensorparserconfigreloadavailable"> <tt>GET /api/v1/sensor/parser/config/reload/available</tt></a></td>
+    </tr>
+    
+<tr class="b">
+      
+<td><a href="#DELETE_apiv1sensorparserconfigname"> <tt>DELETE /api/v1/sensor/parser/config/{name}</tt></a></td>
+    </tr>
+    
+<tr class="a">
+      
+<td><a href="#GET_apiv1sensorparserconfigname"> <tt>GET /api/v1/sensor/parser/config/{name}</tt></a></td>
+    </tr>
+    
+<tr class="b">
+      
+<td><a href="#POST_apiv1stellarapplytransformations"> <tt>POST /api/v1/stellar/apply/transformations</tt></a></td>
+    </tr>
+    
+<tr class="a">
+      
+<td><a href="#GET_apiv1stellarlist"> <tt>GET /api/v1/stellar/list</tt></a></td>
+    </tr>
+    
+<tr class="b">
+      
+<td><a href="#GET_apiv1stellarlistfunctions"> <tt>GET /api/v1/stellar/list/functions</tt></a></td>
+    </tr>
+    
+<tr class="a">
+      
+<td><a href="#GET_apiv1stellarlistsimplefunctions"> <tt>GET /api/v1/stellar/list/simple/functions</tt></a></td>
+    </tr>
+    
+<tr class="b">
+      
+<td><a href="#POST_apiv1stellarvalidaterules"> <tt>POST /api/v1/stellar/validate/rules</tt></a></td>
+    </tr>
+    
+<tr class="a">
+      
+<td><a href="#GET_apiv1storm"> <tt>GET /api/v1/storm</tt></a></td>
+    </tr>
+    
+<tr class="b">
+      
+<td><a href="#GET_apiv1stormclientstatus"> <tt>GET /api/v1/storm/client/status</tt></a></td>
+    </tr>
+    
+<tr class="a">
+      
+<td><a href="#GET_apiv1stormenrichment"> <tt>GET /api/v1/storm/enrichment</tt></a></td>
+    </tr>
+    
+<tr class="b">
+      
+<td><a href="#GET_apiv1stormenrichmentactivate"> <tt>GET /api/v1/storm/enrichment/activate</tt></a></td>
+    </tr>
+    
+<tr class="a">
+      
+<td><a href="#GET_apiv1stormenrichmentdeactivate"> <tt>GET /api/v1/storm/enrichment/deactivate</tt></a></td>
+    </tr>
+    
+<tr class="b">
+      
+<td><a href="#GET_apiv1stormenrichmentstart"> <tt>GET /api/v1/storm/enrichment/start</tt></a></td>
+    </tr>
+    
+<tr class="a">
+      
+<td><a href="#GET_apiv1stormenrichmentstop"> <tt>GET /api/v1/storm/enrichment/stop</tt></a></td>
+    </tr>
+    
+<tr class="b">
+      
+<td><a href="#GET_apiv1stormindexing"> <tt>GET /api/v1/storm/indexing</tt></a></td>
+    </tr>
+    
+<tr class="a">
+      
+<td><a href="#GET_apiv1stormindexingactivate"> <tt>GET /api/v1/storm/indexing/activate</tt></a></td>
+    </tr>
+    
+<tr class="b">
+      
+<td><a href="#GET_apiv1stormindexingdeactivate"> <tt>GET /api/v1/storm/indexing/deactivate</tt></a></td>
+    </tr>
+    
+<tr class="a">
+      
+<td><a href="#GET_apiv1stormindexingstart"> <tt>GET /api/v1/storm/indexing/start</tt></a></td>
+    </tr>
+    
+<tr class="b">
+      
+<td><a href="#GET_apiv1stormindexingstop"> <tt>GET /api/v1/storm/indexing/stop</tt></a></td>
+    </tr>
+    
+<tr class="a">
+      
+<td><a href="#GET_apiv1stormparseractivatename"> <tt>GET /api/v1/storm/parser/activate/{name}</tt></a></td>
+    </tr>
+    
+<tr class="b">
+      
+<td><a href="#GET_apiv1stormparserdeactivatename"> <tt>GET /api/v1/storm/parser/deactivate/{name}</tt></a></td>
+    </tr>
+    
+<tr class="a">
+      
+<td><a href="#GET_apiv1stormparserstartname"> <tt>GET /api/v1/storm/parser/start/{name}</tt></a></td>
+    </tr>
+    
+<tr class="b">
+      
+<td><a href="#GET_apiv1stormparserstopname"> <tt>GET /api/v1/storm/parser/stop/{name}</tt></a></td>
+    </tr>
+    
+<tr class="a">
+      
+<td><a href="#GET_apiv1stormname"> <tt>GET /api/v1/storm/{name}</tt></a></td>
+    </tr>
+    
+<tr class="b">
+      
+<td><a href="#GET_apiv1user"> <tt>GET /api/v1/user</tt></a></td>
+    </tr>
+  </tbody>
+</table>
+<div class="section">
+<h3><a name="GET_apiv1globalconfig"></a><tt>GET /api/v1/global/config</tt></h3>
+
+<ul>
+  
+<li>Description: Retrieves the current Global Config from Zookeeper</li>
+  
+<li>Returns:
+  
+<ul>
+    
+<li>200 - Returns current Global Config JSON in Zookeeper</li>
+    
+<li>404 - Global Config JSON was not found in Zookeeper</li>
+  </ul></li>
+</ul></div>
+<div class="section">
+<h3><a name="DELETE_apiv1globalconfig"></a><tt>DELETE /api/v1/global/config</tt></h3>
+
+<ul>
+  
+<li>Description: Deletes the current Global Config from Zookeeper</li>
+  
+<li>Returns:
+  
+<ul>
+    
+<li>200 - Global Config JSON was deleted</li>
+    
+<li>404 - Global Config JSON was not found in Zookeeper</li>
+  </ul></li>
+</ul></div>
+<div class="section">
+<h3><a name="POST_apiv1globalconfig"></a><tt>POST /api/v1/global/config</tt></h3>
+
+<ul>
+  
+<li>Description: Creates or updates the Global Config in Zookeeper</li>
+  
+<li>Input:
+  
+<ul>
+    
+<li>globalConfig - The Global Config JSON to be saved</li>
+  </ul></li>
+  
+<li>Returns:
+  
+<ul>
+    
+<li>200 - Global Config updated. Returns saved Global Config JSON</li>
+    
+<li>201 - Global Config created. Returns saved Global Config JSON</li>
+  </ul></li>
+</ul></div>
+<div class="section">
+<h3><a name="GET_apiv1grokgetstatement"></a><tt>GET /api/v1/grok/get/statement</tt></h3>
+
+<ul>
+  
+<li>Description: Retrieves a Grok statement from the classpath</li>
+  
+<li>Input:
+  
+<ul>
+    
+<li>path - Path to classpath resource</li>
+  </ul></li>
+  
+<li>Returns:
+  
+<ul>
+    
+<li>200 - Grok statement</li>
+  </ul></li>
+</ul></div>
+<div class="section">
+<h3><a name="GET_apiv1groklist"></a><tt>GET /api/v1/grok/list</tt></h3>
+
+<ul>
+  
+<li>Description: Lists the common Grok statements available in Metron</li>
+  
+<li>Returns:
+  
+<ul>
+    
+<li>200 - JSON object containing pattern label/Grok statements key value pairs</li>
+  </ul></li>
+</ul></div>
+<div class="section">
+<h3><a name="POST_apiv1grokvalidate"></a><tt>POST /api/v1/grok/validate</tt></h3>
+
+<ul>
+  
+<li>Description: Applies a Grok statement to a sample message</li>
+  
+<li>Input:
+  
+<ul>
+    
+<li>grokValidation - Object containing Grok statement and sample message</li>
+  </ul></li>
+  
+<li>Returns:
+  
+<ul>
+    
+<li>200 - JSON results</li>
+  </ul></li>
+</ul></div>
+<div class="section">
+<h3><a name="POST_apiv1hdfs"></a><tt>POST /api/v1/hdfs</tt></h3>
+
+<ul>
+  
+<li>Description: Writes contents to an HDFS file. Warning: this will overwrite the contents of a file if it already exists.</li>
+  
+<li>Input:
+  
+<ul>
+    
+<li>path - Path to HDFS file</li>
+    
+<li>contents - File contents</li>
+  </ul></li>
+  
+<li>Returns:
+  
+<ul>
+    
+<li>200 - Contents were written</li>
+  </ul></li>
+</ul></div>
+<div class="section">
+<h3><a name="GET_apiv1hdfs"></a><tt>GET /api/v1/hdfs</tt></h3>
+
+<ul>
+  
+<li>Description: Reads a file from HDFS and returns the contents</li>
+  
+<li>Input:
+  
+<ul>
+    
+<li>path - Path to HDFS file</li>
+  </ul></li>
+  
+<li>Returns:
+  
+<ul>
+    
+<li>200 - Returns file contents</li>
+  </ul></li>
+</ul></div>
+<div class="section">
+<h3><a name="DELETE_apiv1hdfs"></a><tt>DELETE /api/v1/hdfs</tt></h3>
+
+<ul>
+  
+<li>Description: Deletes a file from HDFS</li>
+  
+<li>Input:
+  
+<ul>
+    
+<li>path - Path to HDFS file</li>
+    
+<li>recursive - Delete files recursively</li>
+  </ul></li>
+  
+<li>Returns:
+  
+<ul>
+    
+<li>200 - File was deleted</li>
+    
+<li>404 - File was not found in HDFS</li>
+  </ul></li>
+</ul></div>
+<div class="section">
+<h3><a name="GET_apiv1hdfslist"></a><tt>GET /api/v1/hdfs/list</tt></h3>
+
+<ul>
+  
+<li>Description: Lists an HDFS directory</li>
+  
+<li>Input:
+  
+<ul>
+    
+<li>path - Path to HDFS directory</li>
+  </ul></li>
+  
+<li>Returns:
+  
+<ul>
+    
+<li>200 - HDFS directory list</li>
+  </ul></li>
+</ul></div>
+<div class="section">
+<h3><a name="GET_apiv1kafkatopic"></a><tt>GET /api/v1/kafka/topic</tt></h3>
+
+<ul>
+  
+<li>Description: Retrieves all Kafka topics</li>
+  
+<li>Returns:
+  
+<ul>
+    
+<li>200 - Returns a list of all Kafka topics</li>
+  </ul></li>
+</ul></div>
+<div class="section">
+<h3><a name="POST_apiv1kafkatopic"></a><tt>POST /api/v1/kafka/topic</tt></h3>
+
+<ul>
+  
+<li>Description: Creates a new Kafka topic</li>
+  
+<li>Input:
+  
+<ul>
+    
+<li>topic - Kafka topic</li>
+  </ul></li>
+  
+<li>Returns:
+  
+<ul>
+    
+<li>200 - Returns saved Kafka topic</li>
+  </ul></li>
+</ul></div>
+<div class="section">
+<h3><a name="GET_apiv1kafkatopicname"></a><tt>GET /api/v1/kafka/topic/{name}</tt></h3>
+
+<ul>
+  
+<li>Description: Retrieves a Kafka topic</li>
+  
+<li>Input:
+  
+<ul>
+    
+<li>name - Kafka topic name</li>
+  </ul></li>
+  
+<li>Returns:
+  
+<ul>
+    
+<li>200 - Returns Kafka topic</li>
+    
+<li>404 - Kafka topic is missing</li>
+  </ul></li>
+</ul></div>
+<div class="section">
+<h3><a name="DELETE_apiv1kafkatopicname"></a><tt>DELETE /api/v1/kafka/topic/{name}</tt></h3>
+
+<ul>
+  
+<li>Description: Deletes a Kafka topic</li>
+  
+<li>Input:
+  
+<ul>
+    
+<li>name - Kafka topic name</li>
+  </ul></li>
+  
+<li>Returns:
+  
+<ul>
+    
+<li>200 - Kafka topic was deleted</li>
+    
+<li>404 - Kafka topic is missing</li>
+  </ul></li>
+</ul></div>
+<div class="section">
+<h3><a name="GET_apiv1kafkatopicnamesample"></a><tt>GET /api/v1/kafka/topic/{name}/sample</tt></h3>
+
+<ul>
+  
+<li>Description: Retrieves a sample message from a Kafka topic using the most recent offset</li>
+  
+<li>Input:
+  
+<ul>
+    
+<li>name - Kafka topic name</li>
+  </ul></li>
+  
+<li>Returns:
+  
+<ul>
+    
+<li>200 - Returns sample message</li>
+    
+<li>404 - Either Kafka topic is missing or contains no messages</li>
+  </ul></li>
+</ul></div>
+<div class="section">
+<h3><a name="GET_apiv1sensorenrichmentconfig"></a><tt>GET /api/v1/sensor/enrichment/config</tt></h3>
+
+<ul>
+  
+<li>Description: Retrieves all SensorEnrichmentConfigs from Zookeeper</li>
+  
+<li>Returns:
+  
+<ul>
+    
+<li>200 - Returns all SensorEnrichmentConfigs</li>
+  </ul></li>
+</ul></div>
+<div class="section">
+<h3><a name="GET_apiv1sensorenrichmentconfiglistavailableenrichments"></a><tt>GET /api/v1/sensor/enrichment/config/list/available/enrichments</tt></h3>
+
+<ul>
+  
+<li>Description: Lists the available enrichments</li>
+  
+<li>Returns:
+  
+<ul>
+    
+<li>200 - Returns a list of available enrichments</li>
+  </ul></li>
+</ul></div>
+<div class="section">
+<h3><a name="GET_apiv1sensorenrichmentconfiglistavailablethreattriageaggregators"></a><tt>GET /api/v1/sensor/enrichment/config/list/available/threat/triage/aggregators</tt></h3>
+
+<ul>
+  
+<li>Description: Lists the available threat triage aggregators</li>
+  
+<li>Returns:
+  
+<ul>
+    
+<li>200 - Returns a list of available threat triage aggregators</li>
+  </ul></li>
+</ul></div>
+<div class="section">
+<h3><a name="DELETE_apiv1sensorenrichmentconfigname"></a><tt>DELETE /api/v1/sensor/enrichment/config/{name}</tt></h3>
+
+<ul>
+  
+<li>Description: Deletes a SensorEnrichmentConfig from Zookeeper</li>
+  
+<li>Input:
+  
+<ul>
+    
+<li>name - SensorEnrichmentConfig name</li>
+  </ul></li>
+  
+<li>Returns:
+  
+<ul>
+    
+<li>200 - SensorEnrichmentConfig was deleted</li>
+    
+<li>404 - SensorEnrichmentConfig is missing</li>
+  </ul></li>
+</ul></div>
+<div class="section">
+<h3><a name="POST_apiv1sensorenrichmentconfigname"></a><tt>POST /api/v1/sensor/enrichment/config/{name}</tt></h3>
+
+<ul>
+  
+<li>Description: Updates or creates a SensorEnrichmentConfig in Zookeeper</li>
+  
+<li>Input:
+  
+<ul>
+    
+<li>sensorEnrichmentConfig - SensorEnrichmentConfig</li>
+    
+<li>name - SensorEnrichmentConfig name</li>
+  </ul></li>
+  
+<li>Returns:
+  
+<ul>
+    
+<li>200 - SensorEnrichmentConfig updated. Returns saved SensorEnrichmentConfig</li>
+    
+<li>201 - SensorEnrichmentConfig created. Returns saved SensorEnrichmentConfig</li>
+  </ul></li>
+</ul></div>
+<div class="section">
+<h3><a name="GET_apiv1sensorenrichmentconfigname"></a><tt>GET /api/v1/sensor/enrichment/config/{name}</tt></h3>
+
+<ul>
+  
+<li>Description: Retrieves a SensorEnrichmentConfig from Zookeeper</li>
+  
+<li>Input:
+  
+<ul>
+    
+<li>name - SensorEnrichmentConfig name</li>
+  </ul></li>
+  
+<li>Returns:
+  
+<ul>
+    
+<li>200 - Returns SensorEnrichmentConfig</li>
+    
+<li>404 - SensorEnrichmentConfig is missing</li>
+  </ul></li>
+</ul></div>
+<div class="section">
+<h3><a name="GET_apiv1sensorindexingconfig"></a><tt>GET /api/v1/sensor/indexing/config</tt></h3>
+
+<ul>
+  
+<li>Description: Retrieves all SensorIndexingConfigs from Zookeeper</li>
+  
+<li>Returns:
+  
+<ul>
+    
+<li>200 - Returns all SensorIndexingConfigs</li>
+  </ul></li>
+</ul></div>
+<div class="section">
+<h3><a name="DELETE_apiv1sensorindexingconfigname"></a><tt>DELETE /api/v1/sensor/indexing/config/{name}</tt></h3>
+
+<ul>
+  
+<li>Description: Deletes a SensorIndexingConfig from Zookeeper</li>
+  
+<li>Input:
+  
+<ul>
+    
+<li>name - SensorIndexingConfig name</li>
+  </ul></li>
+  
+<li>Returns:
+  
+<ul>
+    
+<li>200 - SensorIndexingConfig was deleted</li>
+    
+<li>404 - SensorIndexingConfig is missing</li>
+  </ul></li>
+</ul></div>
+<div class="section">
+<h3><a name="POST_apiv1sensorindexingconfigname"></a><tt>POST /api/v1/sensor/indexing/config/{name}</tt></h3>
+
+<ul>
+  
+<li>Description: Updates or creates a SensorIndexingConfig in Zookeeper</li>
+  
+<li>Input:
+  
+<ul>
+    
+<li>sensorIndexingConfig - SensorIndexingConfig</li>
+    
+<li>name - SensorIndexingConfig name</li>
+  </ul></li>
+  
+<li>Returns:
+  
+<ul>
+    
+<li>200 - SensorIndexingConfig updated. Returns saved SensorIndexingConfig</li>
+    
+<li>201 - SensorIndexingConfig created. Returns saved SensorIndexingConfig</li>
+  </ul></li>
+</ul></div>
+<div class="section">
+<h3><a name="GET_apiv1sensorindexingconfigname"></a><tt>GET /api/v1/sensor/indexing/config/{name}</tt></h3>
+
+<ul>
+  
+<li>Description: Retrieves a SensorIndexingConfig from Zookeeper</li>
+  
+<li>Input:
+  
+<ul>
+    
+<li>name - SensorIndexingConfig name</li>
+  </ul></li>
+  
+<li>Returns:
+  
+<ul>
+    
+<li>200 - Returns SensorIndexingConfig</li>
+    
+<li>404 - SensorIndexingConfig is missing</li>
+  </ul></li>
+</ul></div>
+<div class="section">
+<h3><a name="POST_apiv1sensorparserconfig"></a><tt>POST /api/v1/sensor/parser/config</tt></h3>
+
+<ul>
+  
+<li>Description: Updates or creates a SensorParserConfig in Zookeeper</li>
+  
+<li>Input:
+  
+<ul>
+    
+<li>sensorParserConfig - SensorParserConfig</li>
+  </ul></li>
+  
+<li>Returns:
+  
+<ul>
+    
+<li>200 - SensorParserConfig updated. Returns saved SensorParserConfig</li>
+    
+<li>201 - SensorParserConfig created. Returns saved SensorParserConfig</li>
+  </ul></li>
+</ul></div>
+<div class="section">
+<h3><a name="GET_apiv1sensorparserconfig"></a><tt>GET /api/v1/sensor/parser/config</tt></h3>
+
+<ul>
+  
+<li>Description: Retrieves all SensorParserConfigs from Zookeeper</li>
+  
+<li>Returns:
+  
+<ul>
+    
+<li>200 - Returns all SensorParserConfigs</li>
+  </ul></li>
+</ul></div>
+<div class="section">
+<h3><a name="GET_apiv1sensorparserconfiglistavailable"></a><tt>GET /api/v1/sensor/parser/config/list/available</tt></h3>
+
+<ul>
+  
+<li>Description: Lists the available parser classes that can be found on the classpath</li>
+  
+<li>Returns:
+  
+<ul>
+    
+<li>200 - Returns a list of available parser classes</li>
+  </ul></li>
+</ul></div>
+<div class="section">
+<h3><a name="POST_apiv1sensorparserconfigparseMessage"></a><tt>POST /api/v1/sensor/parser/config/parseMessage</tt></h3>
+
+<ul>
+  
+<li>Description: Parses a sample message given a SensorParserConfig</li>
+  
+<li>Input:
+  
+<ul>
+    
+<li>parseMessageRequest - Object containing a sample message and SensorParserConfig</li>
+  </ul></li>
+  
+<li>Returns:
+  
+<ul>
+    
+<li>200 - Returns parsed message</li>
+  </ul></li>
+</ul></div>
+<div class="section">
+<h3><a name="GET_apiv1sensorparserconfigreloadavailable"></a><tt>GET /api/v1/sensor/parser/config/reload/available</tt></h3>
+
+<ul>
+  
+<li>Description: Scans the classpath for available parser classes and reloads the cached parser class list</li>
+  
+<li>Returns:
+  
+<ul>
+    
+<li>200 - Returns a list of available parser classes</li>
+  </ul></li>
+</ul></div>
+<div class="section">
+<h3><a name="DELETE_apiv1sensorparserconfigname"></a><tt>DELETE /api/v1/sensor/parser/config/{name}</tt></h3>
+
+<ul>
+  
+<li>Description: Deletes a SensorParserConfig from Zookeeper</li>
+  
+<li>Input:
+  
+<ul>
+    
+<li>name - SensorParserConfig name</li>
+  </ul></li>
+  
+<li>Returns:
+  
+<ul>
+    
+<li>200 - SensorParserConfig was deleted</li>
+    
+<li>404 - SensorParserConfig is missing</li>
+  </ul></li>
+</ul></div>
+<div class="section">
+<h3><a name="GET_apiv1sensorparserconfigname"></a><tt>GET /api/v1/sensor/parser/config/{name}</tt></h3>
+
+<ul>
+  
+<li>Description: Retrieves a SensorParserConfig from Zookeeper</li>
+  
+<li>Input:
+  
+<ul>
+    
+<li>name - SensorParserConfig name</li>
+  </ul></li>
+  
+<li>Returns:
+  
+<ul>
+    
+<li>200 - Returns SensorParserConfig</li>
+    
+<li>404 - SensorParserConfig is missing</li>
+  </ul></li>
+</ul></div>
+<div class="section">
+<h3><a name="POST_apiv1stellarapplytransformations"></a><tt>POST /api/v1/stellar/apply/transformations</tt></h3>
+
+<ul>
+  
+<li>Description: Executes transformations against a sample message</li>
+  
+<li>Input:
+  
+<ul>
+    
+<li>transformationValidation - Object containing SensorParserConfig and sample message</li>
+  </ul></li>
+  
+<li>Returns:
+  
+<ul>
+    
+<li>200 - Returns transformation results</li>
+  </ul></li>
+</ul></div>
+<div class="section">
+<h3><a name="GET_apiv1stellarlist"></a><tt>GET /api/v1/stellar/list</tt></h3>
+
+<ul>
+  
+<li>Description: Retrieves field transformations</li>
+  
+<li>Returns:
+  
+<ul>
+    
+<li>200 - Returns a list field transformations</li>
+  </ul></li>
+</ul></div>
+<div class="section">
+<h3><a name="GET_apiv1stellarlistfunctions"></a><tt>GET /api/v1/stellar/list/functions</tt></h3>
+
+<ul>
+  
+<li>Description: Lists the Stellar functions that can be found on the classpath</li>
+  
+<li>Returns:
+  
+<ul>
+    
+<li>200 - Returns a list of Stellar functions</li>
+  </ul></li>
+</ul></div>
+<div class="section">
+<h3><a name="GET_apiv1stellarlistsimplefunctions"></a><tt>GET /api/v1/stellar/list/simple/functions</tt></h3>
+
+<ul>
+  
+<li>Description: Lists the simple Stellar functions (functions with only 1 input) that can be found on the classpath</li>
+  
+<li>Returns:
+  
+<ul>
+    
+<li>200 - Returns a list of simple Stellar functions</li>
+  </ul></li>
+</ul></div>
+<div class="section">
+<h3><a name="POST_apiv1stellarvalidaterules"></a><tt>POST /api/v1/stellar/validate/rules</tt></h3>
+
+<ul>
+  
+<li>Description: Tests Stellar statements to ensure they are well-formed</li>
+  
+<li>Input:
+  
+<ul>
+    
+<li>statements - List of statements to validate</li>
+  </ul></li>
+  
+<li>Returns:
+  
+<ul>
+    
+<li>200 - Returns validation results</li>
+  </ul></li>
+</ul></div>
+<div class="section">
+<h3><a name="GET_apiv1storm"></a><tt>GET /api/v1/storm</tt></h3>
+
+<ul>
+  
+<li>Description: Retrieves the status of all Storm topologies</li>
+  
+<li>Returns:
+  
+<ul>
+    
+<li>200 - Returns a list of topologies with status information</li>
+  </ul></li>
+</ul></div>
+<div class="section">
+<h3><a name="GET_apiv1stormclientstatus"></a><tt>GET /api/v1/storm/client/status</tt></h3>
+
+<ul>
+  
+<li>Description: Retrieves information about the Storm command line client</li>
+  
+<li>Returns:
+  
+<ul>
+    
+<li>200 - Returns storm command line client information</li>
+  </ul></li>
+</ul></div>
+<div class="section">
+<h3><a name="GET_apiv1stormenrichment"></a><tt>GET /api/v1/storm/enrichment</tt></h3>
+
+<ul>
+  
+<li>Description: Retrieves the status of the Storm enrichment topology</li>
+  
+<li>Returns:
+  
+<ul>
+    
+<li>200 - Returns topology status information</li>
+    
+<li>404 - Topology is missing</li>
+  </ul></li>
+</ul></div>
+<div class="section">
+<h3><a name="GET_apiv1stormenrichmentactivate"></a><tt>GET /api/v1/storm/enrichment/activate</tt></h3>
+
+<ul>
+  
+<li>Description: Activates a Storm enrichment topology</li>
+  
+<li>Returns:
+  
+<ul>
+    
+<li>200 - Returns activate response message</li>
+  </ul></li>
+</ul></div>
+<div class="section">
+<h3><a name="GET_apiv1stormenrichmentdeactivate"></a><tt>GET /api/v1/storm/enrichment/deactivate</tt></h3>
+
+<ul>
+  
+<li>Description: Deactivates a Storm enrichment topology</li>
+  
+<li>Returns:
+  
+<ul>
+    
+<li>200 - Returns deactivate response message</li>
+  </ul></li>
+</ul></div>
+<div class="section">
+<h3><a name="GET_apiv1stormenrichmentstart"></a><tt>GET /api/v1/storm/enrichment/start</tt></h3>
+
+<ul>
+  
+<li>Description: Starts a Storm enrichment topology</li>
+  
+<li>Returns:
+  
+<ul>
+    
+<li>200 - Returns start response message</li>
+  </ul></li>
+</ul></div>
+<div class="section">
+<h3><a name="GET_apiv1stormenrichmentstop"></a><tt>GET /api/v1/storm/enrichment/stop</tt></h3>
+
+<ul>
+  
+<li>Description: Stops a Storm enrichment topology</li>
+  
+<li>Input:
+  
+<ul>
+    
+<li>stopNow - Stop the topology immediately</li>
+  </ul></li>
+  
+<li>Returns:
+  
+<ul>
+    
+<li>200 - Returns stop response message</li>
+  </ul></li>
+</ul></div>
+<div class="section">
+<h3><a name="GET_apiv1stormindexing"></a><tt>GET /api/v1/storm/indexing</tt></h3>
+
+<ul>
+  
+<li>Description: Retrieves the status of the Storm indexing topology</li>
+  
+<li>Returns:
+  
+<ul>
+    
+<li>200 - Returns topology status information</li>
+    
+<li>404 - Topology is missing</li>
+  </ul></li>
+</ul></div>
+<div class="section">
+<h3><a name="GET_apiv1stormindexingactivate"></a><tt>GET /api/v1/storm/indexing/activate</tt></h3>
+
+<ul>
+  
+<li>Description: Activates a Storm indexing topology</li>
+  
+<li>Returns:
+  
+<ul>
+    
+<li>200 - Returns activate response message</li>
+  </ul></li>
+</ul></div>
+<div class="section">
+<h3><a name="GET_apiv1stormindexingdeactivate"></a><tt>GET /api/v1/storm/indexing/deactivate</tt></h3>
+
+<ul>
+  
+<li>Description: Deactivates a Storm indexing topology</li>
+  
+<li>Returns:
+  
+<ul>
+    
+<li>200 - Returns deactivate response message</li>
+  </ul></li>
+</ul></div>
+<div class="section">
+<h3><a name="GET_apiv1stormindexingstart"></a><tt>GET /api/v1/storm/indexing/start</tt></h3>
+
+<ul>
+  
+<li>Description: Starts a Storm indexing topology</li>
+  
+<li>Returns:
+  
+<ul>
+    
+<li>200 - Returns start response message</li>
+  </ul></li>
+</ul></div>
+<div class="section">
+<h3><a name="GET_apiv1stormindexingstop"></a><tt>GET /api/v1/storm/indexing/stop</tt></h3>
+
+<ul>
+  
+<li>Description: Stops a Storm enrichment topology</li>
+  
+<li>Input:
+  
+<ul>
+    
+<li>stopNow - Stop the topology immediately</li>
+  </ul></li>
+  
+<li>Returns:
+  
+<ul>
+    
+<li>200 - Returns stop response message</li>
+  </ul></li>
+</ul></div>
+<div class="section">
+<h3><a name="GET_apiv1stormparseractivatename"></a><tt>GET /api/v1/storm/parser/activate/{name}</tt></h3>
+
+<ul>
+  
+<li>Description: Activates a Storm parser topology</li>
+  
+<li>Input:
+  
+<ul>
+    
+<li>name - Parser name</li>
+  </ul></li>
+  
+<li>Returns:
+  
+<ul>
+    
+<li>200 - Returns activate response message</li>
+  </ul></li>
+</ul></div>
+<div class="section">
+<h3><a name="GET_apiv1stormparserdeactivatename"></a><tt>GET /api/v1/storm/parser/deactivate/{name}</tt></h3>
+
+<ul>
+  
+<li>Description: Deactivates a Storm parser topology</li>
+  
+<li>Input:
+  
+<ul>
+    
+<li>name - Parser name</li>
+  </ul></li>
+  
+<li>Returns:
+  
+<ul>
+    
+<li>200 - Returns deactivate response message</li>
+  </ul></li>
+</ul></div>
+<div class="section">
+<h3><a name="GET_apiv1stormparserstartname"></a><tt>GET /api/v1/storm/parser/start/{name}</tt></h3>
+
+<ul>
+  
+<li>Description: Starts a Storm parser topology</li>
+  
+<li>Input:
+  
+<ul>
+    
+<li>name - Parser name</li>
+  </ul></li>
+  
+<li>Returns:
+  
+<ul>
+    
+<li>200 - Returns start response message</li>
+  </ul></li>
+</ul></div>
+<div class="section">
+<h3><a name="GET_apiv1stormparserstopname"></a><tt>GET /api/v1/storm/parser/stop/{name}</tt></h3>
+
+<ul>
+  
+<li>Description: Stops a Storm parser topology</li>
+  
+<li>Input:
+  
+<ul>
+    
+<li>name - Parser name</li>
+    
+<li>stopNow - Stop the topology immediately</li>
+  </ul></li>
+  
+<li>Returns:
+  
+<ul>
+    
+<li>200 - Returns stop response message</li>
+  </ul></li>
+</ul></div>
+<div class="section">
+<h3><a name="GET_apiv1stormname"></a><tt>GET /api/v1/storm/{name}</tt></h3>
+
+<ul>
+  
+<li>Description: Retrieves the status of a Storm topology</li>
+  
+<li>Input:
+  
+<ul>
+    
+<li>name - Topology name</li>
+  </ul></li>
+  
+<li>Returns:
+  
+<ul>
+    
+<li>200 - Returns topology status information</li>
+    
+<li>404 - Topology is missing</li>
+  </ul></li>
+</ul></div>
+<div class="section">
+<h3><a name="GET_apiv1user"></a><tt>GET /api/v1/user</tt></h3>
+
+<ul>
+  
+<li>Description: Retrieves the current user</li>
+  
+<li>Returns:
+  
+<ul>
+    
+<li>200 - Current user</li>
+  </ul></li>
+</ul></div></div>
+<div class="section">
+<h2><a name="Testing"></a>Testing</h2>
+<p>Profiles are includes for both the metron-docker and Quick Dev environments.</p>
+<div class="section">
+<h3><a name="metron-docker"></a>metron-docker</h3>
+<p>Start the <a href="../../metron-docker/index.html">metron-docker</a> environment. Build the metron-rest module and start it with the Spring Boot Maven plugin:</p>
+
+<div class="source">
+<div class="source">
+<pre>mvn clean package
+mvn spring-boot:run -Drun.profiles=docker,dev
+</pre></div></div>
+<p>The metron-rest application will be available at <a class="externalLink" href="http://localhost:8080/swagger-ui.html#/">http://localhost:8080/swagger-ui.html#/</a>.</p></div>
+<div class="section">
+<h3><a name="Quick_Dev"></a>Quick Dev</h3>
+<p>Start the <a href="../../metron-deployment/vagrant/quick-dev-platform/index.html">Quick Dev</a> environment. Build the metron-rest module and start it with the Spring Boot Maven plugin:</p>
+
+<div class="source">
+<div class="source">
+<pre>mvn clean package
+mvn spring-boot:run -Drun.profiles=vagrant,dev
+</pre></div></div>
+<p>The metron-rest application will be available at <a class="externalLink" href="http://localhost:8080/swagger-ui.html#/">http://localhost:8080/swagger-ui.html#/</a>.</p>
+<p>To run the application locally on the Quick Dev host (node1), follow the <a href="#Installation">Installation</a> instructions above. Then set the METRON_SPRING_PROFILES_ACTIVE variable in <tt>/etc/sysconfig/metron</tt>:</p>
+
+<div class="source">
+<div class="source">
+<pre>METRON_SPRING_PROFILES_ACTIVE=&quot;vagrant,dev&quot;
+</pre></div></div>
+<p>and start the application:</p>
+
+<div class="source">
+<div class="source">
+<pre>service metron-rest start
+</pre></div></div>
+<p>In a cluster with Kerberos enabled, update the security settings in <tt>/etc/sysconfig/metron</tt>. Security is disabled by default in the <tt>vagrant</tt> Spring profile so that setting must be overriden with the METRON_SPRING_OPTIONS variable:</p>
+
+<div class="source">
+<div class="source">
+<pre>METRON_SPRING_PROFILES_ACTIVE=&quot;vagrant,dev&quot;
+METRON_JVMFLAGS=&quot;-Djava.security.auth.login.config=$METRON_HOME/client_jaas.conf&quot;
+METRON_SPRING_OPTIONS=&quot;--kerberos.enabled=true&quot;
+</pre></div></div>
+<p>The metron-rest application will be available at <a class="externalLink" href="http://node1:8082/swagger-ui.html#/">http://node1:8082/swagger-ui.html#/</a>.</p></div></div>
+<div class="section">
+<h2><a name="License"></a>License</h2>
+<p>This project depends on the Java Transaction API. See <a class="externalLink" href="https://java.net/projects/jta-spec/">https://java.net/projects/jta-spec/</a> for more details.</p></div>
+                  </div>
+            </div>
+          </div>
+
+    <hr/>
+
+    <footer>
+            <div class="container-fluid">
+              <div class="row span12">Copyright &copy;                    2017
+                        <a href="https://www.apache.org">The Apache Software Foundation</a>.
+            All Rights Reserved.      
+                    
+      </div>
+
+                          
+        
+                </div>
+    </footer>
+  </body>
+</html>



Mime
View raw message