http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/39059e5b/site/current-book/metron-deployment/packaging/docker/ansible-docker/index.html ---------------------------------------------------------------------- diff --git a/site/current-book/metron-deployment/packaging/docker/ansible-docker/index.html b/site/current-book/metron-deployment/packaging/docker/ansible-docker/index.html new file mode 100644 index 0000000..e01ec59 --- /dev/null +++ b/site/current-book/metron-deployment/packaging/docker/ansible-docker/index.html @@ -0,0 +1,239 @@ + + + + + + + + + Metron – Overview + + + + + + + + + + + + + + + + + +
+ + + + + +
+
+ +
+ + +
+ +

Overview

+

+

The Metron ansible-docker container is provided in an effort reduce the installation burden of deploying Metron in a live envirionment. It is provisioned with software required to sucessfully run the deployment scripts.

+
+

Building the Container

+ +
    + +
  1. Install Docker [https://www.docker.com/products/overview]
  2. + +
  3. Navigate to <project-directory>/metron-deployment/packaging/docker/ansible-docker
  4. + +
  5. Build the container docker build -t ansible-docker:2.0.0.2 .
  6. +
+
+

Using the Container

+

Full instructions are found on the wiki [https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=65144361].

+

tl;dr

+ +
    + +
  1. docker run -it -v <project-directory>:/root/incubator-metron ansible-docker:2.0.0.2 bash
  2. + +
  3. cd /root/incubator-metron
  4. + +
  5. mvn clean package -DskipTests
  6. +
+
+
+
+ +
+ + + + http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/39059e5b/site/current-book/metron-deployment/packaging/docker/rpm-docker/index.html ---------------------------------------------------------------------- diff --git a/site/current-book/metron-deployment/packaging/docker/rpm-docker/index.html b/site/current-book/metron-deployment/packaging/docker/rpm-docker/index.html new file mode 100644 index 0000000..ae6948e --- /dev/null +++ b/site/current-book/metron-deployment/packaging/docker/rpm-docker/index.html @@ -0,0 +1,226 @@ + + + + + + + + + Metron – Overview + + + + + + + + + + + + + + + + + +
+ + + + + +
+
+ +
+ + +
+ +

Overview

+

+

The Metron ansible-docker container is provided in an effort reduce the installation burden of deploying Metron in a live envirionment. It is provisioned with software required to sucessfully run the deployment scripts.

+
+

Building the Container

+ +
    + +
  1. Install Docker [https://www.docker.com/products/overview]
  2. + +
  3. Navigate to <project-directory>/metron-deployment/packaging/rpm-docker
  4. + +
  5. Build the container docker build -t rpm-docker .
  6. +
+
+
+
+ +
+ + + + http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/39059e5b/site/current-book/metron-deployment/packer-build/index.html ---------------------------------------------------------------------- diff --git a/site/current-book/metron-deployment/packer-build/index.html b/site/current-book/metron-deployment/packer-build/index.html new file mode 100644 index 0000000..a18e5ab --- /dev/null +++ b/site/current-book/metron-deployment/packer-build/index.html @@ -0,0 +1,274 @@ + + + + + + + + + Metron – Build Metron Images + + + + + + + + + + + + + + + + + +
+ + + + + +
+
+ +
+ + +
+ +

Build Metron Images

+

Based on the fantastic Bento project developed by Chef.

+
+

Images Provided

+ +
    + +
  • hdp-centos-6.7: Centos 6.7 + HDP. Used in the quick-dev-platform Vagrant image
  • + +
  • metron-centos-6.7: Centos 6.7 + HDP + Metron. Used for the codelab-platform Vagrant image.
  • +
+
+

Prerequisites

+ +
    + +
  • Packer 0.10.1
  • + +
  • Virtualbox 5.0.16
  • + +
  • Be sure to build Metron prior to building the images (cd your-project-directory/metron-platform && mvn clean package -DskipTests)
  • +
+

Build Both Images ———————- Navigate to <your-project-directory>/metron-deployment/packer-build Execute bin/bento build

+

Packer will build both images and export .box files to the ./builds directory.

+

Build Single Images ———————- Navigate to your-project-directory/metron-deployment/packer-build

+ +
    + +
  • HDP Centos
  • +
+ +
+
+
bin/bento build hdp-centos-6.7.json
+
+ +
    + +
  • Full Metron
  • +
+ +
+
+
bin/bento build metron-centos-6.7.json
+
+

Using Your New Box File ———————- Modify the relevant Vagrantfile (codelab-platform or quick-dev-platform) replacing the lines:

+ +
+
+
<pre><code>config.vm.box = "<i>box_name</i>"
+config.ssh.insert_key = true</code></pre>
+
+

with

+ +
+
+
<pre></code>config.vm.box = "<i>test_box_name</i>"
+config.vm.box = "<i>PathToBoxfile/Boxfilename</i>"
+config.ssh.insert_key = true</code></pre>
+
+

Launch the image as usual.

+

Node: Vagrant will cache boxes, you can force Vagrant to reload your box by running vagrant box remove test_box_name before launching your new image.

+
+
+
+ +
+ + + + http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/39059e5b/site/current-book/metron-deployment/roles/index.html ---------------------------------------------------------------------- diff --git a/site/current-book/metron-deployment/roles/index.html b/site/current-book/metron-deployment/roles/index.html new file mode 100644 index 0000000..cdcf082 --- /dev/null +++ b/site/current-book/metron-deployment/roles/index.html @@ -0,0 +1,271 @@ + + + + + + + + + Metron – Ansible Roles + + + + + + + + + + + + + + + + + +
+ + + + + +
+
+ +
+ + +
+ +

Ansible Roles

+

+ +
    + +
  • Kibana
  • + +
  • Monit
  • + +
  • OpenTaxii
  • + +
  • Pcap Relay
  • + +
  • Sensor Test Mode
  • +
+
+
+
+ +
+ + + + http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/39059e5b/site/current-book/metron-deployment/roles/kibana/index.html ---------------------------------------------------------------------- diff --git a/site/current-book/metron-deployment/roles/kibana/index.html b/site/current-book/metron-deployment/roles/kibana/index.html new file mode 100644 index 0000000..19388f9 --- /dev/null +++ b/site/current-book/metron-deployment/roles/kibana/index.html @@ -0,0 +1,285 @@ + + + + + + + + + Metron – Kibana 4 + + + + + + + + + + + + + + + + + +
+ + + + + +
+
+ +
+ + +
+ +
+

Kibana 4

+

This role installs Kibana along with the default Metron Dashboard.

+
+

FAQ

+
+

How do I change Metron’s default dashboard?

+

Kibana stores all configuration elements within an Elasticsearch index called .kibana. To deploy Kibana in a desired state, including the Metron Dashboard, we simply take an extract from a functioning Kibana instance and store that in templates/kibana-index.json. The deployment process then restores the index from this extract.

+

(1) Stand-up an instance of Apache Metron and create the Kibana index patterns, visualizations, and dashboard as you see fit.

+

(2) Run the following command to extract the definitions for all the components that you have created. Be sure to delete anything that you don’t want to be part of this extract. It will include all artifacts present in your .kibana index.

+ +
+
+
  elasticdump --input=http://ec2-52-41-121-175.us-west-2.compute.amazonaws.com:9200/.kibana \
+    --output=kibana-index.json \
+    --type=data \
+    --searchBody='{"filter": { "or": [ {"type": {"value": "search"}}, {"type": {"value":"dashboard"}}, {"type": {"value":"visualization"}},{"type": {"value": "config"}},{"type": {"value": "url"}},{"type": {"value": "index-pattern"}} ] }}'
+
+

(3) This will result in a file containing the JSON-based definitions. Overwrite templates/kibana-index.json.

+

(4) After redeploying the code, your changes should now be a part of the default Metron dashboard.

+
+

Why do my dashboard components change their order when reloading the dashboard?

+

This has been a problem in Kibana 4.5.1 and perhaps other versions too. To address this problem find the definition for your dashboard in the Kibana index extract. It will look like the following.

+ +
+
+
{"_index":".kibana","_type":"dashboard","_id":"Metron-Dashboard",...
+
+

Extract the panelsJSON field from the dashboard definition. Reorder the definition of these panels so that they are ordered by row and column. The component in row 1 should come before the component in row 2, etc. After you have ordered the components in this way, Kibana will maintain the order of components in the dashboard.

+
+
+
+ +
+ + + + http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/39059e5b/site/current-book/metron-deployment/roles/monit/index.html ---------------------------------------------------------------------- diff --git a/site/current-book/metron-deployment/roles/monit/index.html b/site/current-book/metron-deployment/roles/monit/index.html new file mode 100644 index 0000000..07f3bcd --- /dev/null +++ b/site/current-book/metron-deployment/roles/monit/index.html @@ -0,0 +1,327 @@ + + + + + + + + + Metron – Monit Integration + + + + + + + + + + + + + + + + + +
+ + + + + +
+
+ +
+ + +
+ +

Monit Integration

+

+

This role will leverage Monit as a process watchdog to manage sensors, topologies, and core services.

+ +
    + +
  • Monit can be used to start, stop, or check status of any of the sensors or topologies.
  • + +
  • When monitoring is enabled (on by default) if a process dies, it will be restarted.
  • + +
  • The Monit web interface is exposed at http://hostname:2812.
  • + +
  • The web interface username and password is defined by the monit_user and monit_password variables. These default to admin/monit.
  • + +
  • Monit CLI tools can also be used to simplify the process of managing Metron components.
  • + +
  • +

    The post-deployment report for Amazon-EC2 provides links to Monit’s web interface labeled as ‘Sensor Status’ and ‘Topology Status.’

    + +
    +
    +
      ok: [localhost] => {
    +"Success": [
    +    "Apache Metron deployed successfully",
    +    "   Metron          @ http://ec2-52-39-143-62.us-west-2.compute.amazonaws.com:5000",
    +    "   Ambari          @ http://ec2-52-39-4-93.us-west-2.compute.amazonaws.com:8080",
    +    "   Sensor Status   @ http://ec2-52-39-4-93.us-west-2.compute.amazonaws.com:2812",
    +    "   Topology Status @ http://ec2-52-39-130-62.us-west-2.compute.amazonaws.com:2812",
    +    "For additional information, see https://metron.incubator.apache.org/'"
    +]
    +  }
    +
  • +
+
+

Usage

+

Start all Metron components

+ +
+
+
monit start all
+
+

Stop all Metron components

+ +
+
+
monit stop all
+
+

Start an individual Metron component

+ +
+
+
monit start bro-parser
+
+

Start all components required to ingest Bro data

+ +
+
+
monit -g bro start
+
+

Start all parsers

+ +
+
+
monit -g parsers start
+
+

What is running?

+ +
+
+
monit summary
+
+
+
+
+ +
+ + + +