Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 235D4200AE4 for ; Wed, 11 May 2016 05:55:34 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 22203160A12; Wed, 11 May 2016 03:55:34 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id C75BD16098A for ; Wed, 11 May 2016 05:55:31 +0200 (CEST) Received: (qmail 61318 invoked by uid 500); 11 May 2016 03:55:31 -0000 Mailing-List: contact commits-help@eagle.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@eagle.incubator.apache.org Delivered-To: mailing list commits@eagle.incubator.apache.org Received: (qmail 61305 invoked by uid 99); 11 May 2016 03:55:30 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 11 May 2016 03:55:30 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id 67683180148 for ; Wed, 11 May 2016 03:55:30 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 5.051 X-Spam-Level: ***** X-Spam-Status: No, score=5.051 tagged_above=-999 required=6.31 tests=[KAM_ASCII_DIVIDERS=0.8, KAM_BADIPHTTP=2, KAM_LAZY_DOMAIN_SECURITY=1, KAM_LOTSOFHASH=0.25, LONG_HEX_URI=1, NORMAL_HTTP_TO_IP=0.001, RP_MATCHES_RCVD=-0.001, WEIRD_PORT=0.001] autolearn=disabled Received: from mx2-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id BC3RZlHo9DvH for ; Wed, 11 May 2016 03:55:18 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx2-lw-eu.apache.org (ASF Mail Server at mx2-lw-eu.apache.org) with ESMTP id 6F01B5F3DA for ; Wed, 11 May 2016 03:55:17 +0000 (UTC) Received: from svn01-us-west.apache.org (svn.apache.org [10.41.0.6]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 3C01EE00A4 for ; Wed, 11 May 2016 03:55:16 +0000 (UTC) Received: from svn01-us-west.apache.org (localhost [127.0.0.1]) by svn01-us-west.apache.org (ASF Mail Server at svn01-us-west.apache.org) with ESMTP id D6D623A021C for ; Wed, 11 May 2016 03:55:15 +0000 (UTC) Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Subject: svn commit: r1743294 [1/3] - in /incubator/eagle/site: ./ development/ docs/ docs/tutorial/ images/docs/ Date: Wed, 11 May 2016 03:55:15 -0000 To: commits@eagle.incubator.apache.org From: hdendukuri@apache.org X-Mailer: svnmailer-1.0.9 Message-Id: <20160511035515.D6D623A021C@svn01-us-west.apache.org> archived-at: Wed, 11 May 2016 03:55:34 -0000 Author: hdendukuri Date: Wed May 11 03:55:14 2016 New Revision: 1743294 URL: http://svn.apache.org/viewvc?rev=1743294&view=rev Log: Updated Quick Starter Guide. Added Download, JMX metric Monitoring setup, development-quick-guide and hdfs-hive-monitoring. Modified documents menu. https://github.com/eaglemonitoring/eaglemonitoring.github.io/pull/40 Added: incubator/eagle/site/docs/development-quick-guide.html (with props) incubator/eagle/site/docs/download.html (with props) incubator/eagle/site/docs/hdfs-hive-monitoring.html (with props) incubator/eagle/site/docs/jmx-metric-monitoring.html (with props) incubator/eagle/site/images/docs/new-jmx-site.png (with props) incubator/eagle/site/images/docs/nn-restart.png (with props) Modified: incubator/eagle/site/development/index.html incubator/eagle/site/docs/FAQ.html incubator/eagle/site/docs/ambari-plugin-install.html incubator/eagle/site/docs/architecture-highlights.html incubator/eagle/site/docs/community.html incubator/eagle/site/docs/configuration.html incubator/eagle/site/docs/deployment-in-production.html incubator/eagle/site/docs/deployment-in-sandbox.html incubator/eagle/site/docs/ecosystem.html incubator/eagle/site/docs/import-hdfs-auditLog.html incubator/eagle/site/docs/index.html incubator/eagle/site/docs/installation.html incubator/eagle/site/docs/policy-api.html incubator/eagle/site/docs/quick-start.html incubator/eagle/site/docs/standalone-install.html incubator/eagle/site/docs/terminology.html incubator/eagle/site/docs/tutorial/classification.html incubator/eagle/site/docs/tutorial/ldap.html incubator/eagle/site/docs/tutorial/policy.html incubator/eagle/site/docs/tutorial/setup.html incubator/eagle/site/docs/tutorial/userprofile.html incubator/eagle/site/docs/usecase.html incubator/eagle/site/docs/user-profile-ml.html incubator/eagle/site/feed.xml Modified: incubator/eagle/site/development/index.html URL: http://svn.apache.org/viewvc/incubator/eagle/site/development/index.html?rev=1743294&r1=1743293&r2=1743294&view=diff ============================================================================== --- incubator/eagle/site/development/index.html (original) +++ incubator/eagle/site/development/index.html Wed May 11 03:55:14 2016 @@ -117,7 +117,7 @@

Eagle Development

-

Start to write from here with markdown

+

Start to write from here with markdown

Sample Content

Modified: incubator/eagle/site/docs/FAQ.html URL: http://svn.apache.org/viewvc/incubator/eagle/site/docs/FAQ.html?rev=1743294&r1=1743293&r2=1743294&view=diff ============================================================================== --- incubator/eagle/site/docs/FAQ.html (original) +++ incubator/eagle/site/docs/FAQ.html Wed May 11 03:55:14 2016 @@ -119,8 +119,6 @@
  • Introduction
    • -
  • Quick Start
    • -
  • Terminology
  • Ecosystem
    • @@ -133,15 +131,21 @@
  • Installation
    • -
  • Deploy Eagle in Sandbox
    • +
  • Quick Starter(Eagle In Sandbox)
    • + +
  • Download
    • + +
  • HDFS & Hive Activity Monitoring
    • -
  • Deploy Eagle in the Production
    • +
  • JMX URL Monitoring
  • Stream Log Data into Kafka
    • -
  • Install Eagle Ambari Plugin
    • +
  • Eagle Cluster Deployment
    • -
  • Configuration
    • +
  • Eagle Configuration
    • + +
  • Enable LDAP
  • @@ -155,7 +159,17 @@
  • Data Classification Tutorial
    • -
  • Eagle LDAP Tutorial
    • +
  • + +
  • How To
    • + +
  • Use Metadata API
    • + +
  • + +
  • Development Guide
    • + +
  • Development Quick Guide
  • @@ -163,8 +177,6 @@
  • Eagle Architecture Highlights
    • -
  • Policy API Design
    • -
  • User Profile Machine Learning
  • @@ -184,14 +196,16 @@

    Add the following line in host machine’s hosts file

    - 
    127.0.0.1 sandbox.hortonworks.com
+    
    127.0.0.1 sandbox.hortonworks.com
 
    
+    
    
   
   
  • 
     
    Q2. Not able to send data into kafka using kafka console producer:
    
 
-    
    /usr/hdp/current/kafka-broker/bin/kafka-console-producer.sh --broker-list localhost:6667 --topic sandbox_hdfs_audit_log
+    
    /usr/hdp/current/kafka-broker/bin/kafka-console-producer.sh --broker-list localhost:6667 --topic sandbox_hdfs_audit_log
 
    
+    
    
 
     
    
       
    Kafka broker are binding to host sandbox.hortonworks.com
    

Modified: incubator/eagle/site/docs/ambari-plugin-install.html
URL: http://svn.apache.org/viewvc/incubator/eagle/site/docs/ambari-plugin-install.html?rev=1743294&r1=1743293&r2=1743294&view=diff
==============================================================================
--- incubator/eagle/site/docs/ambari-plugin-install.html (original)
+++ incubator/eagle/site/docs/ambari-plugin-install.html Wed May 11 03:55:14 2016
@@ -119,8 +119,6 @@
         
           
  • Introduction
    • 
         
-          
  • Quick Start
    • 
-        
           
  • Terminology
    • 
         
           
  • Ecosystem
    • 
@@ -133,15 +131,21 @@
        
         
  • Installation
    • 
         
-          
  • Deploy Eagle in Sandbox
    • 
+          
  • Quick Starter(Eagle In Sandbox)
    • 
+        
+          
  • Download
    • 
+        
+          
  • HDFS & Hive Activity Monitoring
    • 
         
-          
  • Deploy Eagle in the Production
    • 
+          
  • JMX URL Monitoring
    • 
         
           
  • Stream Log Data into Kafka
    • 
         
-          
  • Install Eagle Ambari Plugin
    • 
+          
  • Eagle Cluster Deployment
    • 
         
-          
  • Configuration
    • 
+          
  • Eagle Configuration
    • 
+        
+          
  • Enable LDAP
    • 
         
         
  • 
        
@@ -155,7 +159,17 @@
         
           
  • Data Classification Tutorial
    • 
         
-          
  • Eagle LDAP Tutorial
    • 
+        
  • 
+       
+        
  • How To
    • 
+        
+          
  • Use Metadata API
    • 
+        
+        
  • 
+       
+        
  • Development Guide
    • 
+        
+          
  • Development Quick Guide
    • 
         
         
  • 
        
@@ -163,8 +177,6 @@
         
           
  • Eagle Architecture Highlights
    • 
         
-          
  • Policy API Design
    • 
-        
           
  • User Profile Machine Learning
    • 
         
         
  • 
@@ -188,8 +200,9 @@
   
  • 
     
    Create a Kafka topic if you have not. Here is an example command.
    
 
-    
    $ /usr/hdp/current/kafka-broker/bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic sandbox_hdfs_audit_log
+    
    $ /usr/hdp/current/kafka-broker/bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic sandbox_hdfs_audit_log
 
    
+    
    
   
    • 
   
  • 
     
    Stream HDFS log data to Kafka, and refer to here on how to do it .
    
@@ -205,8 +218,9 @@
   
  • 
     
    Install Eagle Ambari plugin
    
 
-    
    $ /usr/hdp/current/eagle/bin/eagle-ambari.sh install
+    
    $ /usr/hdp/current/eagle/bin/eagle-ambari.sh install
 
    
+    
    
   
    • 
   
  • 
     
    Restart Ambari click on disable and enable Ambari back.
    
@@ -220,10 +234,11 @@
   
  • 
     
    Add Policies and meta data required by running the below script.
    
 
-    
    $ cd <eagle-home>
+    
    $ cd <eagle-home>
 $ examples/sample-sensitivity-resource-create.sh
 $ examples/sample-policy-create.sh
 
    
+    
    
   
    • 
 
 

Modified: incubator/eagle/site/docs/architecture-highlights.html
URL: http://svn.apache.org/viewvc/incubator/eagle/site/docs/architecture-highlights.html?rev=1743294&r1=1743293&r2=1743294&view=diff
==============================================================================
--- incubator/eagle/site/docs/architecture-highlights.html (original)
+++ incubator/eagle/site/docs/architecture-highlights.html Wed May 11 03:55:14 2016
@@ -119,8 +119,6 @@
         
           
  • Introduction
    • 
         
-          
  • Quick Start
    • 
-        
           
  • Terminology
    • 
         
           
  • Ecosystem
    • 
@@ -133,15 +131,21 @@
        
         
  • Installation
    • 
         
-          
  • Deploy Eagle in Sandbox
    • 
+          
  • Quick Starter(Eagle In Sandbox)
    • 
+        
+          
  • Download
    • 
+        
+          
  • HDFS & Hive Activity Monitoring
    • 
         
-          
  • Deploy Eagle in the Production
    • 
+          
  • JMX URL Monitoring
    • 
         
           
  • Stream Log Data into Kafka
    • 
         
-          
  • Install Eagle Ambari Plugin
    • 
+          
  • Eagle Cluster Deployment
    • 
         
-          
  • Configuration
    • 
+          
  • Eagle Configuration
    • 
+        
+          
  • Enable LDAP
    • 
         
         
  • 
        
@@ -155,7 +159,17 @@
         
           
  • Data Classification Tutorial
    • 
         
-          
  • Eagle LDAP Tutorial
    • 
+        
  • 
+       
+        
  • How To
    • 
+        
+          
  • Use Metadata API
    • 
+        
+        
  • 
+       
+        
  • Development Guide
    • 
+        
+          
  • Development Quick Guide
    • 
         
         
  • 
        
@@ -163,8 +177,6 @@
         
           
  • Eagle Architecture Highlights
    • 
         
-          
  • Policy API Design
    • 
-        
           
  • User Profile Machine Learning
    • 
         
         
  • 

Modified: incubator/eagle/site/docs/community.html
URL: http://svn.apache.org/viewvc/incubator/eagle/site/docs/community.html?rev=1743294&r1=1743293&r2=1743294&view=diff
==============================================================================
--- incubator/eagle/site/docs/community.html (original)
+++ incubator/eagle/site/docs/community.html Wed May 11 03:55:14 2016
@@ -119,8 +119,6 @@
         
           
  • Introduction
    • 
         
-          
  • Quick Start
    • 
-        
           
  • Terminology
    • 
         
           
  • Ecosystem
    • 
@@ -133,15 +131,21 @@
        
         
  • Installation
    • 
         
-          
  • Deploy Eagle in Sandbox
    • 
+          
  • Quick Starter(Eagle In Sandbox)
    • 
+        
+          
  • Download
    • 
+        
+          
  • HDFS & Hive Activity Monitoring
    • 
         
-          
  • Deploy Eagle in the Production
    • 
+          
  • JMX URL Monitoring
    • 
         
           
  • Stream Log Data into Kafka
    • 
         
-          
  • Install Eagle Ambari Plugin
    • 
+          
  • Eagle Cluster Deployment
    • 
         
-          
  • Configuration
    • 
+          
  • Eagle Configuration
    • 
+        
+          
  • Enable LDAP
    • 
         
         
  • 
        
@@ -155,7 +159,17 @@
         
           
  • Data Classification Tutorial
    • 
         
-          
  • Eagle LDAP Tutorial
    • 
+        
  • 
+       
+        
  • How To
    • 
+        
+          
  • Use Metadata API
    • 
+        
+        
  • 
+       
+        
  • Development Guide
    • 
+        
+          
  • Development Quick Guide
    • 
         
         
  • 
        
@@ -163,8 +177,6 @@
         
           
  • Eagle Architecture Highlights
    • 
         
-          
  • Policy API Design
    • 
-        
           
  • User Profile Machine Learning
    • 
         
         
  • 

Modified: incubator/eagle/site/docs/configuration.html
URL: http://svn.apache.org/viewvc/incubator/eagle/site/docs/configuration.html?rev=1743294&r1=1743293&r2=1743294&view=diff
==============================================================================
--- incubator/eagle/site/docs/configuration.html (original)
+++ incubator/eagle/site/docs/configuration.html Wed May 11 03:55:14 2016
@@ -119,8 +119,6 @@
         
           
  • Introduction
    • 
         
-          
  • Quick Start
    • 
-        
           
  • Terminology
    • 
         
           
  • Ecosystem
    • 
@@ -133,15 +131,21 @@
        
         
  • Installation
    • 
         
-          
  • Deploy Eagle in Sandbox
    • 
+          
  • Quick Starter(Eagle In Sandbox)
    • 
+        
+          
  • Download
    • 
+        
+          
  • HDFS & Hive Activity Monitoring
    • 
         
-          
  • Deploy Eagle in the Production
    • 
+          
  • JMX URL Monitoring
    • 
         
           
  • Stream Log Data into Kafka
    • 
         
-          
  • Install Eagle Ambari Plugin
    • 
+          
  • Eagle Cluster Deployment
    • 
         
-          
  • Configuration
    • 
+          
  • Eagle Configuration
    • 
+        
+          
  • Enable LDAP
    • 
         
         
  • 
        
@@ -155,7 +159,17 @@
         
           
  • Data Classification Tutorial
    • 
         
-          
  • Eagle LDAP Tutorial
    • 
+        
  • 
+       
+        
  • How To
    • 
+        
+          
  • Use Metadata API
    • 
+        
+        
  • 
+       
+        
  • Development Guide
    • 
+        
+          
  • Development Quick Guide
    • 
         
         
  • 
        
@@ -163,8 +177,6 @@
         
           
  • Eagle Architecture Highlights
    • 
         
-          
  • Policy API Design
    • 
-        
           
  • User Profile Machine Learning
    • 
         
         
  • 
@@ -176,7 +188,7 @@

    • Configuration

    -

    Eagle requires you to create a configuration file under conf/ directory for each topology. This page will give detailed +

    Eagle requires you to create a configuration file under conf/ directory for each topology. This page will give detailed description of Eagle topology configuration.

    Eagle currently supports to customize configurations for three kinds of topologies:

    Modified: incubator/eagle/site/docs/deployment-in-production.html URL: http://svn.apache.org/viewvc/incubator/eagle/site/docs/deployment-in-production.html?rev=1743294&r1=1743293&r2=1743294&view=diff ============================================================================== --- incubator/eagle/site/docs/deployment-in-production.html (original) +++ incubator/eagle/site/docs/deployment-in-production.html Wed May 11 03:55:14 2016 @@ -119,8 +119,6 @@
  • Introduction
    • -
  • Quick Start
    • -
  • Terminology
  • Ecosystem
    • @@ -133,15 +131,21 @@
  • Installation
    • -
  • Deploy Eagle in Sandbox
    • +
  • Quick Starter(Eagle In Sandbox)
    • + +
  • Download
    • + +
  • HDFS & Hive Activity Monitoring
    • -
  • Deploy Eagle in the Production
    • +
  • JMX URL Monitoring
  • Stream Log Data into Kafka
    • -
  • Install Eagle Ambari Plugin
    • +
  • Eagle Cluster Deployment
    • -
  • Configuration
    • +
  • Eagle Configuration
    • + +
  • Enable LDAP
  • @@ -155,7 +159,17 @@
  • Data Classification Tutorial
    • -
  • Eagle LDAP Tutorial
    • +
  • + +
  • How To
    • + +
  • Use Metadata API
    • + +
  • + +
  • Development Guide
    • + +
  • Development Quick Guide
  • @@ -163,8 +177,6 @@
  • Eagle Architecture Highlights
    • -
  • Policy API Design
    • -
  • User Profile Machine Learning
  • @@ -225,9 +237,10 @@ Notice that Eagle requires you have full

  • Create a Kafka topic for importing audit log. Here is an example command to create topic sandbox_hdfs_audit_log.

    - 
    $ cd <kafka-home>
+        
    $ cd <kafka-home>
 $ bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic sandbox_hdfs_audit_log
 
    
+

  • Populate audit log into the Kafka topic created above, and refer to here on How to do it.

    @@ -244,9 +257,9 @@ $ bin/kafka-topics.sh --create --zookeep
    • -

      Edit bin/eagle-env.sh

      +

      Edit bin/eagle-env.sh

      - 
        # TODO: make sure java version is 1.7.x
+        
        # TODO: make sure java version is 1.7.x
   export JAVA_HOME=
 
   # TODO: Storm nimbus host. Default is localhost
@@ -255,11 +268,12 @@ $ bin/kafka-topics.sh --create --zookeep
   # TODO: EAGLE_SERVICE_HOST, default is `hostname -f`
   export EAGLE_SERVICE_HOST=localhost
 
      
+
    • -

      Edit conf/eagle-service.conf

      +

      Edit conf/eagle-service.conf

      - 
        # TODO: hbase.zookeeper.quorum in the format host1,host2,host3,...
+        
        # TODO: hbase.zookeeper.quorum in the format host1,host2,host3,...
   # default is "localhost"
   hbase-zookeeper-quorum="localhost"
 
@@ -271,13 +285,14 @@ $ bin/kafka-topics.sh --create --zookeep
   # default is "/hbase"
   zookeeper-znode-parent="/hbase"
 
      
+

  • Step 2: Install metadata for policies

    - 
      $ cd <eagle-home>
+    
      $ cd <eagle-home>
 
   # create HBase tables
   $ bin/eagle-service-init.sh
@@ -288,13 +303,14 @@ $ bin/kafka-topics.sh --create --zookeep
   # import metadata after Eagle service is successfully started
   $ bin/eagle-topology-init.sh
 
    
+

    • Setup a monitoring site

      -
    • Step 1: Login Eagle web http://${EAGLE_SERVICE_HOST}:9099/eagle-service with account admin/secret +
    • Step 1: Login Eagle web http://${EAGLE_SERVICE_HOST}:9099/eagle-service with account admin/secret login
    • Step 2: Create a site with Eagle web (Example: create a site “Demo” with two data sources to monitor) @@ -308,7 +324,7 @@ $ bin/kafka-topics.sh --create --zookeep

    • Step 4: Submit topologies

      - 
          # start HDFS audilt log monitoring
+    
          # start HDFS audilt log monitoring
     $ bin/eagle-topology.sh --main eagle.security.auditlog.HdfsAuditLogProcessorMain --config conf/XXXX-hdfsAuditLog-application.conf start
 
     # start Hive Query Log Monitoring
@@ -317,6 +333,7 @@ $ bin/kafka-topics.sh --create --zookeep
     # start User Profiles
     $ bin/eagle-topology.sh --main eagle.security.userprofile.UserProfileDetectionMain --config conf/XXXX-userprofile-topology.conf start
 
      
+
    @@ -343,16 +360,18 @@ $ bin/kafka-topics.sh --create --zookeep

  • Stop eagle service

    - 
    $ bin/eagle-service.sh stop
+    
    $ bin/eagle-service.sh stop
 
    
+

  • Stop eagle topologies

    - 
    $ bin/eagle-topology.sh --topology {topology-name} stop
+    
    $ bin/eagle-topology.sh --topology {topology-name} stop
 $ bin/eagle-topology.sh --topology {topology-name} stop
 $ bin/eagle-topology.sh --topology {topology-name} stop
 
    
+
    • Modified: incubator/eagle/site/docs/deployment-in-sandbox.html URL: http://svn.apache.org/viewvc/incubator/eagle/site/docs/deployment-in-sandbox.html?rev=1743294&r1=1743293&r2=1743294&view=diff ============================================================================== --- incubator/eagle/site/docs/deployment-in-sandbox.html (original) +++ incubator/eagle/site/docs/deployment-in-sandbox.html Wed May 11 03:55:14 2016 @@ -119,8 +119,6 @@
  • Introduction
    • -
  • Quick Start
    • -
  • Terminology
  • Ecosystem
    • @@ -133,15 +131,21 @@
  • Installation
    • -
  • Deploy Eagle in Sandbox
    • +
  • Quick Starter(Eagle In Sandbox)
    • + +
  • Download
    • + +
  • HDFS & Hive Activity Monitoring
    • -
  • Deploy Eagle in the Production
    • +
  • JMX URL Monitoring
  • Stream Log Data into Kafka
    • -
  • Install Eagle Ambari Plugin
    • +
  • Eagle Cluster Deployment
    • -
  • Configuration
    • +
  • Eagle Configuration
    • + +
  • Enable LDAP
  • @@ -155,7 +159,17 @@
  • Data Classification Tutorial
    • -
  • Eagle LDAP Tutorial
    • +
  • + +
  • How To
    • + +
  • Use Metadata API
    • + +
  • + +
  • Development Guide
    • + +
  • Development Quick Guide
  • @@ -163,8 +177,6 @@
  • Eagle Architecture Highlights
    • -
  • Policy API Design
    • -
  • User Profile Machine Learning
  • @@ -221,21 +233,23 @@

    Option 1: Download eagle jar from here.

  • -

    Option 2: Build form source code eagle github. After successful build, ‘eagle-xxx-bin.tar.gz’ will be generated under ./eagle-assembly/target

    +

    Option 2: Build form source code eagle github. After successful build, ‘eagle-xxx-bin.tar.gz’ will be generated under ./eagle-assembly/target

    - 
    # installed npm is required before compiling
+        
    # installed npm is required before compiling
 $ mvn clean install -DskipTests=true
 
    
+

  • Copy and extract the package to sandbox

    - 
    #extract
-$ tar -zxvf eagle-0.3.0-bin.tar.gz
-$ mv eagle-0.3.0 /usr/hdp/current/eagle
+    
    #extract
+$ tar -zxvf eagle-0.1.0-bin.tar.gz
+$ mv eagle-0.1.0 /usr/hdp/current/eagle
 
    
+
    • @@ -243,20 +257,12 @@ $ mv eagle-0.3.0 /usr/hdp/current/eagle

    Step 4: Install Eagle in Sandbox

    -

    The following installation actually contains installing and setting up a sandbox site with three data sources HdfsAuditLog, HiveQueryLog, and User Profiles

    - -
      -
    • -

      Option 1: Install Eagle using command line

      +

      The following installation actually contains installing and setting up a sandbox site with three data sources HdfsAuditLog & HiveQueryLog

      - 
      $ cd /usr/hdp/current/eagle
-$ examples/eagle-sandbox-starter.sh
+
        $ cd /usr/hdp/current/eagle
+  $ examples/eagle-sandbox-starter.sh
 
      
-
      • -
    • -

      Option 2: Install Eagle using Eagle Ambari plugin

      -
      • -
    +

    Step 5: Stream HDFS audit log

    @@ -266,7 +272,7 @@ $ examples/eagle-sandbox-starter.sh

  • Step 1: Configure Advanced hadoop-log4j via Ambari UI, and add below “KAFKA_HDFS_AUDIT” log4j appender to hdfs audit logging.

    - 
    log4j.appender.KAFKA_HDFS_AUDIT=org.apache.eagle.log4j.kafka.KafkaLog4jAppender
+    
    log4j.appender.KAFKA_HDFS_AUDIT=org.apache.eagle.log4j.kafka.KafkaLog4jAppender
 log4j.appender.KAFKA_HDFS_AUDIT.Topic=sandbox_hdfs_audit_log
 log4j.appender.KAFKA_HDFS_AUDIT.BrokerList=sandbox.hortonworks.com:6667
 log4j.appender.KAFKA_HDFS_AUDIT.KeyClass=org.apache.eagle.log4j.kafka.hadoop.AuditLogKeyer
@@ -276,22 +282,25 @@ log4j.appender.KAFKA_HDFS_AUDIT.Producer
 #log4j.appender.KAFKA_HDFS_AUDIT.BatchSize=1
 #log4j.appender.KAFKA_HDFS_AUDIT.QueueSize=1
 
    
+    
    
 
     
    HDFS LOG4J Configuration

  • Step 3: Edit Advanced hadoop-env via Ambari UI, and add the reference to KAFKA_HDFS_AUDIT to HADOOP_NAMENODE_OPTS.

    - 
    -Dhdfs.audit.logger=INFO,DRFAAUDIT,KAFKA_HDFS_AUDIT
+    
    -Dhdfs.audit.logger=INFO,DRFAAUDIT,KAFKA_HDFS_AUDIT
 
    
+    
    
 
     
    HDFS Environment Configuration

  • Step 4: Edit Advanced hadoop-env via Ambari UI, and append the following command to it.

    - 
    export HADOOP_CLASSPATH=${HADOOP_CLASSPATH}:/usr/hdp/current/eagle/lib/log4jkafka/lib/*
+    
    export HADOOP_CLASSPATH=${HADOOP_CLASSPATH}:/usr/hdp/current/eagle/lib/log4jkafka/lib/*
 
    
+    
    
 
     
    HDFS Environment Configuration
    • @@ -299,14 +308,15 @@ log4j.appender.KAFKA_HDFS_AUDIT.Producer

    Step 5: save the changes and restart the namenode.

  • -

    Step 6: Check whether logs are flowing into topic sandbox_hdfs_audit_log

    +

    Step 6: Check whether logs are flowing into topic sandbox_hdfs_audit_log

    - 
    $ /usr/hdp/2.2.4.2-2/kafka/bin/kafka-console-consumer.sh --zookeeper sandbox.hortonworks.com:2181 --topic sandbox_hdfs_audit_log
+    
    $ /usr/hdp/current/kafka-broker/bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic sandbox_hdfs_audit_log
 
    
+
    • -

    Now please login to Eagle web http://localhost:9099/eagle-service with account admin/secret, and try the sample demos on +

    Now please login to Eagle web http://localhost:9099/eagle-service with account admin/secret, and try the sample demos on Quick Starer

    (If the NAT network is used in a virtual machine, it’s required to add port 9099 to forwarding ports) Added: incubator/eagle/site/docs/development-quick-guide.html URL: http://svn.apache.org/viewvc/incubator/eagle/site/docs/development-quick-guide.html?rev=1743294&view=auto ============================================================================== --- incubator/eagle/site/docs/development-quick-guide.html (added) +++ incubator/eagle/site/docs/development-quick-guide.html Wed May 11 03:55:14 2016 @@ -0,0 +1,275 @@ + + + + + + Eagle - Development Quick Guide + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

    +
    +
    +
    +
    + + +
    +
    +
    +
    +
    +
      +
    • Slide 1
      • +
    +
    +
    +
    +
    +
    +
    + + + +
    +
    +

    Secure Hadoop in Real Time

    + +
    +
    +
    + + +
    + Fork eagle on GitHub + +
    +
    + + +
    +

    Development Quick Guide

    +

    Source Code

    + +

    Apache Eagle source code is version controlled using Git version control: +Commits Summary

    + + + +

    Issue Tracking

    + +

    Track issues on the “Eagle” Project on the Apache JIRA (browse). +

    + +

    Setup Development Env

    + +

    Refer to this link for how to setup dev env: Setup Development Env +

    + +

    How to Contribute

    + +

    Refer to this link for how to contribute: Contributing to Eagle

    + +


    + +
    +
    + +
    +
    + + + +
    +
    +
    +
    +
    +Apache Eagle is an effort undergoing incubation at The Apache Software Foundation (ASF), sponsored by the Apache Incubator. Incubation is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with other successful ASF projects. While incubation status is not necessarily a reflection of the completeness or stability of the code, it does indicate that the project has yet to be fully endorsed by the ASF. +
    +
    + + + + + + +
    +
    +Copyright © 2015 The Apache Software Foundation, Licensed under the Apache License, Version 2.0. +
    +
    +Apache Eagle, Eagle, Apache, the Apache feather logo, and the Apache Incubator project logo are trademarks of The Apache Software Foundation. +
    +
    +
    +
    +
    + + + + + + + + + + + + + \ No newline at end of file Propchange: incubator/eagle/site/docs/development-quick-guide.html ------------------------------------------------------------------------------ svn:eol-style = native Added: incubator/eagle/site/docs/download.html URL: http://svn.apache.org/viewvc/incubator/eagle/site/docs/download.html?rev=1743294&view=auto ============================================================================== --- incubator/eagle/site/docs/download.html (added) +++ incubator/eagle/site/docs/download.html Wed May 11 03:55:14 2016 @@ -0,0 +1,259 @@ + + + + + + Eagle - Apache Eagle Downloads + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    +
    +
    +
    +
    + + +
    +
    +
    +
    +
    +
      +
    • Slide 1
      • +
    +
    +
    +
    +
    +
    +
    + + + +
    +
    +

    Secure Hadoop in Real Time

    + +
    +
    +
    + + +
    + Fork eagle on GitHub + +
    +
    + + +
    +

    Apache Eagle Downloads

    +

    Latest Release (Source Code)

    + + + +


    + +
    +
    + +
    +
    + + + +
    +
    +
    +
    +
    +Apache Eagle is an effort undergoing incubation at The Apache Software Foundation (ASF), sponsored by the Apache Incubator. Incubation is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with other successful ASF projects. While incubation status is not necessarily a reflection of the completeness or stability of the code, it does indicate that the project has yet to be fully endorsed by the ASF. +
    +
    + + + + + + +
    +
    +Copyright © 2015 The Apache Software Foundation, Licensed under the Apache License, Version 2.0. +
    +
    +Apache Eagle, Eagle, Apache, the Apache feather logo, and the Apache Incubator project logo are trademarks of The Apache Software Foundation. +
    +
    +
    +
    +
    + + + + + + + + + + + + + \ No newline at end of file Propchange: incubator/eagle/site/docs/download.html ------------------------------------------------------------------------------ svn:eol-style = native Modified: incubator/eagle/site/docs/ecosystem.html URL: http://svn.apache.org/viewvc/incubator/eagle/site/docs/ecosystem.html?rev=1743294&r1=1743293&r2=1743294&view=diff ============================================================================== --- incubator/eagle/site/docs/ecosystem.html (original) +++ incubator/eagle/site/docs/ecosystem.html Wed May 11 03:55:14 2016 @@ -119,8 +119,6 @@
  • Introduction
    • -
  • Quick Start
    • -
  • Terminology
  • Ecosystem
    • @@ -133,15 +131,21 @@
  • Installation
    • -
  • Deploy Eagle in Sandbox
    • +
  • Quick Starter(Eagle In Sandbox)
    • + +
  • Download
    • + +
  • HDFS & Hive Activity Monitoring
    • -
  • Deploy Eagle in the Production
    • +
  • JMX URL Monitoring
  • Stream Log Data into Kafka
    • -
  • Install Eagle Ambari Plugin
    • +
  • Eagle Cluster Deployment
    • -
  • Configuration
    • +
  • Eagle Configuration
    • + +
  • Enable LDAP
  • @@ -155,7 +159,17 @@
  • Data Classification Tutorial
    • -
  • Eagle LDAP Tutorial
    • +
  • + +
  • How To
    • + +
  • Use Metadata API
    • + +
  • + +
  • Development Guide
    • + +
  • Development Quick Guide
  • @@ -163,8 +177,6 @@
  • Eagle Architecture Highlights
    • -
  • Policy API Design
    • -
  • User Profile Machine Learning