eagle-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From yonzhang2...@apache.org
Subject [7/7] incubator-eagle git commit: EAGLE-334 clean user profile code clean user profile code in Eagle 0.5 and redesign in Eagle 0.6
Date Fri, 10 Jun 2016 01:29:32 GMT
EAGLE-334 clean user profile code
clean user profile code in Eagle 0.5 and redesign in Eagle 0.6

Author: @yonzhang2012 <yonzhang2012@apache.org>
Reviewer:

Closes: #233


Project: http://git-wip-us.apache.org/repos/asf/incubator-eagle/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-eagle/commit/8b646715
Tree: http://git-wip-us.apache.org/repos/asf/incubator-eagle/tree/8b646715
Diff: http://git-wip-us.apache.org/repos/asf/incubator-eagle/diff/8b646715

Branch: refs/heads/develop
Commit: 8b646715edbe8f0e3a4a2d926c10e4a942117085
Parents: eef4930
Author: yonzhang <yonzhang2012@gmail.com>
Authored: Thu Jun 9 18:31:58 2016 -0700
Committer: yonzhang <yonzhang2012@gmail.com>
Committed: Thu Jun 9 18:31:58 2016 -0700

----------------------------------------------------------------------
 .../eagle-security-hdfs-authlog/README.md       |   42 +
 .../eagle-security-userprofile/common/pom.xml   |  127 --
 .../common/run_mltraining_topology.sh           |   19 -
 .../security/userprofile/UserPartitioner.java   |   32 -
 .../userprofile/UserProfileConstants.java       |   37 -
 .../security/userprofile/UserProfileUtils.java  |   53 -
 .../model/ScheduleCommandEntity.java            |   73 -
 .../model/UserActivityAggModelDeserializer.java |   52 -
 .../model/UserActivityAggModelEntity.java       |   86 -
 .../model/UserCommandStatistics.java            |   83 -
 .../model/UserProfileEigenModelEntity.java      |  204 --
 .../model/UserProfileEntityRepository.java      |   25 -
 .../model/UserProfileKDEModelEntity.java        |  129 --
 .../userprofile/model/package-info.java         |   20 -
 .../common/src/main/resources/log4j.properties  |   35 -
 .../common/src/main/resources/storm.yaml.1      |   29 -
 .../userprofile/model/AuditlogParser.scala      |   54 -
 .../userprofile/model/EntityConversion.scala    |   27 -
 .../model/UserActivityAggModel.scala            |   28 -
 .../model/UserProfileEigenModel.scala           |   44 -
 .../userprofile/model/UserProfileKDEModel.scala |   37 -
 .../userprofile/model/UserProfileModel.scala    |   22 -
 .../common/TestUserProfileUtils.java            |  106 -
 .../common/src/test/resources/log4j.properties  |   35 -
 .../common/src/test/resources/storm.yaml.2      |   29 -
 .../detection/dev-supports/prepare-metadata.sh  |   45 -
 .../detection/dev-supports/prepare-table.sh     |   21 -
 .../produce-useractivity-with-spark.sh          |   18 -
 .../dev-supports/produce-useractivity.sh        |   18 -
 .../dev-supports/useractivity-agg-json.txt      |   16 -
 .../userprofile-executor-definition.json        |    1 -
 .../userprofile-policy-definition.json          |    1 -
 .../userprofile-stream-description.json         |    1 -
 .../userprofile-stream-metadata.json            |    1 -
 .../detection/pom.xml                           |   86 -
 .../detection/run_onlineprediction_topology.sh  |   19 -
 .../eagle/security/userprofile/TimeWindow.java  |   45 -
 .../userprofile/UserActivityAggregator.java     |   34 -
 .../UserProfileAggregatorExecutor.java          |   79 -
 .../userprofile/UserProfileAnomalyDetector.java |   44 -
 .../UserProfileDetectionBatchMain.java          |   68 -
 .../UserProfileDetectionConstants.java          |   25 -
 .../userprofile/UserProfileDetectionMain.java   |   43 -
 .../UserProfileDetectionStreamMain.java         |   56 -
 .../UserProfileMLAlgorithmEvaluator.java        |  118 --
 .../impl/AbstractUserProfileEigenEvaluator.java |   50 -
 .../impl/AbstractUserProfileKDEEvaluator.java   |   40 -
 .../userprofile/impl/TimeWindowImpl.java        |   87 -
 .../impl/UserActivityAggregatorImpl.java        |  169 --
 .../impl/UserProfileAnomalyEigenEvaluator.java  |  173 --
 .../impl/UserProfileAnomalyKDEEvaluator.java    |  142 --
 ....eagle.policy.PolicyEvaluatorServiceProvider |   16 -
 .../src/main/resources/application-batch.conf   |   63 -
 .../src/main/resources/application-stream.conf  |   65 -
 .../src/main/resources/log4j.properties         |   35 -
 .../main/resources/ml-policyDef-UserProfile.txt |   33 -
 .../src/main/resources/samplePolicy.json        |   18 -
 .../src/main/resources/userprofile.storm.yaml   |   29 -
 .../userprofile/TestUserActivityAggregator.java |  104 -
 .../TestUserProfileAnomalyEigenEvaluator.java   |  115 --
 ....eagle.policy.PolicyEvaluatorServiceProvider |   16 -
 .../test/resources/ml-policyDef-UserProfile.txt |    1 -
 .../detection/src/test/resources/storm.yaml.1   |   29 -
 .../eagle-security-userprofile/pom.xml          |   37 -
 .../training/README.md                          |   54 -
 .../training/dev-supports/run-scheduler.sh      |   48 -
 .../training/dev-supports/run-training.sh       |   25 -
 .../training/dev-supports/run-with-kafka.sh     |   30 -
 .../training/dev-supports/run-with-localfile.sh |   33 -
 .../training/dev-supports/run-with-service.sh   |   33 -
 .../training/dev-supports/run-with-spark.sh     |   52 -
 .../dev-supports/trigger-onetime-training.sh    |   23 -
 .../eagle-security-userprofile/training/pom.xml |  219 --
 .../training/src/assembly/assembly.xml          |   67 -
 .../model/JavaUserProfileModeler.java           |   50 -
 .../model/eigen/UserProfileEigenModeler.java    |  297 ---
 .../model/kde/UserProfileKDEModeler.java        |  147 --
 .../src/main/resources/application.conf         |   36 -
 .../training/src/main/resources/hdfs-audit.log  | 1920 ------------------
 .../src/main/resources/log4j.properties         |   35 -
 .../training/src/main/resources/reference.conf  |   35 -
 .../main/sbin/submit-userprofile-training.sh    |   23 -
 .../userprofile/UserProfileJobFactory.scala     |   31 -
 .../userprofile/UserProfileTrainingApp.scala    |   80 -
 .../userprofile/UserProfileTrainingCLI.scala    |   58 -
 .../security/userprofile/daemon/Command.scala   |  116 --
 .../security/userprofile/daemon/Consumers.scala |   83 -
 .../userprofile/daemon/Coordinator.scala        |   58 -
 .../security/userprofile/daemon/Producers.scala |  154 --
 .../userprofile/daemon/SchedulePolicy.scala     |  100 -
 .../security/userprofile/daemon/Scheduler.scala |   62 -
 .../userprofile/daemon/SchedulerContext.scala   |  143 --
 .../daemon/UserProfileCommandDao.scala          |   90 -
 .../security/userprofile/daemon/Utils.scala     |   77 -
 .../job/AuditLogTrainingSparkJob.scala          |  157 --
 .../job/UserProfileTrainingJob.scala            |   64 -
 .../userprofile/model/UserProfileContext.scala  |   22 -
 .../userprofile/model/UserProfileModeler.scala  |   36 -
 .../userprofile/sink/UserActivityAggSink.scala  |   84 -
 .../userprofile/sink/UserProfileModelSink.scala |   89 -
 .../model/UserProfileEigenModelerTest.java      |   56 -
 .../model/UserProfileKDEModelerTest.java        |   54 -
 .../security/userprofile/daemon/UtilsSpec.scala |   39 -
 eagle-security/pom.xml                          |    1 -
 104 files changed, 42 insertions(+), 8128 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/incubator-eagle/blob/8b646715/eagle-security/eagle-security-hdfs-authlog/README.md
----------------------------------------------------------------------
diff --git a/eagle-security/eagle-security-hdfs-authlog/README.md b/eagle-security/eagle-security-hdfs-authlog/README.md
index d77ccf7..e1a1971 100644
--- a/eagle-security/eagle-security-hdfs-authlog/README.md
+++ b/eagle-security/eagle-security-hdfs-authlog/README.md
@@ -27,3 +27,45 @@ limitations under the License.
 
 #### consume parsed hdfs authorization log
 /usr/hdp/2.2.4.2-2/kafka/bin/kafka-console-consumer.sh --topic sandbox_hdfs_auth_log --zookeeper sandbox.hortonworks.com:2181
+
+
+#### run logstash in sandbox to fetch Hdfs authorization log
+
+##### step 1: create logstash config file: hdfs-authlog.conf
+
+~~~
+    input {
+        file {
+            type => "hdfs-auth"
+            path => "/var/log/hadoop/hdfs/SecurityAuth.audit"
+            start_position => end
+            sincedb_path => "/var/log/logstash/sincedb"
+        }
+    }
+
+    output {
+         if [type] == "hdfs-auth" {
+              kafka {
+                  codec => plain {
+                      format => "%{message}"
+                  }
+                  bootstrap_servers => "sandbox.hortonworks.com:6667"
+                  topic_id => "sandbox_hdfs_auth_log"
+                  acks => "0"
+                  timeout_ms => 10000
+                  retries => 3
+                  retry_backoff_ms => 100
+                  batch_size => 16384
+                  send_buffer_bytes => 131072
+                  client_id => "hdfs-auth"
+              }
+              # stdout { codec => rubydebug }
+          }
+    }
+
+~~~
+
+##### step 2: run logstash
+
+bin/logstash -f hdfs-authlog.conf
+

http://git-wip-us.apache.org/repos/asf/incubator-eagle/blob/8b646715/eagle-security/eagle-security-userprofile/common/pom.xml
----------------------------------------------------------------------
diff --git a/eagle-security/eagle-security-userprofile/common/pom.xml b/eagle-security/eagle-security-userprofile/common/pom.xml
deleted file mode 100644
index 1faee1a..0000000
--- a/eagle-security/eagle-security-userprofile/common/pom.xml
+++ /dev/null
@@ -1,127 +0,0 @@
-<?xml version="1.0" encoding="UTF-8"?>
-<!--
-  ~ Licensed to the Apache Software Foundation (ASF) under one or more
-  ~ contributor license agreements.  See the NOTICE file distributed with
-  ~ this work for additional information regarding copyright ownership.
-  ~ The ASF licenses this file to You under the Apache License, Version 2.0
-  ~ (the "License"); you may not use this file except in compliance with
-  ~ the License.  You may obtain a copy of the License at
-  ~
-  ~    http://www.apache.org/licenses/LICENSE-2.0
-  ~
-  ~ Unless required by applicable law or agreed to in writing, software
-  ~ distributed under the License is distributed on an "AS IS" BASIS,
-  ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-  ~ See the License for the specific language governing permissions and
-  ~ limitations under the License.
-  -->
-
-<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
-  <modelVersion>4.0.0</modelVersion>
-  <parent>
-    <groupId>org.apache.eagle</groupId>
-    <artifactId>eagle-security-userprofile-parent</artifactId>
-    <version>0.5.0-incubating-SNAPSHOT</version>
-      <relativePath>../pom.xml</relativePath>
-  </parent>
-  <packaging>jar</packaging>
-  <artifactId>eagle-security-userprofile-common</artifactId>
-  <name>eagle-security-userprofile-common</name>
-  <url>http://maven.apache.org</url>
-
-    <dependencies>
-        <dependency>
-            <groupId>com.googlecode.efficient-java-matrix-library</groupId>
-            <artifactId>ejml</artifactId>
-        </dependency>
-        <dependency>
-            <groupId>org.apache.eagle</groupId>
-            <artifactId>eagle-security-common</artifactId>
-            <version>${project.version}</version>
-            <exclusions>
-                <exclusion>
-                    <artifactId>joda-time</artifactId>
-                    <groupId>joda-time</groupId>
-                </exclusion>
-            </exclusions>
-        </dependency>
-        <dependency>
-          <groupId>org.apache.eagle</groupId>
-          <artifactId>eagle-alert-process</artifactId>
-            <version>${project.version}</version>
-        </dependency>
-        <dependency>
-          <groupId>org.apache.eagle</groupId>
-          <artifactId>eagle-machinelearning-base</artifactId>
-            <version>${project.version}</version>
-        </dependency>
-        <dependency>
-            <groupId>joda-time</groupId>
-            <artifactId>joda-time</artifactId>
-        </dependency>
-        <dependency>
-            <groupId>org.joda</groupId>
-            <artifactId>joda-convert</artifactId>
-        </dependency>
-    </dependencies>
-
-    <build>
-        <!--<sourceDirectory>.</sourceDirectory>-->
-        <!--<testSourceDirectory>src/test/scala</testSourceDirectory>-->
-        <resources>
-            <resource>
-                <directory>src/main/resources</directory>
-            </resource>
-        </resources>
-
-        <plugins>
-            <plugin>
-                <groupId>org.scala-tools</groupId>
-                <artifactId>maven-scala-plugin</artifactId>
-                <executions>
-                    <execution>
-                        <id>scala-compile-first</id>
-                        <phase>process-resources</phase>
-                        <goals>
-                            <goal>add-source</goal>
-                            <goal>compile</goal>
-                        </goals>
-                    </execution>
-                    <execution>
-                        <id>scala-test-compile</id>
-                        <phase>process-test-resources</phase>
-                        <goals>
-                            <goal>testCompile</goal>
-                        </goals>
-                    </execution>
-                </executions>
-            </plugin>
-            <plugin>
-                <groupId>org.apache.maven.plugins</groupId>
-                <artifactId>maven-compiler-plugin</artifactId>
-                <executions>
-                    <execution>
-                        <phase>compile</phase>
-                        <goals>
-                            <goal>compile</goal>
-                        </goals>
-                    </execution>
-                </executions>
-            </plugin>
-
-            <plugin>
-                <groupId>org.apache.maven.plugins</groupId>
-                <artifactId>maven-surefire-plugin</artifactId>
-                <configuration>
-                    <useFile>false</useFile>
-                    <disableXmlReport>true</disableXmlReport>
-                    <includes>
-                        <include>**/*Test.*</include>
-                        <include>**/Test*.*</include>
-                        <include>**/*Suite.*</include>
-                    </includes>
-                </configuration>
-            </plugin>
-        </plugins>
-    </build>
-</project>

http://git-wip-us.apache.org/repos/asf/incubator-eagle/blob/8b646715/eagle-security/eagle-security-userprofile/common/run_mltraining_topology.sh
----------------------------------------------------------------------
diff --git a/eagle-security/eagle-security-userprofile/common/run_mltraining_topology.sh b/eagle-security/eagle-security-userprofile/common/run_mltraining_topology.sh
deleted file mode 100755
index dfab19d..0000000
--- a/eagle-security/eagle-security-userprofile/common/run_mltraining_topology.sh
+++ /dev/null
@@ -1,19 +0,0 @@
-#!/usr/bin/env bash
-
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements.  See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License.  You may obtain a copy of the License at
-#
-#    http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-export JAVA_HOME=/Library/Java/JavaVirtualMachines/jdk1.7.0_51.jdk/Contents/Home
-mvn -X exec:java -Dexec.mainClass="eagle.auditlog.userprofile.training.AuditLogUserProfileMain"

http://git-wip-us.apache.org/repos/asf/incubator-eagle/blob/8b646715/eagle-security/eagle-security-userprofile/common/src/main/java/org/apache/eagle/security/userprofile/UserPartitioner.java
----------------------------------------------------------------------
diff --git a/eagle-security/eagle-security-userprofile/common/src/main/java/org/apache/eagle/security/userprofile/UserPartitioner.java b/eagle-security/eagle-security-userprofile/common/src/main/java/org/apache/eagle/security/userprofile/UserPartitioner.java
deleted file mode 100644
index 1a6023b..0000000
--- a/eagle-security/eagle-security-userprofile/common/src/main/java/org/apache/eagle/security/userprofile/UserPartitioner.java
+++ /dev/null
@@ -1,32 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements.  See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License.  You may obtain a copy of the License at
- *
- *    http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-package org.apache.eagle.security.userprofile;
-
-import kafka.utils.VerifiableProperties;
-import kafka.utils.Utils;
-
-public class UserPartitioner implements kafka.producer.Partitioner{
-	public UserPartitioner(VerifiableProperties prop){
-		
-	}
-
-	@Override
-	public int partition(Object arg0, int arg1) {
-		String user = (String)arg0;
-		return  Utils.abs(user.hashCode()) % arg1;
-	}
-}
\ No newline at end of file

http://git-wip-us.apache.org/repos/asf/incubator-eagle/blob/8b646715/eagle-security/eagle-security-userprofile/common/src/main/java/org/apache/eagle/security/userprofile/UserProfileConstants.java
----------------------------------------------------------------------
diff --git a/eagle-security/eagle-security-userprofile/common/src/main/java/org/apache/eagle/security/userprofile/UserProfileConstants.java b/eagle-security/eagle-security-userprofile/common/src/main/java/org/apache/eagle/security/userprofile/UserProfileConstants.java
deleted file mode 100644
index 4700cf3..0000000
--- a/eagle-security/eagle-security-userprofile/common/src/main/java/org/apache/eagle/security/userprofile/UserProfileConstants.java
+++ /dev/null
@@ -1,37 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements.  See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License.  You may obtain a copy of the License at
- *
- *    http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-package org.apache.eagle.security.userprofile;
-
-import org.apache.eagle.ml.MLConstants;
-
-import java.io.Serializable;
-
-public final class UserProfileConstants implements Serializable{
-    public static final String[] DEFAULT_CMD_TYPES= new String[]{"getfileinfo", "open", "listStatus", "setTimes", "setPermission", "rename", "mkdirs", "create", "setReplication", "contentSummary", "delete", "setOwner", "fsck"};
-    public static final String DEFAULT_TRAINING_APP_NAME = "AuditlogTraining";
-
-    public static final String USER_PROFILE_EIGEN_MODEL_SERVICE = "UserProfileEigenModelService";
-    public static final String USER_PROFILE_KDE_MODEL_SERVICE = "UserProfileKDEModelService";
-    public static final String USER_ACTIVITY_AGG_MODEL_SERVICE = "UserActivityAggModelService";
-
-    public static final String EIGEN_DECOMPOSITION_ALGORITHM = "EigenDecomposition";
-    public static final String KDE_ALGORITHM = "DE";
-
-    public static final String USER_TAG = "user";
-    public static final String SITE_TAG = "site";
-    public static final String ALGORITHM_TAG = MLConstants.ALGORITHM;
-}
\ No newline at end of file

http://git-wip-us.apache.org/repos/asf/incubator-eagle/blob/8b646715/eagle-security/eagle-security-userprofile/common/src/main/java/org/apache/eagle/security/userprofile/UserProfileUtils.java
----------------------------------------------------------------------
diff --git a/eagle-security/eagle-security-userprofile/common/src/main/java/org/apache/eagle/security/userprofile/UserProfileUtils.java b/eagle-security/eagle-security-userprofile/common/src/main/java/org/apache/eagle/security/userprofile/UserProfileUtils.java
deleted file mode 100644
index 00d8e28..0000000
--- a/eagle-security/eagle-security-userprofile/common/src/main/java/org/apache/eagle/security/userprofile/UserProfileUtils.java
+++ /dev/null
@@ -1,53 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements.  See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License.  You may obtain a copy of the License at
- *
- *    http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-package org.apache.eagle.security.userprofile;
-
-import org.joda.time.Period;
-import org.joda.time.Seconds;
-import scala.Int;
-
-public class UserProfileUtils {
-    /**
-     * For example: timestamp stands for time: 1990/01/07 12:45 and period is PT30, then result is 1990/01/07 12:30
-     *
-     * @param seconds
-     * @param period
-     *
-     * @return formatted timestamp
-     */
-    public static long formatSecondsByPeriod(long seconds,Seconds period){
-        return seconds - (seconds % Int.int2long(period.getSeconds()));
-    }
-
-    /**
-     * @param seconds
-     * @param period
-     * @return
-     */
-    public static long formatSecondsByPeriod(long seconds,Period period){
-        return seconds - (seconds % Int.int2long(period.toStandardSeconds().getSeconds()));
-    }
-
-    /**
-     * @param milliseconds
-     * @param period
-     * @return milliseconds
-     */
-    public static long formatMillisecondsByPeriod(long milliseconds,Period period){
-        return formatSecondsByPeriod(milliseconds/1000,period)*1000;
-    }
-}
\ No newline at end of file

http://git-wip-us.apache.org/repos/asf/incubator-eagle/blob/8b646715/eagle-security/eagle-security-userprofile/common/src/main/java/org/apache/eagle/security/userprofile/model/ScheduleCommandEntity.java
----------------------------------------------------------------------
diff --git a/eagle-security/eagle-security-userprofile/common/src/main/java/org/apache/eagle/security/userprofile/model/ScheduleCommandEntity.java b/eagle-security/eagle-security-userprofile/common/src/main/java/org/apache/eagle/security/userprofile/model/ScheduleCommandEntity.java
deleted file mode 100644
index 9727572..0000000
--- a/eagle-security/eagle-security-userprofile/common/src/main/java/org/apache/eagle/security/userprofile/model/ScheduleCommandEntity.java
+++ /dev/null
@@ -1,73 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements.  See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License.  You may obtain a copy of the License at
- *
- *    http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-package org.apache.eagle.security.userprofile.model;
-
-import org.apache.eagle.log.base.taggedlog.TaggedLogAPIEntity;
-import org.apache.eagle.log.entity.meta.*;
-
-@Service(ScheduleCommandEntity.ScheduleTaskService)
-@Table("userprofile")
-@Prefix("schedule_command")
-@ColumnFamily("f")
-@TimeSeries(true)
-@Tags({"site","type"})
-public class ScheduleCommandEntity extends TaggedLogAPIEntity {
-    public static final String ScheduleTaskService = "ScheduleTaskService";
-    public static enum STATUS {
-        INITIALIZED, PENDING,EXECUTING, SUCCEEDED, FAILED
-        // , CANCELED
-    }
-
-    @Column("a")
-    private String status;
-    public static String getScheduleTaskService() {
-        return ScheduleTaskService;
-    }
-
-    public String getStatus() {
-        return status;
-    }
-
-    public void setStatus(String status) {
-        this.status = status;
-        valueChanged("status");
-    }
-
-    public long getUpdateTime() {
-        return updateTime;
-    }
-
-    public void setUpdateTime(long updateTime) {
-        this.updateTime = updateTime;
-        valueChanged("updateTime");
-    }
-
-    public String getDetail() {
-        return detail;
-    }
-
-    public void setDetail(String detail) {
-        this.detail = detail;
-        valueChanged("detail");
-    }
-
-    @Column("b")
-    private String detail;
-    @Column("c")
-    private long updateTime;
-
-}
\ No newline at end of file

http://git-wip-us.apache.org/repos/asf/incubator-eagle/blob/8b646715/eagle-security/eagle-security-userprofile/common/src/main/java/org/apache/eagle/security/userprofile/model/UserActivityAggModelDeserializer.java
----------------------------------------------------------------------
diff --git a/eagle-security/eagle-security-userprofile/common/src/main/java/org/apache/eagle/security/userprofile/model/UserActivityAggModelDeserializer.java b/eagle-security/eagle-security-userprofile/common/src/main/java/org/apache/eagle/security/userprofile/model/UserActivityAggModelDeserializer.java
deleted file mode 100644
index fc9c8ad..0000000
--- a/eagle-security/eagle-security-userprofile/common/src/main/java/org/apache/eagle/security/userprofile/model/UserActivityAggModelDeserializer.java
+++ /dev/null
@@ -1,52 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements.  See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License.  You may obtain a copy of the License at
- *
- *    http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-package org.apache.eagle.security.userprofile.model;
-
-import org.apache.eagle.dataproc.impl.storm.kafka.SpoutKafkaMessageDeserializer;
-import org.apache.eagle.log.base.taggedlog.TaggedLogAPIEntity;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
-
-import java.io.IOException;
-import java.io.UnsupportedEncodingException;
-import java.util.Properties;
-
-public class UserActivityAggModelDeserializer implements SpoutKafkaMessageDeserializer {
-    private static Logger LOG = LoggerFactory.getLogger(UserActivityAggModelDeserializer.class);
-    private final Properties props;
-    private final static String CHARSET = "UTF-8";
-
-    public  UserActivityAggModelDeserializer(Properties props){
-        this.props = props;
-    }
-
-    @Override
-    public Object deserialize(byte[] bytes) {
-        String str = null;
-        try {
-            str = new String(bytes,CHARSET);
-            return TaggedLogAPIEntity.buildObjectMapper().readValue(str, UserActivityAggModelEntity.class);
-        } catch (UnsupportedEncodingException e) {
-            LOG.error("Not support encoding: " + CHARSET, e);
-            throw new IllegalStateException("Not support "+CHARSET,e);
-        } catch (IOException e) {
-            LOG.warn("Failed to deserialize as UserActivityAggModel, ignored: "+str,e);
-//            throw new IllegalArgumentException("Failed to deserialize: "+str,e);
-        }
-        return null;
-    }
-}
\ No newline at end of file

http://git-wip-us.apache.org/repos/asf/incubator-eagle/blob/8b646715/eagle-security/eagle-security-userprofile/common/src/main/java/org/apache/eagle/security/userprofile/model/UserActivityAggModelEntity.java
----------------------------------------------------------------------
diff --git a/eagle-security/eagle-security-userprofile/common/src/main/java/org/apache/eagle/security/userprofile/model/UserActivityAggModelEntity.java b/eagle-security/eagle-security-userprofile/common/src/main/java/org/apache/eagle/security/userprofile/model/UserActivityAggModelEntity.java
deleted file mode 100644
index bbb0edf..0000000
--- a/eagle-security/eagle-security-userprofile/common/src/main/java/org/apache/eagle/security/userprofile/model/UserActivityAggModelEntity.java
+++ /dev/null
@@ -1,86 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements.  See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License.  You may obtain a copy of the License at
- *
- *    http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-package org.apache.eagle.security.userprofile.model;
-
-import org.apache.commons.math3.linear.Array2DRowRealMatrix;
-import org.apache.eagle.log.base.taggedlog.TaggedLogAPIEntity;
-import org.apache.eagle.log.entity.meta.*;
-import org.apache.eagle.security.userprofile.UserProfileConstants;
-import org.codehaus.jackson.annotate.JsonIgnoreProperties;
-import org.codehaus.jackson.map.annotate.JsonSerialize;
-import scala.collection.JavaConversions;
-
-import java.util.HashMap;
-import java.util.List;
-import java.util.Map;
-
-@JsonSerialize(include=JsonSerialize.Inclusion.NON_NULL)
-@Table("userprofile")
-@ColumnFamily("f")
-@Prefix("useractivity")
-@Service(UserProfileConstants.USER_ACTIVITY_AGG_MODEL_SERVICE)
-@TimeSeries(true)
-@JsonIgnoreProperties(ignoreUnknown = true)
-@Tags({"site","user"})
-public class UserActivityAggModelEntity extends TaggedLogAPIEntity{
-    @Column("a")
-    private List<String> cmdTypes;
-    @Column("b")
-    private double[][] cmdMatrix;
-
-    public double[][] getCmdMatrix() {
-        return cmdMatrix;
-    }
-
-    public void setCmdMatrix(double[][] cmdMatrix) {
-        this.cmdMatrix = cmdMatrix;
-        valueChanged("cmdMatrix");
-    }
-
-    public List<String> getCmdTypes() {
-        return cmdTypes;
-    }
-
-    public void setCmdTypes(List<String> cmdTypes) {
-        this.cmdTypes = cmdTypes;
-        valueChanged("cmdTypes");
-    }
-
-    public static UserActivityAggModel toModel(final UserActivityAggModelEntity entity){
-        //return null;
-        UserActivityAggModel model = new UserActivityAggModel(
-                entity.getTags().get(UserProfileConstants.USER_TAG),
-                new Array2DRowRealMatrix(entity.getCmdMatrix()),
-                JavaConversions.<String>asScalaBuffer(entity.getCmdTypes()),
-                entity.getTags().get(UserProfileConstants.SITE_TAG),
-                entity.getTimestamp());
-        return model;
-    }
-
-    public static UserActivityAggModelEntity fromModel(final UserActivityAggModel model){
-        UserActivityAggModelEntity entity = new UserActivityAggModelEntity();
-        entity.setCmdTypes(JavaConversions.seqAsJavaList(model.cmdTypes()));
-        entity.setCmdMatrix(model.matrix().getData());
-        Map<String,String> tags = new HashMap<String,String>(){{
-            put(UserProfileConstants.SITE_TAG,model.site());
-            put(UserProfileConstants.USER_TAG,model.user());
-        }};
-        entity.setTimestamp(model.timestamp());
-        entity.setTags(tags);
-        return entity;
-    }
-}
\ No newline at end of file

http://git-wip-us.apache.org/repos/asf/incubator-eagle/blob/8b646715/eagle-security/eagle-security-userprofile/common/src/main/java/org/apache/eagle/security/userprofile/model/UserCommandStatistics.java
----------------------------------------------------------------------
diff --git a/eagle-security/eagle-security-userprofile/common/src/main/java/org/apache/eagle/security/userprofile/model/UserCommandStatistics.java b/eagle-security/eagle-security-userprofile/common/src/main/java/org/apache/eagle/security/userprofile/model/UserCommandStatistics.java
deleted file mode 100644
index 2e5838c..0000000
--- a/eagle-security/eagle-security-userprofile/common/src/main/java/org/apache/eagle/security/userprofile/model/UserCommandStatistics.java
+++ /dev/null
@@ -1,83 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements.  See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License.  You may obtain a copy of the License at
- *
- *    http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-package org.apache.eagle.security.userprofile.model;
-
-import java.io.Serializable;
-import java.util.HashMap;
-import java.util.Map;
-
-public class UserCommandStatistics implements Serializable {
-    private static final long serialVersionUID = -7145615521424420913L;
-    private String commandName;
-    private double mean;
-    private double stddev;
-    private boolean isLowVariant;
-
-    public UserCommandStatistics(){}
-
-    public UserCommandStatistics(Map<String, Object> rawMap){
-        if(rawMap.get(COMMAND_NAME)!=null) this.commandName = (String) rawMap.get(COMMAND_NAME);
-        if(rawMap.get(MEAN)!=null)this.mean = (double) rawMap.get(MEAN);
-        if(rawMap.get(STDDEV)!=null) this.stddev = (double) rawMap.get(STDDEV);
-        if(rawMap.get(IS_LOW_VARIANT)!=null) this.isLowVariant = (boolean) rawMap.get(IS_LOW_VARIANT);
-    }
-
-    public String getCommandName() {
-        return commandName;
-    }
-    public void setCommandName(String commandName) {
-        this.commandName = commandName;
-    }
-
-    public double getMean() {
-        return mean;
-    }
-
-    public void setMean(double mean) {
-        this.mean = mean;
-    }
-
-    public double getStddev() {
-        return stddev;
-    }
-
-    public void setStddev(double stddev) {
-        this.stddev = stddev;
-    }
-
-    public boolean isLowVariant() {
-        return isLowVariant;
-    }
-
-    public void setLowVariant(boolean isLowVariant) {
-        this.isLowVariant = isLowVariant;
-    }
-
-    public final static String COMMAND_NAME = "commandName";
-    public final static String MEAN = "mean";
-    public final static String STDDEV = "stddev";
-    public final static String IS_LOW_VARIANT = "isLowVariant";
-
-    public Map<String,Object> toMap(){
-        Map<String,Object> map = new HashMap<>();
-        map.put(COMMAND_NAME,this.commandName);
-        map.put(MEAN,this.mean);
-        map.put(STDDEV,this.stddev);
-        map.put(IS_LOW_VARIANT,this.isLowVariant);
-        return map;
-    }
-}
\ No newline at end of file

http://git-wip-us.apache.org/repos/asf/incubator-eagle/blob/8b646715/eagle-security/eagle-security-userprofile/common/src/main/java/org/apache/eagle/security/userprofile/model/UserProfileEigenModelEntity.java
----------------------------------------------------------------------
diff --git a/eagle-security/eagle-security-userprofile/common/src/main/java/org/apache/eagle/security/userprofile/model/UserProfileEigenModelEntity.java b/eagle-security/eagle-security-userprofile/common/src/main/java/org/apache/eagle/security/userprofile/model/UserProfileEigenModelEntity.java
deleted file mode 100644
index 2651f51..0000000
--- a/eagle-security/eagle-security-userprofile/common/src/main/java/org/apache/eagle/security/userprofile/model/UserProfileEigenModelEntity.java
+++ /dev/null
@@ -1,204 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements.  See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License.  You may obtain a copy of the License at
- *
- *    http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-package org.apache.eagle.security.userprofile.model;
-
-import org.apache.eagle.log.base.taggedlog.TaggedLogAPIEntity;
-import org.apache.eagle.ml.model.MLModelAPIEntity;
-import org.apache.eagle.security.userprofile.UserProfileConstants;
-import org.apache.commons.math3.linear.Array2DRowRealMatrix;
-import org.apache.commons.math3.linear.ArrayRealVector;
-import org.apache.commons.math3.linear.RealVector;
-import org.apache.eagle.security.userprofile.UserProfileConstants;
-import org.codehaus.jackson.annotate.JsonIgnoreProperties;
-import org.codehaus.jackson.map.annotate.JsonSerialize;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
-
-import java.io.IOException;
-import java.util.ArrayList;
-import java.util.HashMap;
-import java.util.List;
-import java.util.Map;
-
-@JsonSerialize(include=JsonSerialize.Inclusion.NON_NULL)
-@JsonIgnoreProperties(ignoreUnknown = true)
-public class UserProfileEigenModelEntity {
-    private final static Logger LOG = LoggerFactory.getLogger(UserProfileEigenModelEntity.class);
-
-    public static MLModelAPIEntity serializeModel(UserProfileEigenModel model) throws IOException {
-        if(model == null) return null;
-        if(model.user()==null) throw new IllegalArgumentException("user is null");
-
-        MLModelAPIEntity entity = new MLModelAPIEntity();
-        UserProfileEigenModelEntity content = new UserProfileEigenModelEntity();
-        entity.setVersion(model.version());
-        if(model.uMatrix()!=null) content.setUMatrix(model.uMatrix().getData());
-        if(model.diagonalMatrix()!=null) content.setDiagonalMatrix(model.diagonalMatrix().getData());
-        content.setDimension(model.dimension());
-        if(model.minVector()!=null) content.setMinVector(model.minVector().toArray());
-        if(model.maxVector()!=null) content.setMaxVector(model.maxVector().toArray());
-
-        RealVector[] pcsInVector = model.principalComponents();
-        if(pcsInVector!=null) {
-            List<double[]> prcsInDouleArrayList = new ArrayList<>(pcsInVector.length);
-            for (RealVector vector : pcsInVector) prcsInDouleArrayList.add(vector.toArray());
-            content.setPrincipalComponents(prcsInDouleArrayList);
-        }
-        if(model.maximumL2Norm()!=null) content.setMaximumL2Norm(model.maximumL2Norm().toArray());
-        if(model.minimumL2Norm()!=null) content.setMinimumL2Norm(model.minimumL2Norm().toArray());
-        if(model.statistics()!=null){
-            List<Map<String,Object>> statisticsMaps = new ArrayList<>(model.statistics().length);
-            for(UserCommandStatistics userCommandStatistics :model.statistics()) statisticsMaps.add(userCommandStatistics.toMap());
-            content.setStatistics(statisticsMaps);
-        }
-
-        entity.setContent(TaggedLogAPIEntity.buildObjectMapper().writeValueAsString(content));
-        Map<String,String> tags = new HashMap<>();
-        tags.put(UserProfileConstants.SITE_TAG,model.site());
-        tags.put(UserProfileConstants.USER_TAG,model.user());
-        tags.put(UserProfileConstants.ALGORITHM_TAG,model.algorithm());
-
-        entity.setTags(tags);
-        return entity;
-    }
-
-    public  static UserProfileEigenModel deserializeModel(MLModelAPIEntity entity) throws IOException {
-        if(entity == null) return null;
-        String user = entity.getTags() == null ? null: entity.getTags().get(UserProfileConstants.USER_TAG);
-        String site = entity.getTags()==null? null:entity.getTags().get(UserProfileConstants.SITE_TAG);
-
-        UserProfileEigenModelEntity content = TaggedLogAPIEntity.buildObjectMapper().readValue(entity.getContent(), UserProfileEigenModelEntity.class);
-        RealVector[] principalComponents = null;
-        if(content.getPrincipalComponents()!=null){
-            principalComponents = new RealVector[content.getPrincipalComponents().size()];
-            int i = 0;
-            for(double[] array:content.getPrincipalComponents()){
-                principalComponents[i] = new ArrayRealVector(array);
-                i++;
-            }
-        }
-        UserCommandStatistics[] statistics = null;
-        if(content.getStatistics()!=null){
-            statistics = new UserCommandStatistics[content.getStatistics().size()];
-            int i = 0;
-            for(Map<String,Object> map:content.getStatistics()){
-                statistics[i++] = new UserCommandStatistics(map);
-            }
-        }
-
-        return new UserProfileEigenModel(
-                entity.getVersion(),
-                site,
-                user,
-                content.getUMatrix() == null? null : new Array2DRowRealMatrix(content.getUMatrix()),
-                content.getDiagonalMatrix() == null ? null : new Array2DRowRealMatrix(content.getDiagonalMatrix()),
-                content.getDimension(),
-                content.getMinVector() == null ? null : new ArrayRealVector(content.getMinVector()),
-                content.getMaxVector() == null? null : new ArrayRealVector(content.getMaxVector()),
-                principalComponents,
-                content.getMaximumL2Norm() == null ? null : new ArrayRealVector(content.getMaximumL2Norm()),
-                content.getMinimumL2Norm() == null ? null : new ArrayRealVector(content.getMinimumL2Norm()),
-                statistics
-        );
-    }
-
-    private double[][] uMatrix;
-    private double[][] diagonalMatrix;
-    private int dimension;
-    private double[] minVector;
-    private double[] maxVector;
-    private List<double[]> principalComponents;
-    private double[] maximumL2Norm;
-    private double[] minimumL2Norm;
-    private List<Map<String,Object>> statistics;
-
-    public List<Map<String, Object>> getStatistics() {
-        return statistics;
-    }
-
-    public void setStatistics(List<Map<String, Object>> statistics) {
-        this.statistics = statistics;
-//        valueChanged("statistics");
-    }
-
-    public double[][] getUMatrix() {
-        return uMatrix;
-    }
-
-    public void setUMatrix(double[][] uMatrix) {
-        this.uMatrix = uMatrix;
-//        valueChanged("uMatrix");
-    }
-
-    public double[][] getDiagonalMatrix() {
-        return diagonalMatrix;
-    }
-
-    public void setDiagonalMatrix(double[][] diagonalMatrix) {
-        this.diagonalMatrix = diagonalMatrix;
-//        valueChanged("diagonalMatrix");
-    }
-
-    public int getDimension() {
-        return dimension;
-    }
-
-    public void setDimension(int dimension) {
-        this.dimension = dimension;
-//        valueChanged("dimension");
-    }
-
-    public double[] getMinVector() {
-        return minVector;
-    }
-
-    public void setMinVector(double[] minVector) {
-        this.minVector = minVector;
-    }
-
-    public double[] getMaxVector() {
-        return maxVector;
-    }
-
-    public void setMaxVector(double[] maxVector) {
-        this.maxVector = maxVector;
-    }
-
-    public List<double[]> getPrincipalComponents() {
-        return principalComponents;
-    }
-
-    public void setPrincipalComponents(List<double[]> principalComponents) {
-        this.principalComponents = principalComponents;
-    }
-
-    public double[] getMaximumL2Norm() {
-        return maximumL2Norm;
-    }
-
-    public void setMaximumL2Norm(double[] maximumL2Norm) {
-        this.maximumL2Norm = maximumL2Norm;
-    }
-
-    public double[] getMinimumL2Norm() {
-        return minimumL2Norm;
-    }
-
-    public void setMinimumL2Norm(double[] minimumL2Norm) {
-        this.minimumL2Norm = minimumL2Norm;
-    }
-}
\ No newline at end of file

http://git-wip-us.apache.org/repos/asf/incubator-eagle/blob/8b646715/eagle-security/eagle-security-userprofile/common/src/main/java/org/apache/eagle/security/userprofile/model/UserProfileEntityRepository.java
----------------------------------------------------------------------
diff --git a/eagle-security/eagle-security-userprofile/common/src/main/java/org/apache/eagle/security/userprofile/model/UserProfileEntityRepository.java b/eagle-security/eagle-security-userprofile/common/src/main/java/org/apache/eagle/security/userprofile/model/UserProfileEntityRepository.java
deleted file mode 100644
index 88d2d49..0000000
--- a/eagle-security/eagle-security-userprofile/common/src/main/java/org/apache/eagle/security/userprofile/model/UserProfileEntityRepository.java
+++ /dev/null
@@ -1,25 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements.  See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License.  You may obtain a copy of the License at
- *
- *    http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-package org.apache.eagle.security.userprofile.model;
-
-import org.apache.eagle.log.entity.repo.EntityRepository;
-
-public class UserProfileEntityRepository extends EntityRepository {
-    public UserProfileEntityRepository(){
-        this.registerEntity(ScheduleCommandEntity.class);
-    }
-}
\ No newline at end of file

http://git-wip-us.apache.org/repos/asf/incubator-eagle/blob/8b646715/eagle-security/eagle-security-userprofile/common/src/main/java/org/apache/eagle/security/userprofile/model/UserProfileKDEModelEntity.java
----------------------------------------------------------------------
diff --git a/eagle-security/eagle-security-userprofile/common/src/main/java/org/apache/eagle/security/userprofile/model/UserProfileKDEModelEntity.java b/eagle-security/eagle-security-userprofile/common/src/main/java/org/apache/eagle/security/userprofile/model/UserProfileKDEModelEntity.java
deleted file mode 100644
index 5209289..0000000
--- a/eagle-security/eagle-security-userprofile/common/src/main/java/org/apache/eagle/security/userprofile/model/UserProfileKDEModelEntity.java
+++ /dev/null
@@ -1,129 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements.  See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License.  You may obtain a copy of the License at
- *
- *    http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-package org.apache.eagle.security.userprofile.model;
-
-import org.apache.eagle.log.base.taggedlog.TaggedLogAPIEntity;
-import org.apache.eagle.ml.model.MLModelAPIEntity;
-import org.apache.eagle.security.userprofile.UserProfileConstants;
-
-import org.apache.eagle.security.userprofile.UserProfileConstants;
-import org.codehaus.jackson.annotate.JsonIgnoreProperties;
-import org.codehaus.jackson.map.annotate.JsonSerialize;
-
-import java.io.IOException;
-import java.util.ArrayList;
-import java.util.HashMap;
-import java.util.List;
-import java.util.Map;
-
-@JsonSerialize(include=JsonSerialize.Inclusion.NON_NULL)
-@JsonIgnoreProperties(ignoreUnknown = true)
-public class UserProfileKDEModelEntity {
-    public static UserProfileKDEModel deserializeModel(MLModelAPIEntity entity) throws IOException {
-        if(entity == null) return null;
-        String user = entity.getTags()==null? null:entity.getTags().get(UserProfileConstants.USER_TAG);
-        String site = entity.getTags()==null? null:entity.getTags().get(UserProfileConstants.SITE_TAG);
-        UserProfileKDEModelEntity content = TaggedLogAPIEntity.buildObjectMapper().readValue(entity.getContent(), UserProfileKDEModelEntity.class);
-
-        UserCommandStatistics[] statistics = null;
-        if(content.getStatistics()!=null) {
-            statistics = new UserCommandStatistics[content.getStatistics().size()];
-            int i = 0;
-            for (Map<String, Object> map : content.getStatistics()) {
-                statistics[i] = new UserCommandStatistics(map);
-                i++;
-            }
-        }
-        return new UserProfileKDEModel(
-                entity.getVersion(), site,user,statistics,
-                content.getMinProbabilityEstimate(), content.getMaxProbabilityEstimate(),
-                content.getNintyFivePercentileEstimate(), content.getMedianProbabilityEstimate());
-    }
-
-    public static MLModelAPIEntity serializeModel(UserProfileKDEModel model) throws IOException {
-        if(model == null) return null;
-        MLModelAPIEntity entity = new MLModelAPIEntity();
-        UserProfileKDEModelEntity content = new UserProfileKDEModelEntity();
-        entity.setVersion(model.version());
-        if(model.statistics()!=null){
-            List<Map<String,Object>> statisticsMaps = new ArrayList<>(model.statistics().length);
-            for(UserCommandStatistics userCommandStatistics :model.statistics()){
-                statisticsMaps.add(userCommandStatistics.toMap());
-            }
-            content.setStatistics(statisticsMaps);
-        }
-        content.setMinProbabilityEstimate(model.minProbabilityEstimate());
-        content.setMaxProbabilityEstimate(model.maxProbabilityEstimate());
-        content.setNintyFivePercentileEstimate(model.nintyFivePercentileEstimate());
-        content.setMedianProbabilityEstimate(model.medianProbabilityEstimate());
-
-        Map<String,String> tags = new HashMap<>();
-        tags.put(UserProfileConstants.SITE_TAG,model.site());
-        tags.put(UserProfileConstants.USER_TAG,model.user());
-        tags.put(UserProfileConstants.ALGORITHM_TAG,model.algorithm());
-
-        entity.setTags(tags);
-        entity.setContent(TaggedLogAPIEntity.buildObjectMapper().writeValueAsString(content));
-        return entity;
-    }
-
-    private List<Map<String,Object>> statistics;
-    private double minProbabilityEstimate;
-    private double maxProbabilityEstimate;
-    private double nintyFivePercentileEstimate;
-    private double medianProbabilityEstimate;
-
-    public double getMedianProbabilityEstimate() {
-        return medianProbabilityEstimate;
-    }
-
-    public void setMedianProbabilityEstimate(double medianProbabilityEstimate) {
-        this.medianProbabilityEstimate = medianProbabilityEstimate;
-    }
-
-    public List<Map<String,Object>> getStatistics() {
-        return statistics;
-    }
-
-    public void setStatistics(List<Map<String,Object>> statistics) {
-        this.statistics = statistics;
-    }
-
-    public double getMinProbabilityEstimate() {
-        return minProbabilityEstimate;
-    }
-
-    public void setMinProbabilityEstimate(double minProbabilityEstimate) {
-        this.minProbabilityEstimate = minProbabilityEstimate;
-    }
-
-    public double getMaxProbabilityEstimate() {
-        return maxProbabilityEstimate;
-    }
-
-    public void setMaxProbabilityEstimate(double maxProbabilityEstimate) {
-        this.maxProbabilityEstimate = maxProbabilityEstimate;
-    }
-
-    public double getNintyFivePercentileEstimate() {
-        return nintyFivePercentileEstimate;
-    }
-
-    public void setNintyFivePercentileEstimate(double nintyFivePercentileEstimate) {
-        this.nintyFivePercentileEstimate = nintyFivePercentileEstimate;
-    }
-}
\ No newline at end of file

http://git-wip-us.apache.org/repos/asf/incubator-eagle/blob/8b646715/eagle-security/eagle-security-userprofile/common/src/main/java/org/apache/eagle/security/userprofile/model/package-info.java
----------------------------------------------------------------------
diff --git a/eagle-security/eagle-security-userprofile/common/src/main/java/org/apache/eagle/security/userprofile/model/package-info.java b/eagle-security/eagle-security-userprofile/common/src/main/java/org/apache/eagle/security/userprofile/model/package-info.java
deleted file mode 100644
index 20fa623..0000000
--- a/eagle-security/eagle-security-userprofile/common/src/main/java/org/apache/eagle/security/userprofile/model/package-info.java
+++ /dev/null
@@ -1,20 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements.  See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License.  You may obtain a copy of the License at
- *
- *    http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-/**
- * Persistence related model and utilities
- */
-package org.apache.eagle.security.userprofile.model;

http://git-wip-us.apache.org/repos/asf/incubator-eagle/blob/8b646715/eagle-security/eagle-security-userprofile/common/src/main/resources/log4j.properties
----------------------------------------------------------------------
diff --git a/eagle-security/eagle-security-userprofile/common/src/main/resources/log4j.properties b/eagle-security/eagle-security-userprofile/common/src/main/resources/log4j.properties
deleted file mode 100644
index 3499c46..0000000
--- a/eagle-security/eagle-security-userprofile/common/src/main/resources/log4j.properties
+++ /dev/null
@@ -1,35 +0,0 @@
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements.  See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License.  You may obtain a copy of the License at
-#
-#    http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-log4j.rootLogger=INFO, stdout
-
- eagle.log.dir=./logs
- eagle.log.file=eagle.log
-
-# standard output
-log4j.appender.stdout=org.apache.log4j.ConsoleAppender
-log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
-log4j.appender.stdout.layout.ConversionPattern=%d{ISO8601} %p [%t] %c{2}[%L]: %m%n
-
-# Daily Rolling File Appender
- log4j.appender.DRFA=org.apache.log4j.DailyRollingFileAppender
- log4j.appender.DRFA.File=${eagle.log.dir}/${eagle.log.file}
- log4j.appender.DRFA.DatePattern=.yyyy-MM-dd
-## 30-day backup
-# log4j.appender.DRFA.MaxBackupIndex=30
- log4j.appender.DRFA.layout=org.apache.log4j.PatternLayout
-
-# Pattern format: Date LogLevel LoggerName LogMessage
-log4j.appender.DRFA.layout.ConversionPattern=%d{ISO8601} %p [%t] %c{2}[%L]: %m%n
\ No newline at end of file

http://git-wip-us.apache.org/repos/asf/incubator-eagle/blob/8b646715/eagle-security/eagle-security-userprofile/common/src/main/resources/storm.yaml.1
----------------------------------------------------------------------
diff --git a/eagle-security/eagle-security-userprofile/common/src/main/resources/storm.yaml.1 b/eagle-security/eagle-security-userprofile/common/src/main/resources/storm.yaml.1
deleted file mode 100644
index 2ea7262..0000000
--- a/eagle-security/eagle-security-userprofile/common/src/main/resources/storm.yaml.1
+++ /dev/null
@@ -1,29 +0,0 @@
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements.  See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License.  You may obtain a copy of the License at
-#
-#    http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-### topology.* configs are for specific executing storms
-topology.enable.message.timeouts: true
-topology.debug: false
-topology.workers: 1
-topology.acker.executors: 10
-topology.tasks: null
-# maximum amount of time a message has to complete before it's considered failed
-topology.message.timeout.secs: 60000
-topology.receiver.buffer.size: 8
-topology.transfer.buffer.size: 1024
-topology.executor.receive.buffer.size: 16384
-topology.executor.send.buffer.size: 16384
-storm.zookeeper.session.timeout: 600000
-storm.zookeeper.connection.timeout: 90000

http://git-wip-us.apache.org/repos/asf/incubator-eagle/blob/8b646715/eagle-security/eagle-security-userprofile/common/src/main/scala/org/apache/eagle/security/userprofile/model/AuditlogParser.scala
----------------------------------------------------------------------
diff --git a/eagle-security/eagle-security-userprofile/common/src/main/scala/org/apache/eagle/security/userprofile/model/AuditlogParser.scala b/eagle-security/eagle-security-userprofile/common/src/main/scala/org/apache/eagle/security/userprofile/model/AuditlogParser.scala
deleted file mode 100644
index 47d47d9..0000000
--- a/eagle-security/eagle-security-userprofile/common/src/main/scala/org/apache/eagle/security/userprofile/model/AuditlogParser.scala
+++ /dev/null
@@ -1,54 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements.  See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License.  You may obtain a copy of the License at
- *
- *    http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-package org.apache.eagle.security.userprofile.model
-
-import org.apache.eagle.security.hdfs.{HDFSAuditLogObject, HDFSAuditLogParser}
-import org.apache.eagle.security.userprofile.UserProfileUtils
-import org.joda.time.Period
-
-/**
- * {"getfileinfo", "open", "listStatus", "setTimes", "setPermission", "rename", "mkdirs", "create", "setReplication", "contentSummary", "delete", "setOwner", "fsck"}
- */
-case class UserCommandActivity(timestamp:Long,user:String,cmd:String,periodWindowSeconds:Long)
-
-/**
- * SELECT a.ugi, a.dt, a.hour, a.cmd, count(1)
- * from audit_log a
- * WHERE a.dt>='${STARTDATE}' AND a.dt<='${ENDDATE}'AND a.hour >= '${STARTTIME}' AND a.hour < '${ENDTIME}'
- * GROUP BY a.ugi,a.dt, a.hour, a.cmd;
- */
-case class AuditLogTransformer(period:Period) extends Serializable {
-  val parser = new HDFSAuditLogParser()
-  // serialize periodInSeconds to avoid recreating Seconds object
-  val periodInSeconds = period.toStandardSeconds
-
-  def getObject(value: String) : Option[HDFSAuditLogObject] = {
-    Option(parser.parse(value))
-  }
-
-  def transform(value: String) : Option[UserCommandActivity] = {
-    Option(parser.parse(value)) match {
-      case Some(entity) =>
-        val timestamp = entity.timestamp
-        val user = entity.user
-        val cmd = entity.cmd
-        val periodWindowSeconds = UserProfileUtils.formatSecondsByPeriod (timestamp / 1000, periodInSeconds)
-        Some(UserCommandActivity (timestamp, user, cmd, periodWindowSeconds))
-      case None => None
-    }
-  }
-}
\ No newline at end of file

http://git-wip-us.apache.org/repos/asf/incubator-eagle/blob/8b646715/eagle-security/eagle-security-userprofile/common/src/main/scala/org/apache/eagle/security/userprofile/model/EntityConversion.scala
----------------------------------------------------------------------
diff --git a/eagle-security/eagle-security-userprofile/common/src/main/scala/org/apache/eagle/security/userprofile/model/EntityConversion.scala b/eagle-security/eagle-security-userprofile/common/src/main/scala/org/apache/eagle/security/userprofile/model/EntityConversion.scala
deleted file mode 100644
index 266bf88..0000000
--- a/eagle-security/eagle-security-userprofile/common/src/main/scala/org/apache/eagle/security/userprofile/model/EntityConversion.scala
+++ /dev/null
@@ -1,27 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements.  See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License.  You may obtain a copy of the License at
- *
- *    http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-package org.apache.eagle.security.userprofile.model
-
-import org.apache.eagle.log.base.taggedlog.TaggedLogAPIEntity
-
-/**
- * @since  7/22/15
- */
-private[userprofile] trait EntityConversion[M <: TaggedLogAPIEntity] {
-  def toEntity: M
-  def fromEntity (m: M)
-}
\ No newline at end of file

http://git-wip-us.apache.org/repos/asf/incubator-eagle/blob/8b646715/eagle-security/eagle-security-userprofile/common/src/main/scala/org/apache/eagle/security/userprofile/model/UserActivityAggModel.scala
----------------------------------------------------------------------
diff --git a/eagle-security/eagle-security-userprofile/common/src/main/scala/org/apache/eagle/security/userprofile/model/UserActivityAggModel.scala b/eagle-security/eagle-security-userprofile/common/src/main/scala/org/apache/eagle/security/userprofile/model/UserActivityAggModel.scala
deleted file mode 100644
index edd5aa7..0000000
--- a/eagle-security/eagle-security-userprofile/common/src/main/scala/org/apache/eagle/security/userprofile/model/UserActivityAggModel.scala
+++ /dev/null
@@ -1,28 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements.  See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License.  You may obtain a copy of the License at
- *
- *    http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-package org.apache.eagle.security.userprofile.model
-
-import org.apache.commons.math3.linear.RealMatrix
-
-/**
- * @since  7/31/15
- */
-case class UserActivityAggModel(user:String,matrix:RealMatrix,cmdTypes:Seq[String],site:String,timestamp:Long)
-  extends EntityConversion[UserActivityAggModelEntity]{
-  override def toEntity: UserActivityAggModelEntity = UserActivityAggModelEntity.fromModel(this)
-  override def fromEntity(entity: UserActivityAggModelEntity): Unit = UserActivityAggModelEntity.toModel(entity)
-}

http://git-wip-us.apache.org/repos/asf/incubator-eagle/blob/8b646715/eagle-security/eagle-security-userprofile/common/src/main/scala/org/apache/eagle/security/userprofile/model/UserProfileEigenModel.scala
----------------------------------------------------------------------
diff --git a/eagle-security/eagle-security-userprofile/common/src/main/scala/org/apache/eagle/security/userprofile/model/UserProfileEigenModel.scala b/eagle-security/eagle-security-userprofile/common/src/main/scala/org/apache/eagle/security/userprofile/model/UserProfileEigenModel.scala
deleted file mode 100644
index cdb4205..0000000
--- a/eagle-security/eagle-security-userprofile/common/src/main/scala/org/apache/eagle/security/userprofile/model/UserProfileEigenModel.scala
+++ /dev/null
@@ -1,44 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements.  See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License.  You may obtain a copy of the License at
- *
- *    http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-package org.apache.eagle.security.userprofile.model
-
-import org.apache.commons.math3.linear.{RealMatrix, RealVector}
-import org.apache.eagle.ml.model.MLModelAPIEntity
-import org.apache.eagle.security.userprofile.UserProfileConstants
-import org.codehaus.jackson.map.annotate.JsonSerialize
-
-/**
- * @since  7/22/15
- */
-@JsonSerialize(include = JsonSerialize.Inclusion.NON_NULL)
-case class UserProfileEigenModel(
-  override val version:Long,
-  override val site:String,
-  override val user:String,
-  uMatrix: RealMatrix,
-  diagonalMatrix: RealMatrix,
-  dimension: Int,
-  minVector: RealVector,
-  maxVector: RealVector,
-  principalComponents: Array[RealVector],
-  maximumL2Norm:RealVector,
-  minimumL2Norm:RealVector,
-  statistics:Array[UserCommandStatistics]
-) extends UserProfileModel(site,user,UserProfileConstants.EIGEN_DECOMPOSITION_ALGORITHM,version) with EntityConversion[MLModelAPIEntity] {
-  override def toEntity: MLModelAPIEntity = UserProfileEigenModelEntity.serializeModel(this)
-  override def fromEntity(m: MLModelAPIEntity): Unit = UserProfileEigenModelEntity.deserializeModel(m)
-}
\ No newline at end of file

http://git-wip-us.apache.org/repos/asf/incubator-eagle/blob/8b646715/eagle-security/eagle-security-userprofile/common/src/main/scala/org/apache/eagle/security/userprofile/model/UserProfileKDEModel.scala
----------------------------------------------------------------------
diff --git a/eagle-security/eagle-security-userprofile/common/src/main/scala/org/apache/eagle/security/userprofile/model/UserProfileKDEModel.scala b/eagle-security/eagle-security-userprofile/common/src/main/scala/org/apache/eagle/security/userprofile/model/UserProfileKDEModel.scala
deleted file mode 100644
index bd0af47..0000000
--- a/eagle-security/eagle-security-userprofile/common/src/main/scala/org/apache/eagle/security/userprofile/model/UserProfileKDEModel.scala
+++ /dev/null
@@ -1,37 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements.  See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License.  You may obtain a copy of the License at
- *
- *    http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-package org.apache.eagle.security.userprofile.model
-
-import org.apache.eagle.ml.model.MLModelAPIEntity
-import org.apache.eagle.security.userprofile.UserProfileConstants
-
-/**
- * @since  7/22/15
- */
-case class UserProfileKDEModel (
-   override val version:Long,
-   override val site:String,
-   override val user:String,
-   statistics: Array[UserCommandStatistics],
-   minProbabilityEstimate:Double,
-   maxProbabilityEstimate: Double,
-   nintyFivePercentileEstimate: Double,
-   medianProbabilityEstimate: Double
-) extends UserProfileModel(site,user,UserProfileConstants.KDE_ALGORITHM,version ) with EntityConversion[MLModelAPIEntity]{
-  override def toEntity: MLModelAPIEntity = UserProfileKDEModelEntity.serializeModel(this)
-  override def fromEntity(m: MLModelAPIEntity): Unit = UserProfileKDEModelEntity.deserializeModel(m)
-}
\ No newline at end of file

http://git-wip-us.apache.org/repos/asf/incubator-eagle/blob/8b646715/eagle-security/eagle-security-userprofile/common/src/main/scala/org/apache/eagle/security/userprofile/model/UserProfileModel.scala
----------------------------------------------------------------------
diff --git a/eagle-security/eagle-security-userprofile/common/src/main/scala/org/apache/eagle/security/userprofile/model/UserProfileModel.scala b/eagle-security/eagle-security-userprofile/common/src/main/scala/org/apache/eagle/security/userprofile/model/UserProfileModel.scala
deleted file mode 100644
index decb0ed..0000000
--- a/eagle-security/eagle-security-userprofile/common/src/main/scala/org/apache/eagle/security/userprofile/model/UserProfileModel.scala
+++ /dev/null
@@ -1,22 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements.  See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License.  You may obtain a copy of the License at
- *
- *    http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-package org.apache.eagle.security.userprofile.model
-
-/**
- * @since  7/21/15
- */
-abstract class UserProfileModel(val site:String,val user:String,val algorithm:String, val version:Long)
\ No newline at end of file

http://git-wip-us.apache.org/repos/asf/incubator-eagle/blob/8b646715/eagle-security/eagle-security-userprofile/common/src/test/java/eagle/auditlog/userprofile/common/TestUserProfileUtils.java
----------------------------------------------------------------------
diff --git a/eagle-security/eagle-security-userprofile/common/src/test/java/eagle/auditlog/userprofile/common/TestUserProfileUtils.java b/eagle-security/eagle-security-userprofile/common/src/test/java/eagle/auditlog/userprofile/common/TestUserProfileUtils.java
deleted file mode 100644
index d98907f..0000000
--- a/eagle-security/eagle-security-userprofile/common/src/test/java/eagle/auditlog/userprofile/common/TestUserProfileUtils.java
+++ /dev/null
@@ -1,106 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements.  See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License.  You may obtain a copy of the License at
- *
- *    http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-package eagle.auditlog.userprofile.common;
-
-import org.apache.eagle.security.userprofile.UserProfileUtils;
-import org.apache.eagle.common.DateTimeUtil;
-import org.joda.time.Period;
-import org.joda.time.Seconds;
-import org.junit.Assert;
-import org.junit.Test;
-
-import java.text.ParseException;
-
-public class TestUserProfileUtils {
-    @Test
-    public void testJodaTimePeriod() throws ParseException {
-        String periodText = "PT10m";
-        Period period = new Period(periodText);
-        int seconds = period.toStandardSeconds().getSeconds();
-        Assert.assertEquals(600, seconds);
-        Assert.assertEquals(60, period.toStandardSeconds().dividedBy(10).getSeconds());
-    }
-
-    @Test
-    public void testFormatSecondsByPeriod15M() throws ParseException {
-
-        Period period = new Period("PT15m");
-        Seconds seconds = period.toStandardSeconds();
-        Assert.assertEquals(15*60,seconds.getSeconds());
-
-        long time = DateTimeUtil.humanDateToSeconds("2015-07-01 13:56:12");
-        long expect = DateTimeUtil.humanDateToSeconds("2015-07-01 13:45:00");
-        long result = UserProfileUtils.formatSecondsByPeriod(time,seconds);
-        Assert.assertEquals(expect,result);
-
-        time = DateTimeUtil.humanDateToSeconds("2015-07-01 03:14:59");
-        expect = DateTimeUtil.humanDateToSeconds("2015-07-01 03:00:00");
-        result = UserProfileUtils.formatSecondsByPeriod(time, seconds);
-        Assert.assertEquals(expect,result);
-
-        time = DateTimeUtil.humanDateToSeconds("2015-07-01 03:14:59");
-        expect = DateTimeUtil.humanDateToSeconds("2015-07-01 03:00:00");
-        result = UserProfileUtils.formatSecondsByPeriod(time, seconds);
-        Assert.assertEquals(expect,result);
-    }
-
-    @Test
-     public void testFormatSecondsByPeriod1H() throws ParseException {
-
-        Period period = new Period("PT1h");
-        Seconds seconds = period.toStandardSeconds();
-        Assert.assertEquals(60*60,seconds.getSeconds());
-
-        long time = DateTimeUtil.humanDateToSeconds("2015-07-01 13:56:12");
-        long expect = DateTimeUtil.humanDateToSeconds("2015-07-01 13:00:00");
-        long result = UserProfileUtils.formatSecondsByPeriod(time,seconds);
-        Assert.assertEquals(expect,result);
-
-        time = DateTimeUtil.humanDateToSeconds("2015-07-01 03:14:59");
-        expect = DateTimeUtil.humanDateToSeconds("2015-07-01 03:00:00");
-        result = UserProfileUtils.formatSecondsByPeriod(time, seconds);
-        Assert.assertEquals(expect,result);
-
-        time = DateTimeUtil.humanDateToSeconds("2015-07-01 03:30:59");
-        expect = DateTimeUtil.humanDateToSeconds("2015-07-01 03:00:00");
-        result = UserProfileUtils.formatSecondsByPeriod(time, seconds);
-        Assert.assertEquals(expect,result);
-    }
-
-    /*@Test
-    public void testFormatSecondsByPeriod1Min() throws ParseException {
-
-        Period period = new Period("PT1M");
-        Seconds seconds = period.toStandardSeconds();
-        Assert.assertEquals(60,seconds.getSeconds());
-
-        long time = DateTimeUtil.humanDateToSeconds("2015-07-01 13:56:15");
-        long expect = DateTimeUtil.humanDateToSeconds("2015-07-01 13:55:16");
-        long result = UserProfileUtils.formatSecondsByPeriod(time,seconds);
-        Assert.assertEquals(expect,result);
-
-        //time = DateTimeUtil.humanDateToSeconds("2015-07-01 03:14:59");
-        //expect = DateTimeUtil.humanDateToSeconds("2015-07-01 03:00:00");
-        //result = UserProfileUtils.formatSecondsByPeriod(time, seconds);
-        //Assert.assertEquals(expect,result);
-
-        //time = DateTimeUtil.humanDateToSeconds("2015-07-01 03:30:59");
-        //expect = DateTimeUtil.humanDateToSeconds("2015-07-01 03:00:00");
-        //result = UserProfileUtils.formatSecondsByPeriod(time, seconds);
-        //Assert.assertEquals(expect,result);
-    }*/
-}
\ No newline at end of file

http://git-wip-us.apache.org/repos/asf/incubator-eagle/blob/8b646715/eagle-security/eagle-security-userprofile/common/src/test/resources/log4j.properties
----------------------------------------------------------------------
diff --git a/eagle-security/eagle-security-userprofile/common/src/test/resources/log4j.properties b/eagle-security/eagle-security-userprofile/common/src/test/resources/log4j.properties
deleted file mode 100644
index 3499c46..0000000
--- a/eagle-security/eagle-security-userprofile/common/src/test/resources/log4j.properties
+++ /dev/null
@@ -1,35 +0,0 @@
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements.  See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License.  You may obtain a copy of the License at
-#
-#    http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-log4j.rootLogger=INFO, stdout
-
- eagle.log.dir=./logs
- eagle.log.file=eagle.log
-
-# standard output
-log4j.appender.stdout=org.apache.log4j.ConsoleAppender
-log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
-log4j.appender.stdout.layout.ConversionPattern=%d{ISO8601} %p [%t] %c{2}[%L]: %m%n
-
-# Daily Rolling File Appender
- log4j.appender.DRFA=org.apache.log4j.DailyRollingFileAppender
- log4j.appender.DRFA.File=${eagle.log.dir}/${eagle.log.file}
- log4j.appender.DRFA.DatePattern=.yyyy-MM-dd
-## 30-day backup
-# log4j.appender.DRFA.MaxBackupIndex=30
- log4j.appender.DRFA.layout=org.apache.log4j.PatternLayout
-
-# Pattern format: Date LogLevel LoggerName LogMessage
-log4j.appender.DRFA.layout.ConversionPattern=%d{ISO8601} %p [%t] %c{2}[%L]: %m%n
\ No newline at end of file

http://git-wip-us.apache.org/repos/asf/incubator-eagle/blob/8b646715/eagle-security/eagle-security-userprofile/common/src/test/resources/storm.yaml.2
----------------------------------------------------------------------
diff --git a/eagle-security/eagle-security-userprofile/common/src/test/resources/storm.yaml.2 b/eagle-security/eagle-security-userprofile/common/src/test/resources/storm.yaml.2
deleted file mode 100644
index 2ea7262..0000000
--- a/eagle-security/eagle-security-userprofile/common/src/test/resources/storm.yaml.2
+++ /dev/null
@@ -1,29 +0,0 @@
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements.  See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License.  You may obtain a copy of the License at
-#
-#    http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-### topology.* configs are for specific executing storms
-topology.enable.message.timeouts: true
-topology.debug: false
-topology.workers: 1
-topology.acker.executors: 10
-topology.tasks: null
-# maximum amount of time a message has to complete before it's considered failed
-topology.message.timeout.secs: 60000
-topology.receiver.buffer.size: 8
-topology.transfer.buffer.size: 1024
-topology.executor.receive.buffer.size: 16384
-topology.executor.send.buffer.size: 16384
-storm.zookeeper.session.timeout: 600000
-storm.zookeeper.connection.timeout: 90000

http://git-wip-us.apache.org/repos/asf/incubator-eagle/blob/8b646715/eagle-security/eagle-security-userprofile/detection/dev-supports/prepare-metadata.sh
----------------------------------------------------------------------
diff --git a/eagle-security/eagle-security-userprofile/detection/dev-supports/prepare-metadata.sh b/eagle-security/eagle-security-userprofile/detection/dev-supports/prepare-metadata.sh
deleted file mode 100755
index 5b1891a..0000000
--- a/eagle-security/eagle-security-userprofile/detection/dev-supports/prepare-metadata.sh
+++ /dev/null
@@ -1,45 +0,0 @@
-#!/usr/bin/env bash
-
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements.  See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License.  You may obtain a copy of the License at
-#
-#    http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-
-export SERVICE_HOST="localhost"
-export SERVICE_PORT=9099
-
-echo "Creating alert definition for AlertDefinitionService with userprofile-policy-definition.json"
-result=$(curl -XPOST -H "Content-Type: application/json" \
-        "http://$SERVICE_HOST:$SERVICE_PORT/eagle-service/rest/entities?serviceName=AlertDefinitionService" \
-        -d '[ { "prefix": "alertdef", "tags": { "site": "sandbox", "dataSource": "userProfile", "alertExecutorId": "userProfileAnomalyDetectionExecutor", "policyId": "userProfile", "policyType": "MachineLearning" }, "desc": "user profile anomaly detection", "policyDef": "{\"type\":\"MachineLearning\",\"alertContext\":{\"site\":\"sandbox\",\"dataSource\":\"userProfile\",\"component\":\"testComponent\",\"description\":\"ML based user profile anomaly detection\",\"severity\":\"WARNING\",\"notificationByEmail\":\"true\"},\"algorithms\":[{\"name\":\"EigenDecomposition\",\"evaluator\":\"eagle.security.userprofile.impl.UserProfileAnomalyEigenEvaluator\",\"description\":\"EigenBasedAnomalyDetection\",\"features\":\"getfileinfo, open, listStatus, setTimes, setPermission, rename, mkdirs, create, setReplication, contentSummary, delete, setOwner, fsck\"},{\"name\":\"KDE\",\"evaluator\":\"eagle.security.userprofile.impl.UserProfileAnomalyKDEEvaluator\",\"description\":\"DensityBasedAnomalyDetect
 ion\",\"features\":\"getfileinfo, open, listStatus, setTimes, setPermission, rename, mkdirs, create, setReplication, contentSummary, delete, setOwner, fsck\"}]}", "dedupeDef": "{\"alertDedupIntervalMin\":\"0\",\"emailDedupIntervalMin\":\"0\"}", "notificationDef": "", "remediationDef": "", "enabled": true } ]' 2>/dev/null)
-
-echo "=> $result"
-
-echo ""
-
-echo "Creating alert executor for AlertExecutorService with userprofile-executor-definition.json"
-result=$(curl -XPOST -H "Content-Type: application/json" \
-        "http://$SERVICE_HOST:$SERVICE_PORT/eagle-service/rest/entities?serviceName=AlertExecutorService" \
-        -d '[ { "prefix": "alertExecutor", "tags":{ "site":"sandbox", "dataSource":"userProfile", "alertExecutorId" : "userProfileAnomalyDetectionExecutor", "streamName":"userActivity" }, "desc": "user activity data source" } ]' 2>/dev/null)
-
-echo "=> $result"
-
-echo ""
-
-echo "Creating alert executor for AlertStreamService with userprofile-stream-description.json"
-result=$(curl -XPOST -H "Content-Type: application/json" \
-        "http://$SERVICE_HOST:$SERVICE_PORT/eagle-service/rest/entities?serviceName=AlertStreamService" \
-        -d '[ { "prefix": "alertStream", "tags": { "streamName": "userActivity", "site":"sandbox", "dataSource":"userProfile" }, "alertExecutorIdList": [ "userProfileAnomalyDetectionExecutor" ] } ]' 2>/dev/null)
-
-echo "=> $result"



Mime
View raw message