metron-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ceste...@apache.org
Subject incubator-metron git commit: METRON-656: Make Stellar 'in' closer to functioning like python closes apache/incubator-metron#416
Date Mon, 16 Jan 2017 14:21:41 GMT
Repository: incubator-metron
Updated Branches:
  refs/heads/master 9ec2cdcdc -> 2db6e35b8


METRON-656: Make Stellar 'in' closer to functioning like python closes apache/incubator-metron#416


Project: http://git-wip-us.apache.org/repos/asf/incubator-metron/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-metron/commit/2db6e35b
Tree: http://git-wip-us.apache.org/repos/asf/incubator-metron/tree/2db6e35b
Diff: http://git-wip-us.apache.org/repos/asf/incubator-metron/diff/2db6e35b

Branch: refs/heads/master
Commit: 2db6e35b895ca60698b1f1150e0772ed71dc59c4
Parents: 9ec2cdc
Author: cstella <cestella@gmail.com>
Authored: Mon Jan 16 09:21:40 2017 -0500
Committer: cstella <cestella@gmail.com>
Committed: Mon Jan 16 09:21:40 2017 -0500

----------------------------------------------------------------------
 metron-platform/metron-common/README.md         | 17 +++++---
 .../metron/common/stellar/StellarCompiler.java  | 32 +++++++++------
 .../metron/common/stellar/StellarTest.java      | 43 +++++++++++++++++++-
 3 files changed, 73 insertions(+), 19 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/2db6e35b/metron-platform/metron-common/README.md
----------------------------------------------------------------------
diff --git a/metron-platform/metron-common/README.md b/metron-platform/metron-common/README.md
index cbf0180..21e82fb 100644
--- a/metron-platform/metron-common/README.md
+++ b/metron-platform/metron-common/README.md
@@ -25,6 +25,7 @@ The query language supports the following:
 * Simple equality comparison operations `==`, `!=`
 * if/then/else comparisons (i.e. `if var1 < 10 then 'less than 10' else '10 or more'`)
 * Determining whether a field exists (via `exists`)
+* An `in` operator that works like the `in` in Python
 * The ability to have parenthesis to make order of operations explicit
 * User defined functions
 
@@ -39,11 +40,17 @@ The following keywords need to be single quote escaped in order to be
used in St
 | <= | \> | \>= |
 | ? | \+ | \- |
 | , | \* | / |
+|  | \* | / |
 
 Using parens such as: "foo" : "\<ok\>" requires escaping; "foo": "\'\<ok\>\'"
 
+## Stellar Language Inclusion Checks (`in` and `not in`)
+1. `in` supports string contains. e.g. `'foo' in 'foobar' == true`
+2. `in` supports collection contains. e.g. `'foo' in [ 'foo', 'bar' ] == true`
+3. `in` supports map key contains. e.g. `'foo' in { 'foo' : 5} == true`
+4. `not in` is the negation of the in expression. e.g. `'grok' not in 'foobar' == true`
 
-## Stellar Language Comparisons ('<', '<=', '>', '>=')
+## Stellar Language Comparisons (`<`, `<=`, `>`, `>=`)
 
 1. If either side of the comparison is null then return false.
 2. If both values being compared implement number then the following:
@@ -54,11 +61,11 @@ Using parens such as: "foo" : "\<ok\>" requires escaping; "foo":
"\'\<ok\>\'"
 3. If both sides are of the same type and are comparable then use the compareTo method to
compare values.
 4. If none of the above are met then an exception is thrown.
 
-## Stellar Language Equality Check ('==', '!=')
+## Stellar Language Equality Check (`==`, `!=`)
 
-Below is how the '==' operator is expected to work:
+Below is how the `==` operator is expected to work:
 
-1. If either side of the expression is null then check equality using Java's '==' expression.
+1. If either side of the expression is null then check equality using Java's `==` expression.
 2. Else if both sides of the expression are of Java's type Number then:
    * If either side of the expression is a double then use the double value of both sides
to test equality.
    * Else if either side of the expression is a float then use the float value of both sides
to test equality.
@@ -66,7 +73,7 @@ Below is how the '==' operator is expected to work:
    * Otherwise use int value of both sides to test equality
 3. Otherwise use equals method compare the left side with the right side.
 
-The '!=' operator is the negation of the above.
+The `!=` operator is the negation of the above.
 
 ## Stellar Core Functions
 

http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/2db6e35b/metron-platform/metron-common/src/main/java/org/apache/metron/common/stellar/StellarCompiler.java
----------------------------------------------------------------------
diff --git a/metron-platform/metron-common/src/main/java/org/apache/metron/common/stellar/StellarCompiler.java
b/metron-platform/metron-common/src/main/java/org/apache/metron/common/stellar/StellarCompiler.java
index e6bcf38..4822b31 100644
--- a/metron-platform/metron-common/src/main/java/org/apache/metron/common/stellar/StellarCompiler.java
+++ b/metron-platform/metron-common/src/main/java/org/apache/metron/common/stellar/StellarCompiler.java
@@ -79,22 +79,30 @@ public class StellarCompiler extends StellarBaseListener {
   }
 
   private boolean handleIn(Token<?> left, Token<?> right) {
-    Object key = null;
+    Object key = right.getValue();
 
-    Set<Object> set = null;
-    if (left.getValue() instanceof Collection) {
-      set = new HashSet<>((List<Object>) left.getValue());
-    } else if (left.getValue() != null) {
-      set = ImmutableSet.of(left.getValue());
-    } else {
-      set = new HashSet<>();
-    }
 
-    key = right.getValue();
-    if (key == null || set.isEmpty()) {
+    if (left.getValue() != null) {
+      if(left.getValue() instanceof String && key instanceof String) {
+        return ((String)left.getValue()).contains(key.toString());
+      }
+      else if(left.getValue() instanceof Collection) {
+        return ((Collection)left.getValue()).contains(key);
+      }
+      else if(left.getValue() instanceof Map) {
+        return ((Map)left.getValue()).containsKey(key);
+      }
+      else {
+        if(key == null) {
+          return key == left.getValue();
+        }
+        else {
+          return key.equals(left.getValue());
+        }
+      }
+    } else {
       return false;
     }
-    return set.contains(key);
   }
 
   @Override

http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/2db6e35b/metron-platform/metron-common/src/test/java/org/apache/metron/common/stellar/StellarTest.java
----------------------------------------------------------------------
diff --git a/metron-platform/metron-common/src/test/java/org/apache/metron/common/stellar/StellarTest.java
b/metron-platform/metron-common/src/test/java/org/apache/metron/common/stellar/StellarTest.java
index 2eaa60d..29df491 100644
--- a/metron-platform/metron-common/src/test/java/org/apache/metron/common/stellar/StellarTest.java
+++ b/metron-platform/metron-common/src/test/java/org/apache/metron/common/stellar/StellarTest.java
@@ -402,19 +402,58 @@ public class StellarTest {
   }
 
   @Test
-  public void testList() throws Exception {
+  public void testInCollection() throws Exception {
     final Map<String, String> variableMap = new HashMap<String, String>() {{
       put("foo", "casey");
       put("empty", "");
-      put("spaced", "metron is great");
     }};
     Assert.assertTrue(runPredicate("foo in [ 'casey', 'david' ]", v -> variableMap.get(v)));
+    Assert.assertFalse(runPredicate("foo in [ ]", v -> variableMap.get(v)));
     Assert.assertTrue(runPredicate("foo in [ foo, 'david' ]", v -> variableMap.get(v)));
     Assert.assertTrue(runPredicate("foo in [ 'casey', 'david' ] and 'casey' == foo", v ->
variableMap.get(v)));
     Assert.assertTrue(runPredicate("foo in [ 'casey', 'david' ] and foo == 'casey'", v ->
variableMap.get(v)));
     Assert.assertTrue(runPredicate("foo in [ 'casey' ]", v -> variableMap.get(v)));
     Assert.assertFalse(runPredicate("foo not in [ 'casey', 'david' ]", v -> variableMap.get(v)));
     Assert.assertFalse(runPredicate("foo not in [ 'casey', 'david' ] and 'casey' == foo",
v -> variableMap.get(v)));
+    Assert.assertTrue(runPredicate("null in [ null, 'something' ]", v -> variableMap.get(v)));
+    Assert.assertFalse(runPredicate("null not in [ null, 'something' ]", v -> variableMap.get(v)));
+  }
+
+  @Test
+  public void testInMap() throws Exception {
+    final Map<String, String> variableMap = new HashMap<String, String>() {{
+      put("foo", "casey");
+      put("empty", "");
+    }};
+    Assert.assertTrue(runPredicate("'casey' in { foo : 5 }", v -> variableMap.get(v)));
+    Assert.assertFalse(runPredicate("'casey' not in { foo : 5 }", v -> variableMap.get(v)));
+    Assert.assertTrue(runPredicate("foo in { foo : 5 }", v -> variableMap.get(v)));
+    Assert.assertFalse(runPredicate("foo not in { foo : 5 }", v -> variableMap.get(v)));
+    Assert.assertTrue(runPredicate("'foo' in { 'foo' : 5 }", v -> variableMap.get(v)));
+    Assert.assertFalse(runPredicate("'foo' not in { 'foo' : 5 }", v -> variableMap.get(v)));
+    Assert.assertTrue(runPredicate("foo in { 'casey' : 5 }", v -> variableMap.get(v)));
+    Assert.assertFalse(runPredicate("foo not in { 'casey' : 5 }", v -> variableMap.get(v)));
+    Assert.assertFalse(runPredicate("empty in { foo : 5 }", v -> variableMap.get(v)));
+    Assert.assertTrue(runPredicate("empty not in { foo : 5 }", v -> variableMap.get(v)));
+    Assert.assertFalse(runPredicate("'foo' in { }", v -> variableMap.get(v)));
+    Assert.assertFalse(runPredicate("null in { 'foo' : 5 }", v -> variableMap.get(v)));
+    Assert.assertTrue(runPredicate("null not in { 'foo' : 5 }", v -> variableMap.get(v)));
+  }
+
+  @Test
+  public void testInString() throws Exception {
+    final Map<String, String> variableMap = new HashMap<String, String>() {{
+      put("foo", "casey");
+      put("empty", "");
+    }};
+    Assert.assertTrue(runPredicate("'case' in foo", v -> variableMap.get(v)));
+    Assert.assertFalse(runPredicate("'case' not in foo", v -> variableMap.get(v)));
+    Assert.assertFalse(runPredicate("'case' in empty", v -> variableMap.get(v)));
+    Assert.assertTrue(runPredicate("'case' not in empty", v -> variableMap.get(v)));
+    Assert.assertFalse(runPredicate("'case' in [ foo ]", v -> variableMap.get(v)));
+    Assert.assertTrue(runPredicate("'case' not in [ foo ]", v -> variableMap.get(v)));
+    Assert.assertFalse(runPredicate("null in foo", v -> variableMap.get(v)));
+    Assert.assertTrue(runPredicate("null not in foo", v -> variableMap.get(v)));
   }
 
   @Test


Mime
View raw message