metron-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ceste...@apache.org
Subject metron git commit: METRON-1173: Fix pointers to old stellar docs closes apache/incubator-metron#746
Date Wed, 13 Sep 2017 23:58:22 GMT
Repository: metron
Updated Branches:
  refs/heads/master 2c56a13e5 -> 2ae1f5adf


METRON-1173: Fix pointers to old stellar docs closes apache/incubator-metron#746


Project: http://git-wip-us.apache.org/repos/asf/metron/repo
Commit: http://git-wip-us.apache.org/repos/asf/metron/commit/2ae1f5ad
Tree: http://git-wip-us.apache.org/repos/asf/metron/tree/2ae1f5ad
Diff: http://git-wip-us.apache.org/repos/asf/metron/diff/2ae1f5ad

Branch: refs/heads/master
Commit: 2ae1f5adf8428832550f99bcc4cc76645597851a
Parents: 2c56a13
Author: JonZeolla <zeolla@gmail.com>
Authored: Wed Sep 13 16:58:11 2017 -0700
Committer: cstella <cestella@gmail.com>
Committed: Wed Sep 13 16:58:11 2017 -0700

----------------------------------------------------------------------
 metron-analytics/metron-profiler-client/README.md                | 4 ++--
 .../src/main/config/zeppelin/metron/metron-pcap.json             | 2 +-
 metron-sensors/bro-plugin-kafka/README.md                        | 2 +-
 3 files changed, 4 insertions(+), 4 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/metron/blob/2ae1f5ad/metron-analytics/metron-profiler-client/README.md
----------------------------------------------------------------------
diff --git a/metron-analytics/metron-profiler-client/README.md b/metron-analytics/metron-profiler-client/README.md
index 27aa3f4..63d0cd4 100644
--- a/metron-analytics/metron-profiler-client/README.md
+++ b/metron-analytics/metron-profiler-client/README.md
@@ -305,7 +305,7 @@ The person doing the querying must carry forward the knowledge of the
Profiler c
 
 ### Examples
 
-The following are usage examples that show how the Stellar API can be used to read profiles
generated by the [Metron Profiler](../metron-profiler).  This API would be used in conjunction
with other Stellar functions like [`MAAS_MODEL_APPLY`](../../metron-platform/metron-common#maas_model_apply)
to perform model scoring on streaming data.
+The following are usage examples that show how the Stellar API can be used to read profiles
generated by the [Metron Profiler](../metron-profiler).  This API would be used in conjunction
with other Stellar functions like [`MAAS_MODEL_APPLY`](../../metron-stellar/stellar-common#maas_model_apply)
to perform model scoring on streaming data.
 
 These examples assume a profile has been defined called 'snort-alerts' that tracks the number
of Snort alerts associated with an IP address over time.  The profile definition might look
similar to the following.
 
@@ -462,4 +462,4 @@ Follow these steps in the Stellar REPL to see how it can be used to help
create
     
     This profile simply counts the number of messages by IP source address.  Notice that
the value is '3' for the entity '10.0.0.1' as we applied 3 messages with an 'ip_src_addr'
of '10.0.0.1'.  There will always be one measurement for each [profile, entity] pair.
     
-1. If you are unhappy with the data that has been generated, then 'wash, rinse and repeat'
this process.  Once you are happy with the profile that was created, follow the [Getting Started](../metron-profiler#getting-started)
guide to use the profile against your live, streaming data in a Metron cluster.
\ No newline at end of file
+1. If you are unhappy with the data that has been generated, then 'wash, rinse and repeat'
this process.  Once you are happy with the profile that was created, follow the [Getting Started](../metron-profiler#getting-started)
guide to use the profile against your live, streaming data in a Metron cluster.

http://git-wip-us.apache.org/repos/asf/metron/blob/2ae1f5ad/metron-platform/metron-pcap-backend/src/main/config/zeppelin/metron/metron-pcap.json
----------------------------------------------------------------------
diff --git a/metron-platform/metron-pcap-backend/src/main/config/zeppelin/metron/metron-pcap.json
b/metron-platform/metron-pcap-backend/src/main/config/zeppelin/metron/metron-pcap.json
index c18b5fb..447056f 100644
--- a/metron-platform/metron-pcap-backend/src/main/config/zeppelin/metron/metron-pcap.json
+++ b/metron-platform/metron-pcap-backend/src/main/config/zeppelin/metron/metron-pcap.json
@@ -1 +1 @@
-{"paragraphs":[{"text":"%md\n# Execute Packet Capture Queries\n\nSpecify the following
to filter the packet capture query:\n* *end time* - The ending time of the query in yyyyMMdd
format (e.g. 20170428)\n* *start time* - The starting time of the query in yyyyMMdd format
(e.g. 20170428)\n* *query* - The [Stellar](https://github.com/apache/metron/tree/master/metron-platform/metron-common#stellar-language)
query (i.e. a Stellar expression that returns `true` or `false`) to specify the packets.\n\nThe
available fields to use in the queries are as follows:\n* `ip_src_addr` - The source IP address
of the packets filtered\n* `ip_src_port` - The source port of the packets filtered\n* `ip_dst_addr`
- The destination IP address of the packets filtered\n* `ip_dst_port` - The destination port
of the packets filtered\n* `packet` - The raw packet (for use with the `BYTEARRAY_MATCHER`
function)\n\nYou can use any [Stellar function](https://github.com/apache/metron/tree/master/metron-platform/me
 tron-common#stellar-core-functions)\n\n## Simple Boolean Expressions\n\nFor example:\n* `ip_dst_port
== 8080` would return all packets where the destination port is `8080`\n* `ip_dst_port in
[ 8080, 80 ]` would return all packets where the destination port is either `8080` or `80`\n\n##
Common Network Functions on Metadata\n\nFor example:\n* `IN_SUBNET(ip_dst_addr, '192.168.0.0/24')`
would return all packets whose destination conforms to the CIDR `192.168.0.0/24`\n* `IN_SUBNET(ip_dst_addr,
'192.168.0.0/24') && ip_dst_port == 8080` would return all packets matching the CIDR
and whose destination port is `8080`\n \n## Filtering based on the Packet Contents\n\nWe use
byteseek regular expressions to filter packets.  The syntax for these is described [here](https://github.com/nishihatapalmer/byteseek/blob/master/sequencesyntax.md).\n*
`BYTEARRAY_MATCHER('\\\\'/api/v1\\\\'', packet)` would return all packets that contain the
string `/api/v1` in them anywhere.\n* `ip_dst_port==8080 && BYTE
 ARRAY_MATCHER('\\\\'/api/v1\\\\'', packet)` would return all packets that contain the string
`/api/v1` and have a destination port of `8080`\n* `BYTEARRAY_MATCHER('ff(.){5}ff', packet)`
would return all packets containing a binary regex with `0xff` followed by any 5 bytes and
then `0xff`\n\n# The Output\nThe output will be a table of links to the various parts of the
packet capture files.  The files will be named in temporal order.","dateUpdated":"2017-05-01T17:36:27+0000","config":{"colWidth":12,"editorMode":"ace/mode/markdown","editorHide":true,"graph":{"mode":"table","height":300,"optionOpen":false,"keys":[],"values":[],"groups":[],"scatter":{}},"enabled":true},"settings":{"params":{"query":"ip_dst_port==8080
&& BYTEARRAY_MATCHER('\\\\\\'/api/v1/\\\\\\'', packet)"},"forms":{}},"jobName":"paragraph_1493658709556_1010155640","id":"20170428-183346_777875025","result":{"code":"SUCCESS","type":"HTML","msg":"<h1>Execute
Packet Capture Queries</h1>\n<p>Specify the following to filter th
 e packet capture query:</p>\n<ul>\n<li><em>end time</em> -
The ending time of the query in yyyyMMdd format (e.g. 20170428)</li>\n<li><em>start
time</em> - The starting time of the query in yyyyMMdd format (e.g. 20170428)</li>\n<li><em>query</em>
- The <a href=\"https://github.com/apache/metron/tree/master/metron-platform/metron-common#stellar-language\">Stellar</a>
query (i.e. a Stellar expression that returns <code>true</code> or <code>false</code>)
to specify the packets.</li>\n</ul>\n<p>The available fields to use in the
queries are as follows:</p>\n<ul>\n<li><code>ip_src_addr</code>
- The source IP address of the packets filtered</li>\n<li><code>ip_src_port</code>
- The source port of the packets filtered</li>\n<li><code>ip_dst_addr</code>
- The destination IP address of the packets filtered</li>\n<li><code>ip_dst_port</code>
- The destination port of the packets filtered</li>\n<li><code>packet</code>
- The raw packet (for use with the <code>BYTEARRAY_MATCHER</code> function)</l
 i>\n</ul>\n<p>You can use any <a href=\"https://github.com/apache/metron/tree/master/metron-platform/metron-common#stellar-core-functions\">Stellar
function</a></p>\n<h2>Simple Boolean Expressions</h2>\n<p>For
example:</p>\n<ul>\n<li><code>ip_dst_port == 8080</code> would
return all packets where the destination port is <code>8080</code></li>\n<li><code>ip_dst_port
in [ 8080, 80 ]</code> would return all packets where the destination port is either
<code>8080</code> or <code>80</code></li>\n</ul>\n<h2>Common
Network Functions on Metadata</h2>\n<p>For example:</p>\n<ul>\n<li><code>IN_SUBNET(ip_dst_addr,
'192.168.0.0/24')</code> would return all packets whose destination conforms to the
CIDR <code>192.168.0.0/24</code></li>\n<li><code>IN_SUBNET(ip_dst_addr,
'192.168.0.0/24') &amp;&amp; ip_dst_port == 8080</code> would return all packets
matching the CIDR and whose destination port is <code>8080</code></li>\n</ul>\n<h2>Filtering
based on the Packet Contents</h2>\n<p>We use byteseek reg
 ular expressions to filter packets.  The syntax for these is described <a href=\"https://github.com/nishihatapalmer/byteseek/blob/master/sequencesyntax.md\">here</a>.</p>\n<ul>\n<li><code>BYTEARRAY_MATCHER('\\\\'/api/v1\\\\'',
packet)</code> would return all packets that contain the string <code>/api/v1</code>
in them anywhere.</li>\n<li><code>ip_dst_port==8080 &amp;&amp; BYTEARRAY_MATCHER('\\\\'/api/v1\\\\'',
packet)</code> would return all packets that contain the string <code>/api/v1</code>
and have a destination port of <code>8080</code></li>\n<li><code>BYTEARRAY_MATCHER('ff(.){5}ff',
packet)</code> would return all packets containing a binary regex with <code>0xff</code>
followed by any 5 bytes and then <code>0xff</code></li>\n</ul>\n<h1>The
Output</h1>\n<p>The output will be a table of links to the various parts of the
packet capture files.  The files will be named in temporal order.</p>\n"},"dateCreated":"2017-05-01T17:11:49+0000","status":"FINISHED","progressUpdateIntervalMs
 ":500,"$$hashKey":"object:1724","dateFinished":"2017-05-01T17:36:26+0000","dateStarted":"2017-05-01T17:36:26+0000","focus":true},{"text":"%sh\nexport
PCAP_ZEPPELIN_RUN=$(find /usr -name pcap_zeppelin_run.sh)\nexport RECORDS_PER_FILE=10000\nexport
NUMBER_OF_REDUCERS=10\nexport DATE_FORMAT=\"yyyyMMdd\"\nexport PCAP_DATA_PATH=\"/apps/metron/pcap\"\n\n$PCAP_ZEPPELIN_RUN
\"${query}\" \"${start time}\" \"${optional end time}\"","dateUpdated":"2017-05-01T17:35:04+0000","config":{"colWidth":12,"editorMode":"ace/mode/sh","editorHide":false,"graph":{"mode":"table","height":164,"optionOpen":false,"keys":[{"name":"Packets
conforming to BYTEARRAY_MATCHER('\\'/api/v1\\'', packet) && ip_dst_port == 8080 starting
at 20170428 ending now","index":0,"aggr":"sum"}],"values":[],"groups":[],"scatter":{"xAxis":{"name":"Packets
conforming to BYTEARRAY_MATCHER('\\'/api/v1\\'', packet) && ip_dst_port == 8080 starting
at 20170428 ending now","index":0,"aggr":"sum"}}},"enabled":true},"settings":{"params":{"sta
 rt time":"20170428","end time":"","query":"BYTEARRAY_MATCHER('\\\\'/api/v1\\\\'', packet)
&& ip_dst_port == 8080","optional end time":""},"forms":{"optional end time":{"name":"optional
end time","defaultValue":"","hidden":false},"query":{"name":"query","defaultValue":"","hidden":false},"start
time":{"name":"start time","defaultValue":"","hidden":false}}},"jobName":"paragraph_1493658709562_1009386142","id":"20170428-181957_829993114","result":{"code":"SUCCESS","type":"TABLE","msg":"Packets
conforming to BYTEARRAY_MATCHER('\\'/api/v1\\'', packet) && ip_dst_port == 8080 starting
at 20170428 ending now\n%html <a href=\"http://node1:50070/webhdfs/v1/user/zeppelin/queries/pcap-data-20170501173442578/pcap-data-20170501173442578%2B0000.pcap?op=OPEN\">pcap-data-20170501173442578+0000.pcap</a>\n","comment":"","msgTable":[[{"value":"%html
<a href=\"http://node1:50070/webhdfs/v1/user/zeppelin/queries/pcap-data-20170501173442578/pcap-data-20170501173442578%2B0000.pcap?op=OPEN\">pcap-data-2017050
 1173442578+0000.pcap</a>"}]],"columnNames":[{"name":"Packets conforming to BYTEARRAY_MATCHER('\\'/api/v1\\'',
packet) && ip_dst_port == 8080 starting at 20170428 ending now","index":0,"aggr":"sum"}],"rows":[["%html
<a href=\"http://node1:50070/webhdfs/v1/user/zeppelin/queries/pcap-data-20170501173442578/pcap-data-20170501173442578%2B0000.pcap?op=OPEN\">pcap-data-20170501173442578+0000.pcap</a>"]]},"dateCreated":"2017-05-01T17:11:49+0000","dateStarted":"2017-05-01T17:35:04+0000","dateFinished":"2017-05-01T17:34:55+0000","status":"RUNNING","progressUpdateIntervalMs":500,"$$hashKey":"object:1725","focus":true},{"dateUpdated":"2017-05-01T17:34:55+0000","config":{"colWidth":12,"graph":{"mode":"table","height":300,"optionOpen":false,"keys":[],"values":[],"groups":[],"scatter":{}},"enabled":true,"editorMode":"ace/mode/markdown","editorHide":true},"settings":{"params":{},"forms":{}},"jobName":"paragraph_1493658709563_1009001393","id":"20170428-184335_1604096389","dateCreated":"2017-05-01T17
 :11:49+0000","status":"FINISHED","progressUpdateIntervalMs":500,"$$hashKey":"object:1726","dateFinished":"2017-05-01T17:34:54+0000","dateStarted":"2017-05-01T17:34:54+0000","result":{"code":"SUCCESS","type":"HTML","msg":"<h1>Troubleshooting</h1>\n<p>If
you are having problems with the above form, the following may help.</p>\n<h2>I
see <code>Terminated by SIGINTERRUPT</code> or something similar in the output!</h2>\n<p>PCAP
filtering happens via a batch process and on busy systems, this can take some time.  You very
well may need to request this of your system administrator.\n<br  />They can do this
by changing the <code>shell.command.timeout.millisecs</code> property for the
<code>sh</code> interpreter to a larger value, likely <code>100000</code>.</p>\n<h2>I
do not see a table of URLs to pcap files in my output, what happened?</h2>\n<p>If
an error happens, the log of the pcap querying utility will be displayed instead of an output.
 Please contact an administrator with this output 
 to debug further.</p>\n"},"text":"%md\n# Troubleshooting\n\nIf you are having problems
with the above form, the following may help.\n\n## I see `Terminated by SIGINTERRUPT` or something
similar in the output!\n\nPCAP filtering happens via a batch process and on busy systems,
this can take some time.  You very well may need to request this of your system administrator.\nThey
can do this by changing the `shell.command.timeout.millisecs` property for the `sh` interpreter
to a larger value, likely `100000`.\n\n## I do not see a table of URLs to pcap files in my
output, what happened?\n\nIf an error happens, the log of the pcap querying utility will be
displayed instead of an output.  Please contact an administrator with this output to debug
further.","focus":true},{"config":{"colWidth":12,"graph":{"mode":"table","height":300,"optionOpen":false,"keys":[],"values":[],"groups":[],"scatter":{}},"enabled":true},"settings":{"params":{},"forms":{}},"jobName":"paragraph_1493659778398_1082698332
 ","id":"20170501-172938_213921861","dateCreated":"2017-05-01T17:29:38+0000","status":"READY","progressUpdateIntervalMs":500,"focus":true,"$$hashKey":"object:1972"}],"name":"metron/pcap","id":"2CEEXVR1W","angularObjects":{"2CFNGE6TP:shared_process":[],"2CGUX5TSW:shared_process":[],"2CETVR2AB:shared_process":[],"2CF163WFX:shared_process":[],"2CG4YXKUV:shared_process":[],"2CG6QDFF7:shared_process":[]},"config":{"looknfeel":"default"},"info":{}}
\ No newline at end of file
+{"paragraphs":[{"text":"%md\n# Execute Packet Capture Queries\n\nSpecify the following
to filter the packet capture query:\n* *end time* - The ending time of the query in yyyyMMdd
format (e.g. 20170428)\n* *start time* - The starting time of the query in yyyyMMdd format
(e.g. 20170428)\n* *query* - The [Stellar](https://github.com/apache/metron/tree/master/metron-platform/metron-common#stellar-language)
query (i.e. a Stellar expression that returns `true` or `false`) to specify the packets.\n\nThe
available fields to use in the queries are as follows:\n* `ip_src_addr` - The source IP address
of the packets filtered\n* `ip_src_port` - The source port of the packets filtered\n* `ip_dst_addr`
- The destination IP address of the packets filtered\n* `ip_dst_port` - The destination port
of the packets filtered\n* `packet` - The raw packet (for use with the `BYTEARRAY_MATCHER`
function)\n\nYou can use any [Stellar function](https://github.com/apache/metron/tree/master/metron-stellar/ste
 llar-common#stellar-core-functions)\n\n## Simple Boolean Expressions\n\nFor example:\n* `ip_dst_port
== 8080` would return all packets where the destination port is `8080`\n* `ip_dst_port in
[ 8080, 80 ]` would return all packets where the destination port is either `8080` or `80`\n\n##
Common Network Functions on Metadata\n\nFor example:\n* `IN_SUBNET(ip_dst_addr, '192.168.0.0/24')`
would return all packets whose destination conforms to the CIDR `192.168.0.0/24`\n* `IN_SUBNET(ip_dst_addr,
'192.168.0.0/24') && ip_dst_port == 8080` would return all packets matching the CIDR
and whose destination port is `8080`\n \n## Filtering based on the Packet Contents\n\nWe use
byteseek regular expressions to filter packets.  The syntax for these is described [here](https://github.com/nishihatapalmer/byteseek/blob/master/sequencesyntax.md).\n*
`BYTEARRAY_MATCHER('\\\\'/api/v1\\\\'', packet)` would return all packets that contain the
string `/api/v1` in them anywhere.\n* `ip_dst_port==8080 && BYTE
 ARRAY_MATCHER('\\\\'/api/v1\\\\'', packet)` would return all packets that contain the string
`/api/v1` and have a destination port of `8080`\n* `BYTEARRAY_MATCHER('ff(.){5}ff', packet)`
would return all packets containing a binary regex with `0xff` followed by any 5 bytes and
then `0xff`\n\n# The Output\nThe output will be a table of links to the various parts of the
packet capture files.  The files will be named in temporal order.","dateUpdated":"2017-05-01T17:36:27+0000","config":{"colWidth":12,"editorMode":"ace/mode/markdown","editorHide":true,"graph":{"mode":"table","height":300,"optionOpen":false,"keys":[],"values":[],"groups":[],"scatter":{}},"enabled":true},"settings":{"params":{"query":"ip_dst_port==8080
&& BYTEARRAY_MATCHER('\\\\\\'/api/v1/\\\\\\'', packet)"},"forms":{}},"jobName":"paragraph_1493658709556_1010155640","id":"20170428-183346_777875025","result":{"code":"SUCCESS","type":"HTML","msg":"<h1>Execute
Packet Capture Queries</h1>\n<p>Specify the following to filter th
 e packet capture query:</p>\n<ul>\n<li><em>end time</em> -
The ending time of the query in yyyyMMdd format (e.g. 20170428)</li>\n<li><em>start
time</em> - The starting time of the query in yyyyMMdd format (e.g. 20170428)</li>\n<li><em>query</em>
- The <a href=\"https://github.com/apache/metron/tree/master/metron-platform/metron-common#stellar-language\">Stellar</a>
query (i.e. a Stellar expression that returns <code>true</code> or <code>false</code>)
to specify the packets.</li>\n</ul>\n<p>The available fields to use in the
queries are as follows:</p>\n<ul>\n<li><code>ip_src_addr</code>
- The source IP address of the packets filtered</li>\n<li><code>ip_src_port</code>
- The source port of the packets filtered</li>\n<li><code>ip_dst_addr</code>
- The destination IP address of the packets filtered</li>\n<li><code>ip_dst_port</code>
- The destination port of the packets filtered</li>\n<li><code>packet</code>
- The raw packet (for use with the <code>BYTEARRAY_MATCHER</code> function)</l
 i>\n</ul>\n<p>You can use any <a href=\"https://github.com/apache/metron/tree/master/metron-stellar/stellar-common#stellar-core-functions\">Stellar
function</a></p>\n<h2>Simple Boolean Expressions</h2>\n<p>For
example:</p>\n<ul>\n<li><code>ip_dst_port == 8080</code> would
return all packets where the destination port is <code>8080</code></li>\n<li><code>ip_dst_port
in [ 8080, 80 ]</code> would return all packets where the destination port is either
<code>8080</code> or <code>80</code></li>\n</ul>\n<h2>Common
Network Functions on Metadata</h2>\n<p>For example:</p>\n<ul>\n<li><code>IN_SUBNET(ip_dst_addr,
'192.168.0.0/24')</code> would return all packets whose destination conforms to the
CIDR <code>192.168.0.0/24</code></li>\n<li><code>IN_SUBNET(ip_dst_addr,
'192.168.0.0/24') &amp;&amp; ip_dst_port == 8080</code> would return all packets
matching the CIDR and whose destination port is <code>8080</code></li>\n</ul>\n<h2>Filtering
based on the Packet Contents</h2>\n<p>We use byteseek reg
 ular expressions to filter packets.  The syntax for these is described <a href=\"https://github.com/nishihatapalmer/byteseek/blob/master/sequencesyntax.md\">here</a>.</p>\n<ul>\n<li><code>BYTEARRAY_MATCHER('\\\\'/api/v1\\\\'',
packet)</code> would return all packets that contain the string <code>/api/v1</code>
in them anywhere.</li>\n<li><code>ip_dst_port==8080 &amp;&amp; BYTEARRAY_MATCHER('\\\\'/api/v1\\\\'',
packet)</code> would return all packets that contain the string <code>/api/v1</code>
and have a destination port of <code>8080</code></li>\n<li><code>BYTEARRAY_MATCHER('ff(.){5}ff',
packet)</code> would return all packets containing a binary regex with <code>0xff</code>
followed by any 5 bytes and then <code>0xff</code></li>\n</ul>\n<h1>The
Output</h1>\n<p>The output will be a table of links to the various parts of the
packet capture files.  The files will be named in temporal order.</p>\n"},"dateCreated":"2017-05-01T17:11:49+0000","status":"FINISHED","progressUpdateIntervalMs
 ":500,"$$hashKey":"object:1724","dateFinished":"2017-05-01T17:36:26+0000","dateStarted":"2017-05-01T17:36:26+0000","focus":true},{"text":"%sh\nexport
PCAP_ZEPPELIN_RUN=$(find /usr -name pcap_zeppelin_run.sh)\nexport RECORDS_PER_FILE=10000\nexport
NUMBER_OF_REDUCERS=10\nexport DATE_FORMAT=\"yyyyMMdd\"\nexport PCAP_DATA_PATH=\"/apps/metron/pcap\"\n\n$PCAP_ZEPPELIN_RUN
\"${query}\" \"${start time}\" \"${optional end time}\"","dateUpdated":"2017-05-01T17:35:04+0000","config":{"colWidth":12,"editorMode":"ace/mode/sh","editorHide":false,"graph":{"mode":"table","height":164,"optionOpen":false,"keys":[{"name":"Packets
conforming to BYTEARRAY_MATCHER('\\'/api/v1\\'', packet) && ip_dst_port == 8080 starting
at 20170428 ending now","index":0,"aggr":"sum"}],"values":[],"groups":[],"scatter":{"xAxis":{"name":"Packets
conforming to BYTEARRAY_MATCHER('\\'/api/v1\\'', packet) && ip_dst_port == 8080 starting
at 20170428 ending now","index":0,"aggr":"sum"}}},"enabled":true},"settings":{"params":{"sta
 rt time":"20170428","end time":"","query":"BYTEARRAY_MATCHER('\\\\'/api/v1\\\\'', packet)
&& ip_dst_port == 8080","optional end time":""},"forms":{"optional end time":{"name":"optional
end time","defaultValue":"","hidden":false},"query":{"name":"query","defaultValue":"","hidden":false},"start
time":{"name":"start time","defaultValue":"","hidden":false}}},"jobName":"paragraph_1493658709562_1009386142","id":"20170428-181957_829993114","result":{"code":"SUCCESS","type":"TABLE","msg":"Packets
conforming to BYTEARRAY_MATCHER('\\'/api/v1\\'', packet) && ip_dst_port == 8080 starting
at 20170428 ending now\n%html <a href=\"http://node1:50070/webhdfs/v1/user/zeppelin/queries/pcap-data-20170501173442578/pcap-data-20170501173442578%2B0000.pcap?op=OPEN\">pcap-data-20170501173442578+0000.pcap</a>\n","comment":"","msgTable":[[{"value":"%html
<a href=\"http://node1:50070/webhdfs/v1/user/zeppelin/queries/pcap-data-20170501173442578/pcap-data-20170501173442578%2B0000.pcap?op=OPEN\">pcap-data-2017050
 1173442578+0000.pcap</a>"}]],"columnNames":[{"name":"Packets conforming to BYTEARRAY_MATCHER('\\'/api/v1\\'',
packet) && ip_dst_port == 8080 starting at 20170428 ending now","index":0,"aggr":"sum"}],"rows":[["%html
<a href=\"http://node1:50070/webhdfs/v1/user/zeppelin/queries/pcap-data-20170501173442578/pcap-data-20170501173442578%2B0000.pcap?op=OPEN\">pcap-data-20170501173442578+0000.pcap</a>"]]},"dateCreated":"2017-05-01T17:11:49+0000","dateStarted":"2017-05-01T17:35:04+0000","dateFinished":"2017-05-01T17:34:55+0000","status":"RUNNING","progressUpdateIntervalMs":500,"$$hashKey":"object:1725","focus":true},{"dateUpdated":"2017-05-01T17:34:55+0000","config":{"colWidth":12,"graph":{"mode":"table","height":300,"optionOpen":false,"keys":[],"values":[],"groups":[],"scatter":{}},"enabled":true,"editorMode":"ace/mode/markdown","editorHide":true},"settings":{"params":{},"forms":{}},"jobName":"paragraph_1493658709563_1009001393","id":"20170428-184335_1604096389","dateCreated":"2017-05-01T17
 :11:49+0000","status":"FINISHED","progressUpdateIntervalMs":500,"$$hashKey":"object:1726","dateFinished":"2017-05-01T17:34:54+0000","dateStarted":"2017-05-01T17:34:54+0000","result":{"code":"SUCCESS","type":"HTML","msg":"<h1>Troubleshooting</h1>\n<p>If
you are having problems with the above form, the following may help.</p>\n<h2>I
see <code>Terminated by SIGINTERRUPT</code> or something similar in the output!</h2>\n<p>PCAP
filtering happens via a batch process and on busy systems, this can take some time.  You very
well may need to request this of your system administrator.\n<br  />They can do this
by changing the <code>shell.command.timeout.millisecs</code> property for the
<code>sh</code> interpreter to a larger value, likely <code>100000</code>.</p>\n<h2>I
do not see a table of URLs to pcap files in my output, what happened?</h2>\n<p>If
an error happens, the log of the pcap querying utility will be displayed instead of an output.
 Please contact an administrator with this output 
 to debug further.</p>\n"},"text":"%md\n# Troubleshooting\n\nIf you are having problems
with the above form, the following may help.\n\n## I see `Terminated by SIGINTERRUPT` or something
similar in the output!\n\nPCAP filtering happens via a batch process and on busy systems,
this can take some time.  You very well may need to request this of your system administrator.\nThey
can do this by changing the `shell.command.timeout.millisecs` property for the `sh` interpreter
to a larger value, likely `100000`.\n\n## I do not see a table of URLs to pcap files in my
output, what happened?\n\nIf an error happens, the log of the pcap querying utility will be
displayed instead of an output.  Please contact an administrator with this output to debug
further.","focus":true},{"config":{"colWidth":12,"graph":{"mode":"table","height":300,"optionOpen":false,"keys":[],"values":[],"groups":[],"scatter":{}},"enabled":true},"settings":{"params":{},"forms":{}},"jobName":"paragraph_1493659778398_1082698332
 ","id":"20170501-172938_213921861","dateCreated":"2017-05-01T17:29:38+0000","status":"READY","progressUpdateIntervalMs":500,"focus":true,"$$hashKey":"object:1972"}],"name":"metron/pcap","id":"2CEEXVR1W","angularObjects":{"2CFNGE6TP:shared_process":[],"2CGUX5TSW:shared_process":[],"2CETVR2AB:shared_process":[],"2CF163WFX:shared_process":[],"2CG4YXKUV:shared_process":[],"2CG6QDFF7:shared_process":[]},"config":{"looknfeel":"default"},"info":{}}

http://git-wip-us.apache.org/repos/asf/metron/blob/2ae1f5ad/metron-sensors/bro-plugin-kafka/README.md
----------------------------------------------------------------------
diff --git a/metron-sensors/bro-plugin-kafka/README.md b/metron-sensors/bro-plugin-kafka/README.md
index e219360..427988b 100644
--- a/metron-sensors/bro-plugin-kafka/README.md
+++ b/metron-sensors/bro-plugin-kafka/README.md
@@ -141,7 +141,7 @@ event bro_init() &priority=-5
 
 #### Notes
  * `logs_to_send` is mutually exclusive with `$pred`, thus for each log you want to set `$pred`
on, you must individually setup a `Log::add_filter` and refrain from including that log in
`logs_to_send`.
- * You can also filter IPv6 logs from within your Metron cluster [using Stellar](../../metron-platform/metron-common#IS_IP).
 In that case, you wouldn't apply a predicate in your bro configuration, and instead Stellar
would filter the logs out before they were processed by the enrichment layer of Metron.
+ * You can also filter IPv6 logs from within your Metron cluster [using Stellar](../../metron-stellar/stellar-common#IS_IP).
 In that case, you wouldn't apply a predicate in your bro configuration, and instead Stellar
would filter the logs out before they were processed by the enrichment layer of Metron.
  * It is also possible to use the `is_v6_subnet()` bro function in your predicate, as of
their [2.5 release](https://www.bro.org/sphinx-git/install/release-notes.html#bro-2-5), however
the above example should work on [bro 2.4](https://www.bro.org/sphinx-git/install/release-notes.html#bro-2-4)
and newer, which has been the focus of the kafka plugin.
 
 Settings


Mime
View raw message