metron-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From shoggi <sho...@gmail.com>
Subject Re: Profiler issue
Date Fri, 24 Feb 2017 14:09:40 GMT
Sure

Here is the output..

# ./metron-deployment/scripts/platform-info.sh
Metron 0.3.0
--
* master
--
commit 59f8f03acbf66d5ac63f9156546833b0c98f57f2
Author: mmiklavc <michael.miklavcic@gmail.com>
Date:   Wed Nov 30 11:24:05 2016 -0700

    METRON-565: apps/metron/enrichment/indexed directory path does not get
created for metron cluster deployed via Ambari (mmiklavc) closes
apache/incubator-metron#365
--
 .../roles/elasticsearch/tasks/elasticsearch.yml    |    9 ++++-----
 .../roles/pcap_replay/templates/pcap-replay        |    9 ++++++---
 2 files changed, 10 insertions(+), 8 deletions(-)
--
./metron-deployment/scripts/platform-info.sh: line 48: ansible: command not
found
--
./metron-deployment/scripts/platform-info.sh: line 52: vagrant: command not
found
--
Python 2.6.6
--
Apache Maven 3.3.9 (bb52d8502b132ec0a5a3f4c09453c07478323dc5;
2015-11-10T17:41:47+01:00)
Maven home: /opt/apache-maven-3.3.9
Java version: 1.8.0_111, vendor: Oracle Corporation
Java home: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.111-0.b15.el6_8.x86_64/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "2.6.32-642.11.1.el6.x86_64", arch: "amd64",
family: "unix"
--
Linux x 2.6.32-642.11.1.el6.x86_64 #1 SMP Fri Nov 18 19:25:05 UTC 2016
x86_64 x86_64 x86_64 GNU/Linux



On Fri, Feb 24, 2017 at 2:55 PM, Nick Allen <nick@nickallen.org> wrote:

> Can you run the "./metron-deployment/scripts/platform-info.sh" script and
> send the output to us?
>
> On Fri, Feb 24, 2017 at 8:38 AM, shoggi <shoggi@gmail.com> wrote:
>
>> Hi All
>>
>> I tried to set up following profiler config.
>>
>> > CONFIG_GET("PROFILER")
>> {
>>   "profiles": [
>>     {
>>       "profile": "url-length",
>>       "foreach": "domain_without_subdomains",
>>       "onlyif": "source.type == 'squid'",
>>       "update": { "m": "STATS_ADD(m, LENGTH(url))" },
>>       "result": "m"
>>     }
>>   ]
>> }
>>
>> Whenever I send squid data through, I get a null pointer exception & the
>> profiler dies. Other enrichments on the squid feed work fine and the
>> enriched logs reaches elasticsearch. I followed this guide:
>> https://cwiki.apache.org/confluence/display/METRON/Profiling+Your+Streams to
>> check my syntax and proceedings.
>>
>> Any help is highly appreciated.
>>
>> Cheers
>> Shoji
>>
>>
>>
>> 2017-02-24 14:20:35.832 o.a.s.util [ERROR] Async loop died!
>> java.lang.RuntimeException: java.lang.NullPointerException
>>         at org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:464)
>> ~[storm-core-1.0.1.2.5.0.0-1245.jar:1.0.1.2.5.0.0-1245]
>>         at org.apache.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:430)
>> ~[storm-core-1.0.1.2.5.0.0-1245.jar:1.0.1.2.5.0.0-1245]
>>         at org.apache.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:73)
>> ~[storm-core-1.0.1.2.5.0.0-1245.jar:1.0.1.2.5.0.0-1245]
>>         at org.apache.storm.daemon.executor$fn__6571$fn__6584$fn__6637.invoke(executor.clj:853)
>> ~[storm-core-1.0.1.2.5.0.0-1245.jar:1.0.1.2.5.0.0-1245]
>>         at org.apache.storm.util$async_loop$fn__554.invoke(util.clj:484)
>> [storm-core-1.0.1.2.5.0.0-1245.jar:1.0.1.2.5.0.0-1245]
>>         at clojure.lang.AFn.run(AFn.java:22) [clojure-1.7.0.jar:?]
>>         at java.lang.Thread.run(Thread.java:745) [?:1.8.0_77]
>> Caused by: java.lang.NullPointerException
>>         at org.apache.metron.profiler.stellar.DefaultStellarExecutor.ex
>> ecute(DefaultStellarExecutor.java:117) ~[stormjar.jar:?]
>>         at org.apache.metron.profiler.bolt.ProfileSplitterBolt.applyPro
>> file(ProfileSplitterBolt.java:139) ~[stormjar.jar:?]
>>         at org.apache.metron.profiler.bolt.ProfileSplitterBolt.doExecut
>> e(ProfileSplitterBolt.java:122) ~[stormjar.jar:?]
>>         at org.apache.metron.profiler.bolt.ProfileSplitterBolt.execute(ProfileSplitterBolt.java:97)
>> ~[stormjar.jar:?]
>>         at org.apache.storm.daemon.executor$fn__6571$tuple_action_fn__6573.invoke(executor.clj:734)
>> ~[storm-core-1.0.1.2.5.0.0-1245.jar:1.0.1.2.5.0.0-1245]
>>         at org.apache.storm.daemon.executor$mk_task_receiver$fn__6492.invoke(executor.clj:466)
>> ~[storm-core-1.0.1.2.5.0.0-1245.jar:1.0.1.2.5.0.0-1245]
>>         at org.apache.storm.disruptor$clojure_handler$reify__6005.onEvent(disruptor.clj:40)
>> ~[storm-core-1.0.1.2.5.0.0-1245.jar:1.0.1.2.5.0.0-1245]
>>         at org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:451)
>> ~[storm-core-1.0.1.2.5.0.0-1245.jar:1.0.1.2.5.0.0-1245]
>>         ... 6 more
>> 2017-02-24 14:20:35.834 o.a.s.d.executor [ERROR]
>> java.lang.RuntimeException: java.lang.NullPointerException
>>         at org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:464)
>> ~[storm-core-1.0.1.2.5.0.0-1245.jar:1.0.1.2.5.0.0-1245]
>>         at org.apache.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:430)
>> ~[storm-core-1.0.1.2.5.0.0-1245.jar:1.0.1.2.5.0.0-1245]
>>         at org.apache.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:73)
>> ~[storm-core-1.0.1.2.5.0.0-1245.jar:1.0.1.2.5.0.0-1245]
>>         at org.apache.storm.daemon.executor$fn__6571$fn__6584$fn__6637.invoke(executor.clj:853)
>> ~[storm-core-1.0.1.2.5.0.0-1245.jar:1.0.1.2.5.0.0-1245]
>>         at org.apache.storm.util$async_loop$fn__554.invoke(util.clj:484)
>> [storm-core-1.0.1.2.5.0.0-1245.jar:1.0.1.2.5.0.0-1245]
>>         at clojure.lang.AFn.run(AFn.java:22) [clojure-1.7.0.jar:?]
>>         at java.lang.Thread.run(Thread.java:745) [?:1.8.0_77]
>> Caused by: java.lang.NullPointerException
>>         at org.apache.metron.profiler.stellar.DefaultStellarExecutor.ex
>> ecute(DefaultStellarExecutor.java:117) ~[stormjar.jar:?]
>>         at org.apache.metron.profiler.bolt.ProfileSplitterBolt.applyPro
>> file(ProfileSplitterBolt.java:139) ~[stormjar.jar:?]
>>         at org.apache.metron.profiler.bolt.ProfileSplitterBolt.doExecut
>> e(ProfileSplitterBolt.java:122) ~[stormjar.jar:?]
>>         at org.apache.metron.profiler.bolt.ProfileSplitterBolt.execute(ProfileSplitterBolt.java:97)
>> ~[stormjar.jar:?]
>>         at org.apache.storm.daemon.executor$fn__6571$tuple_action_fn__6573.invoke(executor.clj:734)
>> ~[storm-core-1.0.1.2.5.0.0-1245.jar:1.0.1.2.5.0.0-1245]
>>         at org.apache.storm.daemon.executor$mk_task_receiver$fn__6492.invoke(executor.clj:466)
>> ~[storm-core-1.0.1.2.5.0.0-1245.jar:1.0.1.2.5.0.0-1245]
>>         at org.apache.storm.disruptor$clojure_handler$reify__6005.onEvent(disruptor.clj:40)
>> ~[storm-core-1.0.1.2.5.0.0-1245.jar:1.0.1.2.5.0.0-1245]
>>         at org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:451)
>> ~[storm-core-1.0.1.2.5.0.0-1245.jar:1.0.1.2.5.0.0-1245]
>>         ... 6 more
>> 2017-02-24 14:20:35.908 o.a.s.util [ERROR] Halting process: ("Worker
>> died")
>> java.lang.RuntimeException: ("Worker died")
>>         at org.apache.storm.util$exit_process_BANG_.doInvoke(util.clj:341)
>> [storm-core-1.0.1.2.5.0.0-1245.jar:1.0.1.2.5.0.0-1245]
>>         at clojure.lang.RestFn.invoke(RestFn.java:423)
>> [clojure-1.7.0.jar:?]
>>         at org.apache.storm.daemon.worker$fn__7176$fn__7177.invoke(worker.clj:765)
>> [storm-core-1.0.1.2.5.0.0-1245.jar:1.0.1.2.5.0.0-1245]
>>         at org.apache.storm.daemon.executor$mk_executor_data$fn__6388$
>> fn__6389.invoke(executor.clj:275) [storm-core-1.0.1.2.5.0.0-1245
>> .jar:1.0.1.2.5.0.0-1245]
>>         at org.apache.storm.util$async_loop$fn__554.invoke(util.clj:494)
>> [storm-core-1.0.1.2.5.0.0-1245.jar:1.0.1.2.5.0.0-1245]
>>         at clojure.lang.AFn.run(AFn.java:22) [clojure-1.7.0.jar:?]
>>         at java.lang.Thread.run(Thread.java:745) [?:1.8.0_77]
>>
>>
>>
>>
>

Mime
View raw message