metron-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Casey Stella <ceste...@gmail.com>
Subject Re: Median Absolute Deviation - Profiler Error
Date Tue, 28 Feb 2017 14:23:14 GMT
So, I do recall in 0.3.0 a bug where if you modify a profile, it still
continues to run the old copy of the profile.  I was going to have you
bounce the profiler topology.  Did you do that?  If not, then please do
it.  If so, then are the NPE's the same?  I've seen 2 distinct exceptions
in the stacktraces, one around casting a MAD$State objct to a List and
another as a real NPE.

Let's level-set, can you bounce the profiler topology and then send the NPE
exception and stack trace?

Casey

On Tue, Feb 28, 2017 at 3:56 AM, shoggi <shoggi@gmail.com> wrote:

> I restarted Storm and ingest some data over the last few hours. I still
> get the casual NPE but some data gets profiled. It definitely skips data as
> I fed 160 unique tld's but only get 9 mad profile entries. The list seems
> random as I would least expect my top talkers to show up.
>
>
> [Stellar]>>> OUTLIER_MAD_STATE_MERGE(PROFILE_GET('sketchy_mad', 'krxd.net',
> 5, 'HOURS'))
> org.apache.metron.statistics.outlier.MedianAbsoluteDeviationFunctio
> ns$State@5b307115
> [Stellar]>>> PROFILE_GET('sketchy_mad', 'krxd.net', 5, 'HOURS')
> []
> [Stellar]>>> OUTLIER_MAD_STATE_MERGE(PROFILE_GET('sketchy_mad', 'krxd.net',
> 5, 'MINUTES'))
> org.apache.metron.statistics.outlier.MedianAbsoluteDeviationFunctio
> ns$State@714c575b
> [Stellar]>>> PROFILE_GET('sketchy_mad', 'krxd.net', 5, 'MINUTES')
> []
>
> [Stellar]>>> bytes := 700
> [Stellar]>>> OUTLIER_MAD_SCORE( OUTLIER_MAD_STATE_MERGE( PROFILE_GET(
> 'sketchy_mad', 'krxd.net', 5, 'MINUTES')), bytes)
> NaN
> [Stellar]>>> OUTLIER_MAD_SCORE( OUTLIER_MAD_STATE_MERGE( PROFILE_GET(
> 'sketchy_mad', 'krxd.net', 5, 'HOURS')), bytes)
> NaN
> [Stellar]>>> OUTLIER_MAD_SCORE( OUTLIER_MAD_STATE_MERGE( PROFILE_GET(
> 'sketchy_mad', 'krxd.net', 5, 'DAYS')), bytes)
> 3.8971111111111107
>
>
> # echo "scan 'profiler'" | hbase shell > /tmp/test.txt && egrep
> 'sketchy_mad' /tmp/test.txt | awk '{print $1}'
> \x00\x00\x00xsketchy_madoptimicdn.com\x00\x00\x00\x00\x00\x19;s
> \x00\x00\x01dsketchy_madajax.googleapis.com\x00\x00\x00\x00\x00\x19;g
> \x00\x00\x02\x86sketchy_madrlcdn.com\x00\x00\x00\x00\x00\x19;c
> \x00\x00\x02\x86sketchy_madshareaholic.com\x00\x00\x00\x00\x00\x19;c
> \x00\x00\x02\xDBsketchy_madnewsnetz.ch\x00\x00\x00\x00\x00\x19;k
> \xFF\xFF\xFC\x8Esketchy_madajax.googleapis.com\x00\x00\x00\x00\x00\x19;r
> \xFF\xFF\xFC\xEBsketchy_madkrxd.net\x00\x00\x00\x00\x00\x19;[
> \xFF\xFF\xFE\x87sketchy_madkrxd.net\x00\x00\x00\x00\x00\x19;^
> \xFF\xFF\xFE\xF2sketchy_madoptimicdn.com\x00\x00\x00\x00\x00\x19;\x5C
>
> On Tue, Feb 28, 2017 at 8:10 AM, Casey Stella <cestella@gmail.com> wrote:
>
>> Ok, from the Stellar REPL (started with -z and the zookeeper quorum),
>> what does the following yield:
>>
>>    - OUTLIER_MAD_STATE_MERGE(PROFILE_GET('sketchy_mad', 'krxd.net', 5,
>>    'HOURS'))
>>    - PROFILE_GET('sketchy_mad', 'krxd.net', 5, 'HOURS')
>>    - OUTLIER_MAD_STATE_MERGE(PROFILE_GET('sketchy_mad', 'krxd.net', 5,
>>    'MINUTES'))
>>    - PROFILE_GET('sketchy_mad', 'krxd.net', 5, 'MINUTES')
>>
>>
>> On Tue, Feb 28, 2017 at 1:36 AM, shoggi <shoggi@gmail.com> wrote:
>>
>>> This is what I've got..
>>>
>>>     {
>>>       "profile": "sketchy_mad",
>>>       "foreach": "if exists(domain_without_subdomains) then
>>> domain_without_subdomains else 'n/a'",
>>>       "onlyif": "exists(domain_without_subdomains) && source.type ==
>>> 'squid'",
>>>       "init" : {
>>>         "s": "OUTLIER_MAD_STATE_MERGE(PROFILE_GET('sketchy_mad',
>>> domain_without_subdomains, 5, 'MINUTES'))"
>>>       },
>>>       "update": {
>>>         "s": "OUTLIER_MAD_ADD(s, bytes)"
>>>       },
>>>       "result": "s"
>>>     }
>>>
>>> On Tue, Feb 28, 2017 at 3:34 AM, Casey Stella <cestella@gmail.com>
>>> wrote:
>>>
>>>> Sorry, waht does the current profile config look like for that very
>>>> last error?
>>>>
>>>> On Mon, Feb 27, 2017 at 6:39 PM, shoggi <shoggi@gmail.com> wrote:
>>>>
>>>>> Seems not to be the culprit..
>>>>>
>>>>>
>>>>> 2017-02-28 00:35:43.367 o.a.m.p.b.ProfileBuilderBolt [INFO] Flushing
>>>>> profile: profile=sketchy_mad, entity=optimicdn.com
>>>>> 2017-02-28 00:35:43.368 o.a.m.p.b.ProfileBuilderBolt [ERROR]
>>>>> Unexpected failure: message='Unable to execute:
>>>>> org.apache.metron.statistics.outlier.MedianAbsoluteDeviationFunctions$State
>>>>> cannot be cast to java.util.List', tuple='source: __system:-1, stream:
>>>>> __tick, id: {}, [900]'
>>>>> org.apache.metron.common.dsl.ParseException: Unable to execute:
>>>>> org.apache.metron.statistics.outlier.MedianAbsoluteDeviationFunctions$State
>>>>> cannot be cast to java.util.List
>>>>> at org.apache.metron.common.stellar.StellarCompiler.getResult(StellarCompiler.java:428)
>>>>> ~[stormjar.jar:?]
>>>>> at org.apache.metron.common.stellar.BaseStellarProcessor.parse(BaseStellarProcessor.java:90)
>>>>> ~[stormjar.jar:?]
>>>>> at org.apache.metron.profiler.stellar.DefaultStellarExecutor.ex
>>>>> ecute(DefaultStellarExecutor.java:148) ~[stormjar.jar:?]
>>>>> at org.apache.metron.profiler.stellar.DefaultStellarExecutor.as
>>>>> sign(DefaultStellarExecutor.java:94) ~[stormjar.jar:?]
>>>>> at org.apache.metron.profiler.bolt.ProfileBuilderBolt.lambda$nu
>>>>> ll$2(ProfileBuilderBolt.java:258) ~[stormjar.jar:?]
>>>>> at java.util.LinkedHashMap.forEach(LinkedHashMap.java:676)
>>>>> ~[?:1.8.0_77]
>>>>> at org.apache.metron.profiler.bolt.ProfileBuilderBolt.lambda$fl
>>>>> ush$4(ProfileBuilderBolt.java:258) ~[stormjar.jar:?]
>>>>> at java.util.concurrent.ConcurrentMap.forEach(ConcurrentMap.java:114)
>>>>> ~[?:1.8.0_77]
>>>>> at org.apache.metron.profiler.bolt.ProfileBuilderBolt.flush(ProfileBuilderBolt.java:237)
>>>>> ~[stormjar.jar:?]
>>>>> at org.apache.metron.profiler.bolt.ProfileBuilderBolt.doExecute(ProfileBuilderBolt.java:164)
>>>>> ~[stormjar.jar:?]
>>>>> at org.apache.metron.profiler.bolt.ProfileBuilderBolt.execute(ProfileBuilderBolt.java:144)
>>>>> [stormjar.jar:?]
>>>>> at org.apache.storm.daemon.executor$fn__6571$tuple_action_fn__6573.invoke(executor.clj:734)
>>>>> [storm-core-1.0.1.2.5.0.0-1245.jar:1.0.1.2.5.0.0-1245]
>>>>> at org.apache.storm.daemon.executor$mk_task_receiver$fn__6492.invoke(executor.clj:469)
>>>>> [storm-core-1.0.1.2.5.0.0-1245.jar:1.0.1.2.5.0.0-1245]
>>>>> at org.apache.storm.disruptor$clojure_handler$reify__6005.onEvent(disruptor.clj:40)
>>>>> [storm-core-1.0.1.2.5.0.0-1245.jar:1.0.1.2.5.0.0-1245]
>>>>> at org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:451)
>>>>> [storm-core-1.0.1.2.5.0.0-1245.jar:1.0.1.2.5.0.0-1245]
>>>>> at org.apache.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:430)
>>>>> [storm-core-1.0.1.2.5.0.0-1245.jar:1.0.1.2.5.0.0-1245]
>>>>> at org.apache.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:73)
>>>>> [storm-core-1.0.1.2.5.0.0-1245.jar:1.0.1.2.5.0.0-1245]
>>>>> at org.apache.storm.daemon.executor$fn__6571$fn__6584$fn__6637.invoke(executor.clj:853)
>>>>> [storm-core-1.0.1.2.5.0.0-1245.jar:1.0.1.2.5.0.0-1245]
>>>>> at org.apache.storm.util$async_loop$fn__554.invoke(util.clj:484)
>>>>> [storm-core-1.0.1.2.5.0.0-1245.jar:1.0.1.2.5.0.0-1245]
>>>>> at clojure.lang.AFn.run(AFn.java:22) [clojure-1.7.0.jar:?]
>>>>> at java.lang.Thread.run(Thread.java:745) [?:1.8.0_77]
>>>>> Caused by: java.lang.ClassCastException: org.apache.metron.statistics.o
>>>>> utlier.MedianAbsoluteDeviationFunctions$State cannot be cast to
>>>>> java.util.List
>>>>> at org.apache.metron.statistics.outlier.MedianAbsoluteDeviation
>>>>> Functions$StateUpdate.apply(MedianAbsoluteDeviationFunctions.java:85)
>>>>> ~[stormjar.jar:?]
>>>>> at org.apache.metron.common.stellar.StellarCompiler.exitTransfo
>>>>> rmationFunc(StellarCompiler.java:246) ~[stormjar.jar:?]
>>>>> at org.apache.metron.common.stellar.generated.StellarParser$Tra
>>>>> nsformationFuncContext.exitRule(StellarParser.java:1617)
>>>>> ~[stormjar.jar:?]
>>>>> at org.antlr.v4.runtime.Parser.triggerExitRuleEvent(Parser.java:422)
>>>>> ~[stormjar.jar:?]
>>>>> at org.antlr.v4.runtime.Parser.exitRule(Parser.java:632)
>>>>> ~[stormjar.jar:?]
>>>>> at org.apache.metron.common.stellar.generated.StellarParser.fun
>>>>> ctions(StellarParser.java:1640) ~[stormjar.jar:?]
>>>>> at org.apache.metron.common.stellar.generated.StellarParser.ari
>>>>> thmetic_operands(StellarParser.java:1750) ~[stormjar.jar:?]
>>>>> at org.apache.metron.common.stellar.generated.StellarParser.ari
>>>>> thmetic_expr_mul(StellarParser.java:1537) ~[stormjar.jar:?]
>>>>> at org.apache.metron.common.stellar.generated.StellarParser.ari
>>>>> thmetic_expr(StellarParser.java:1397) ~[stormjar.jar:?]
>>>>> at org.apache.metron.common.stellar.generated.StellarParser.tra
>>>>> nsformation_expr(StellarParser.java:278) ~[stormjar.jar:?]
>>>>> at org.apache.metron.common.stellar.generated.StellarParser.tra
>>>>> nsformation(StellarParser.java:147) ~[stormjar.jar:?]
>>>>> at org.apache.metron.common.stellar.BaseStellarProcessor.parse(BaseStellarProcessor.java:89)
>>>>> ~[stormjar.jar:?]
>>>>> ... 19 more
>>>>>
>>>>> On Tue, Feb 28, 2017 at 12:18 AM, Casey Stella <cestella@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Ok, try something for me and tell me if it fixes it: adjust the
>>>>>> config to remove the tickUpdate (it didn't work anyway and we have
since
>>>>>> removed it in 0.3.1).
>>>>>>
>>>>>>
>>>>>> On Mon, Feb 27, 2017 at 6:07 PM, shoggi <shoggi@gmail.com>
wrote:
>>>>>>
>>>>>>> Yes, this is 0.3.0.
>>>>>>>
>>>>>>> Here is the full trace.
>>>>>>>
>>>>>>> 2017-02-28 00:05:43.367 o.a.m.p.b.ProfileBuilderBolt [INFO] Flushing
>>>>>>> profile: profile=sketchy_mad, entity=krxd.net
>>>>>>> 2017-02-28 00:05:43.374 o.a.m.p.b.ProfileBuilderBolt [ERROR]
>>>>>>> Unexpected failure: message='Unable to execute:
>>>>>>> org.apache.metron.statistics.outlier.MedianAbsoluteDeviationFunctions$State
>>>>>>> cannot be cast to java.util.List', tuple='source: __system:-1,
stream:
>>>>>>> __tick, id: {}, [900]'
>>>>>>> org.apache.metron.common.dsl.ParseException: Unable to execute:
>>>>>>> org.apache.metron.statistics.outlier.MedianAbsoluteDeviationFunctions$State
>>>>>>> cannot be cast to java.util.List
>>>>>>> at org.apache.metron.common.stellar.StellarCompiler.getResult(StellarCompiler.java:428)
>>>>>>> ~[stormjar.jar:?]
>>>>>>> at org.apache.metron.common.stellar.BaseStellarProcessor.parse(BaseStellarProcessor.java:90)
>>>>>>> ~[stormjar.jar:?]
>>>>>>> at org.apache.metron.profiler.stellar.DefaultStellarExecutor.ex
>>>>>>> ecute(DefaultStellarExecutor.java:148) ~[stormjar.jar:?]
>>>>>>> at org.apache.metron.profiler.stellar.DefaultStellarExecutor.as
>>>>>>> sign(DefaultStellarExecutor.java:94) ~[stormjar.jar:?]
>>>>>>> at org.apache.metron.profiler.bolt.ProfileBuilderBolt.lambda$nu
>>>>>>> ll$2(ProfileBuilderBolt.java:258) ~[stormjar.jar:?]
>>>>>>> at java.util.LinkedHashMap.forEach(LinkedHashMap.java:676)
>>>>>>> ~[?:1.8.0_77]
>>>>>>> at org.apache.metron.profiler.bolt.ProfileBuilderBolt.lambda$fl
>>>>>>> ush$4(ProfileBuilderBolt.java:258) ~[stormjar.jar:?]
>>>>>>> at java.util.concurrent.ConcurrentMap.forEach(ConcurrentMap.java:114)
>>>>>>> ~[?:1.8.0_77]
>>>>>>> at org.apache.metron.profiler.bolt.ProfileBuilderBolt.flush(ProfileBuilderBolt.java:237)
>>>>>>> ~[stormjar.jar:?]
>>>>>>> at org.apache.metron.profiler.bolt.ProfileBuilderBolt.doExecute(ProfileBuilderBolt.java:164)
>>>>>>> ~[stormjar.jar:?]
>>>>>>> at org.apache.metron.profiler.bolt.ProfileBuilderBolt.execute(ProfileBuilderBolt.java:144)
>>>>>>> [stormjar.jar:?]
>>>>>>> at org.apache.storm.daemon.executor$fn__6571$tuple_action_fn__6573.invoke(executor.clj:734)
>>>>>>> [storm-core-1.0.1.2.5.0.0-1245.jar:1.0.1.2.5.0.0-1245]
>>>>>>> at org.apache.storm.daemon.executor$mk_task_receiver$fn__6492.invoke(executor.clj:469)
>>>>>>> [storm-core-1.0.1.2.5.0.0-1245.jar:1.0.1.2.5.0.0-1245]
>>>>>>> at org.apache.storm.disruptor$clojure_handler$reify__6005.onEvent(disruptor.clj:40)
>>>>>>> [storm-core-1.0.1.2.5.0.0-1245.jar:1.0.1.2.5.0.0-1245]
>>>>>>> at org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:451)
>>>>>>> [storm-core-1.0.1.2.5.0.0-1245.jar:1.0.1.2.5.0.0-1245]
>>>>>>> at org.apache.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:430)
>>>>>>> [storm-core-1.0.1.2.5.0.0-1245.jar:1.0.1.2.5.0.0-1245]
>>>>>>> at org.apache.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:73)
>>>>>>> [storm-core-1.0.1.2.5.0.0-1245.jar:1.0.1.2.5.0.0-1245]
>>>>>>> at org.apache.storm.daemon.executor$fn__6571$fn__6584$fn__6637.invoke(executor.clj:853)
>>>>>>> [storm-core-1.0.1.2.5.0.0-1245.jar:1.0.1.2.5.0.0-1245]
>>>>>>> at org.apache.storm.util$async_loop$fn__554.invoke(util.clj:484)
>>>>>>> [storm-core-1.0.1.2.5.0.0-1245.jar:1.0.1.2.5.0.0-1245]
>>>>>>> at clojure.lang.AFn.run(AFn.java:22) [clojure-1.7.0.jar:?]
>>>>>>> at java.lang.Thread.run(Thread.java:745) [?:1.8.0_77]
>>>>>>> Caused by: java.lang.ClassCastException:
>>>>>>> org.apache.metron.statistics.outlier.MedianAbsoluteDeviationFunctions$State
>>>>>>> cannot be cast to java.util.List
>>>>>>> at org.apache.metron.statistics.outlier.MedianAbsoluteDeviation
>>>>>>> Functions$StateUpdate.apply(MedianAbsoluteDeviationFunctions.java:85)
>>>>>>> ~[stormjar.jar:?]
>>>>>>> at org.apache.metron.common.stellar.StellarCompiler.exitTransfo
>>>>>>> rmationFunc(StellarCompiler.java:246) ~[stormjar.jar:?]
>>>>>>> at org.apache.metron.common.stellar.generated.StellarParser$Tra
>>>>>>> nsformationFuncContext.exitRule(StellarParser.java:1617)
>>>>>>> ~[stormjar.jar:?]
>>>>>>> at org.antlr.v4.runtime.Parser.triggerExitRuleEvent(Parser.java:422)
>>>>>>> ~[stormjar.jar:?]
>>>>>>> at org.antlr.v4.runtime.Parser.exitRule(Parser.java:632)
>>>>>>> ~[stormjar.jar:?]
>>>>>>> at org.apache.metron.common.stellar.generated.StellarParser.fun
>>>>>>> ctions(StellarParser.java:1640) ~[stormjar.jar:?]
>>>>>>> at org.apache.metron.common.stellar.generated.StellarParser.ari
>>>>>>> thmetic_operands(StellarParser.java:1750) ~[stormjar.jar:?]
>>>>>>> at org.apache.metron.common.stellar.generated.StellarParser.ari
>>>>>>> thmetic_expr_mul(StellarParser.java:1537) ~[stormjar.jar:?]
>>>>>>> at org.apache.metron.common.stellar.generated.StellarParser.ari
>>>>>>> thmetic_expr(StellarParser.java:1397) ~[stormjar.jar:?]
>>>>>>> at org.apache.metron.common.stellar.generated.StellarParser.tra
>>>>>>> nsformation_expr(StellarParser.java:278) ~[stormjar.jar:?]
>>>>>>> at org.apache.metron.common.stellar.generated.StellarParser.tra
>>>>>>> nsformation(StellarParser.java:147) ~[stormjar.jar:?]
>>>>>>> at org.apache.metron.common.stellar.BaseStellarProcessor.parse(BaseStellarProcessor.java:89)
>>>>>>> ~[stormjar.jar:?]
>>>>>>> ... 19 more
>>>>>>> 2017-02-28 00:05:43.377 o.a.s.d.executor [ERROR]
>>>>>>> org.apache.metron.common.dsl.ParseException: Unable to execute:
>>>>>>> org.apache.metron.statistics.outlier.MedianAbsoluteDeviationFunctions$State
>>>>>>> cannot be cast to java.util.List
>>>>>>> at org.apache.metron.common.stellar.StellarCompiler.getResult(StellarCompiler.java:428)
>>>>>>> ~[stormjar.jar:?]
>>>>>>> at org.apache.metron.common.stellar.BaseStellarProcessor.parse(BaseStellarProcessor.java:90)
>>>>>>> ~[stormjar.jar:?]
>>>>>>> at org.apache.metron.profiler.stellar.DefaultStellarExecutor.ex
>>>>>>> ecute(DefaultStellarExecutor.java:148) ~[stormjar.jar:?]
>>>>>>> at org.apache.metron.profiler.stellar.DefaultStellarExecutor.as
>>>>>>> sign(DefaultStellarExecutor.java:94) ~[stormjar.jar:?]
>>>>>>> at org.apache.metron.profiler.bolt.ProfileBuilderBolt.lambda$nu
>>>>>>> ll$2(ProfileBuilderBolt.java:258) ~[stormjar.jar:?]
>>>>>>> at java.util.LinkedHashMap.forEach(LinkedHashMap.java:676)
>>>>>>> ~[?:1.8.0_77]
>>>>>>> at org.apache.metron.profiler.bolt.ProfileBuilderBolt.lambda$fl
>>>>>>> ush$4(ProfileBuilderBolt.java:258) ~[stormjar.jar:?]
>>>>>>> at java.util.concurrent.ConcurrentMap.forEach(ConcurrentMap.java:114)
>>>>>>> ~[?:1.8.0_77]
>>>>>>> at org.apache.metron.profiler.bolt.ProfileBuilderBolt.flush(ProfileBuilderBolt.java:237)
>>>>>>> ~[stormjar.jar:?]
>>>>>>> at org.apache.metron.profiler.bolt.ProfileBuilderBolt.doExecute(ProfileBuilderBolt.java:164)
>>>>>>> ~[stormjar.jar:?]
>>>>>>> at org.apache.metron.profiler.bolt.ProfileBuilderBolt.execute(ProfileBuilderBolt.java:144)
>>>>>>> [stormjar.jar:?]
>>>>>>> at org.apache.storm.daemon.executor$fn__6571$tuple_action_fn__6573.invoke(executor.clj:734)
>>>>>>> [storm-core-1.0.1.2.5.0.0-1245.jar:1.0.1.2.5.0.0-1245]
>>>>>>> at org.apache.storm.daemon.executor$mk_task_receiver$fn__6492.invoke(executor.clj:469)
>>>>>>> [storm-core-1.0.1.2.5.0.0-1245.jar:1.0.1.2.5.0.0-1245]
>>>>>>> at org.apache.storm.disruptor$clojure_handler$reify__6005.onEvent(disruptor.clj:40)
>>>>>>> [storm-core-1.0.1.2.5.0.0-1245.jar:1.0.1.2.5.0.0-1245]
>>>>>>> at org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:451)
>>>>>>> [storm-core-1.0.1.2.5.0.0-1245.jar:1.0.1.2.5.0.0-1245]
>>>>>>> at org.apache.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:430)
>>>>>>> [storm-core-1.0.1.2.5.0.0-1245.jar:1.0.1.2.5.0.0-1245]
>>>>>>> at org.apache.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:73)
>>>>>>> [storm-core-1.0.1.2.5.0.0-1245.jar:1.0.1.2.5.0.0-1245]
>>>>>>> at org.apache.storm.daemon.executor$fn__6571$fn__6584$fn__6637.invoke(executor.clj:853)
>>>>>>> [storm-core-1.0.1.2.5.0.0-1245.jar:1.0.1.2.5.0.0-1245]
>>>>>>> at org.apache.storm.util$async_loop$fn__554.invoke(util.clj:484)
>>>>>>> [storm-core-1.0.1.2.5.0.0-1245.jar:1.0.1.2.5.0.0-1245]
>>>>>>> at clojure.lang.AFn.run(AFn.java:22) [clojure-1.7.0.jar:?]
>>>>>>> at java.lang.Thread.run(Thread.java:745) [?:1.8.0_77]
>>>>>>> Caused by: java.lang.ClassCastException:
>>>>>>> org.apache.metron.statistics.outlier.MedianAbsoluteDeviationFunctions$State
>>>>>>> cannot be cast to java.util.List
>>>>>>> at org.apache.metron.statistics.outlier.MedianAbsoluteDeviation
>>>>>>> Functions$StateUpdate.apply(MedianAbsoluteDeviationFunctions.java:85)
>>>>>>> ~[stormjar.jar:?]
>>>>>>> at org.apache.metron.common.stellar.StellarCompiler.exitTransfo
>>>>>>> rmationFunc(StellarCompiler.java:246) ~[stormjar.jar:?]
>>>>>>> at org.apache.metron.common.stellar.generated.StellarParser$Tra
>>>>>>> nsformationFuncContext.exitRule(StellarParser.java:1617)
>>>>>>> ~[stormjar.jar:?]
>>>>>>> at org.antlr.v4.runtime.Parser.triggerExitRuleEvent(Parser.java:422)
>>>>>>> ~[stormjar.jar:?]
>>>>>>> at org.antlr.v4.runtime.Parser.exitRule(Parser.java:632)
>>>>>>> ~[stormjar.jar:?]
>>>>>>> at org.apache.metron.common.stellar.generated.StellarParser.fun
>>>>>>> ctions(StellarParser.java:1640) ~[stormjar.jar:?]
>>>>>>> at org.apache.metron.common.stellar.generated.StellarParser.ari
>>>>>>> thmetic_operands(StellarParser.java:1750) ~[stormjar.jar:?]
>>>>>>> at org.apache.metron.common.stellar.generated.StellarParser.ari
>>>>>>> thmetic_expr_mul(StellarParser.java:1537) ~[stormjar.jar:?]
>>>>>>> at org.apache.metron.common.stellar.generated.StellarParser.ari
>>>>>>> thmetic_expr(StellarParser.java:1397) ~[stormjar.jar:?]
>>>>>>> at org.apache.metron.common.stellar.generated.StellarParser.tra
>>>>>>> nsformation_expr(StellarParser.java:278) ~[stormjar.jar:?]
>>>>>>> at org.apache.metron.common.stellar.generated.StellarParser.tra
>>>>>>> nsformation(StellarParser.java:147) ~[stormjar.jar:?]
>>>>>>> at org.apache.metron.common.stellar.BaseStellarProcessor.parse(BaseStellarProcessor.java:89)
>>>>>>> ~[stormjar.jar:?]
>>>>>>> ... 19 more
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Mon, Feb 27, 2017 at 11:33 PM, Casey Stella <cestella@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Any chance there's a full stacktrace associated with that
NPE
>>>>>>>> anywhere?  Also, I assume this is 0.3.0, right?
>>>>>>>>
>>>>>>>> On Mon, Feb 27, 2017 at 5:29 PM, shoggi <shoggi@gmail.com>
wrote:
>>>>>>>>
>>>>>>>>> Hi all
>>>>>>>>>
>>>>>>>>> Has anyone come across following profiler error, when
trying to
>>>>>>>>> work with the MAD functions? The configuration loads
fine and I only get
>>>>>>>>> the error when the profiler is pushing its data. I don't
think its a typo
>>>>>>>>> but maybe the logic is not sound.
>>>>>>>>>
>>>>>>>>> Error from the profiler worker:
>>>>>>>>> 2017-02-27 23:20:43.368 o.a.m.p.b.ProfileBuilderBolt
[INFO]
>>>>>>>>> Flushing profile: profile=sketchy_mad, entity=krxd.net
>>>>>>>>> 2017-02-27 23:20:43.368 o.a.m.p.b.ProfileBuilderBolt
[ERROR]
>>>>>>>>> Unexpected failure: message='null', tuple='source: __system:-1,
stream:
>>>>>>>>> __tick, id: {}, [900]'
>>>>>>>>> java.lang.NullPointerException
>>>>>>>>> 2017-02-27 23:20:43.368 o.a.s.d.executor [ERROR]
>>>>>>>>> java.lang.NullPointerException
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> My Profiler config:
>>>>>>>>>
>>>>>>>>>     {
>>>>>>>>>       "profile": "sketchy_mad",
>>>>>>>>>       "foreach": "if exists(domain_without_subdomains)
then
>>>>>>>>> domain_without_subdomains else 'n/a'",
>>>>>>>>>       "onlyif": "exists(domain_without_subdomains) &&
source.type
>>>>>>>>> == 'squid'",
>>>>>>>>>       "init" : {
>>>>>>>>>         "s": "OUTLIER_MAD_STATE_MERGE(PROFILE_GET('sketchy_mad',
>>>>>>>>> domain_without_subdomains, 5, 'MINUTES'))"
>>>>>>>>>       },
>>>>>>>>>       "tickUpdate" : {
>>>>>>>>>         "s": "OUTLIER_MAD_STATE_MERGE(PROFILE_GET('sketchy_mad',
>>>>>>>>> domain_without_subdomains, 5, 'MINUTES'), s)"
>>>>>>>>>       },
>>>>>>>>>       "update": {
>>>>>>>>>         "s": "OUTLIER_MAD_ADD(s, bytes)"
>>>>>>>>>       },
>>>>>>>>>       "result": "s"
>>>>>>>>>     }
>>>>>>>>>
>>>>>>>>> Regards
>>>>>>>>> shoggi
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>

Mime
View raw message