metron-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ali Nazemian <alinazem...@gmail.com>
Subject Re: Define a function that can be used in Stellar
Date Fri, 02 Feb 2018 13:42:52 GMT
Does it mean every time the function gets called it will load the config,
but if I use the global one it will only read it one time and it will be
available in memory?

On 2 Feb. 2018 21:53, "Simon Elliston Ball" <simon@simonellistonball.com>
wrote:

> Shouldn’t be. The one this I would point out though is that you don’t
> necessarily know which supervisor you will be running from, so pulling from
> HDFS would make sense. That said, the performance implications are probably
> not great. A good option here would be to have the config available in the
> global config for example and refer to that, since most instances of
> stellar apply global config to their context.
>
> Simon
>
>
> On 2 Feb 2018, at 07:14, Ali Nazemian <alinazemian@gmail.com> wrote:
>
> Will be any problem if the Stellar function we want to implement need to
> load an external config file?
>
> Cheers,
> Ali
>
> On Thu, Jan 18, 2018 at 4:58 PM, Ali Nazemian <alinazemian@gmail.com>
> wrote:
>
>> Thanks, All.
>>
>> Yes, Nick. It is highly related to our use case and the way that we are
>> going to enrich events with assets and vulnerability properties. It is not
>> a general case at all.
>>
>> Cheers,
>> Ali
>>
>> On Thu, Jan 18, 2018 at 5:43 AM, Matt Foley <mattf@apache.org> wrote:
>>
>>> Besides the example code Simon mentioned at
>>> https://github.com/apache/metron/tree/master/metron-stellar/
>>> stellar-3rd-party-example ,
>>> there is some documentation at http://metron.apache.org/curre
>>> nt-book/metron-stellar/stellar-common/3rdPartyStellar.html
>>>
>>>
>>>
>>> *From: *Nick Allen <nick@nickallen.org>
>>> *Reply-To: *"user@metron.apache.org" <user@metron.apache.org>
>>> *Date: *Wednesday, January 17, 2018 at 4:46 AM
>>> *To: *"user@metron.apache.org" <user@metron.apache.org>
>>> *Subject: *Re: Define a function that can be used in Stellar
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> If something we have already does not fit the bill, I would recommend
>>> creating that function in Java.   Since you described it as "a bit complex"
>>> and "the logic would be complicated" I don't see any value in defining
>>> something like this in Stellar with named functions.
>>>
>>>
>>>
>>> Best
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> On Wed, Jan 17, 2018 at 7:38 AM Simon Elliston Ball <
>>> simon@simonellistonball.com> wrote:
>>>
>>> Have you looked at the recent TLSH functions in Stellar? We already have
>>> that for similarity preserving hashes.
>>>
>>>
>>>
>>> Simon
>>>
>>>
>>>
>>>
>>> On 17 Jan 2018, at 12:35, Ali Nazemian <alinazemian@gmail.com> wrote:
>>>
>>> It is a bit complex. We want to create a function that accepts a list of
>>> arguments for an asset and generate an asset identifier that can be used as
>>> a row_key for the enrichment store. The logic would be complicated, though.
>>> We may need to include some sort of similarity aware hash function as a
>>> part of this custom function.
>>>
>>>
>>>
>>> On Wed, Jan 17, 2018 at 10:32 PM, Nick Allen <nick@nickallen.org> wrote:
>>>
>>> Ali - Can you describe the logic that you are trying to perform? That
>>> would be useful as a use case to help drive a discussion around creating
>>> named functions in Stellar.
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> On Wed, Jan 17, 2018 at 6:29 AM Ali Nazemian <alinazemian@gmail.com>
>>> wrote:
>>>
>>> Thanks, Simon. We have already got a script to deal with classpath
>>> management for the parsers. We should be able to use it for this extension
>>> as well.
>>>
>>>
>>>
>>> Yeah, I agree. It will be much easier to define functions on the fly and
>>> use them afterwards. It could be defined as Lambda or custom function.
>>>
>>>
>>>
>>> Regards,
>>>
>>> Ali
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> On Wed, Jan 17, 2018 at 9:42 PM, Simon Elliston Ball <
>>> simon@simonellistonball.com> wrote:
>>>
>>> https://github.com/apache/metron/tree/master/metron-stellar/
>>> stellar-3rd-party-example gives good details on how to add a stellar
>>> function.
>>>
>>>
>>>
>>> Stellar will pick up an annotated function on its class path, so to add
>>> function there is no need to rebuild metron module, but you do need your
>>> modules on the classpath, and, pending 777, to deal with things like class
>>> path clash in your dependencies.
>>>
>>>
>>>
>>> Another idea worth discussion on the dev list is probably the notion of
>>> defining stellar functions in stellar, which would be a much simpler
>>> solution than custom java functions if you can already express you logic in
>>> stellar.
>>>
>>>
>>>
>>> Simon
>>>
>>>
>>>
>>>
>>>
>>> On 17 Jan 2018, at 10:37, Ali Nazemian <alinazemian@gmail.com> wrote:
>>>
>>>
>>>
>>> Hi Simon,
>>>
>>>
>>>
>>> Yes, that is exactly what we are looking for. Is there any example
>>> regarding adding a Stellar function in Java? Hopefully, we don't need to
>>> rebuild the corresponding modules for this?
>>>
>>>
>>>
>>> Cheers,
>>>
>>> Ali
>>>
>>>
>>>
>>> On Wed, Jan 17, 2018 at 8:40 PM, Simon Elliston Ball <
>>> simon@simonellistonball.com> wrote:
>>>
>>> At present you can certainly create custom stellar functions in Java.
>>> I’m guessing however that what you’re looking to do is create a kind of
>>> function that combines a number of stellar functions to avoid repetition,
>>> or to ensure consistency of certain parameters for example. Is that what
>>> you’re looking for? Maybe some sort of syntax to create a named stellar
>>> function similar to the way we create lambdas?
>>>
>>> Simon
>>>
>>>
>>> > On 17 Jan 2018, at 07:25, Ali Nazemian <alinazemian@gmail.com> wrote:
>>> >
>>> > Hi all,
>>> >
>>> > Is there any way that we can define a function that can be used rather
>>> than duplicating a logic multiple times?
>>> >
>>> > Cheers,
>>> > Ali
>>>
>>>
>>>
>>>
>>>
>>> --
>>>
>>> A.Nazemian
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> --
>>>
>>> A.Nazemian
>>>
>>>
>>>
>>>
>>>
>>> --
>>>
>>> A.Nazemian
>>>
>>>
>>
>>
>> --
>> A.Nazemian
>>
>
>
>
> --
> A.Nazemian
>
>
>

Mime
View raw message