metron-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Simon Elliston Ball <si...@simonellistonball.com>
Subject Re: Apache Metron functions implementation
Date Fri, 02 Feb 2018 12:25:29 GMT
Hi Helder, 

It is very much possible, and very easy to create your own functions and models on top of
Metron. 

There are two main ways in which you would do this, depending on the type of use case you’re
looking at. 

Metron uses a language called Stellar as part of the enrichment stage (and elsewhere) to implement
a number of algorithms which can then be composed in configuration. You can also extend this
language to implement your own algorithms in the real time stream (https://github.com/apache/metron/tree/master/metron-stellar/stellar-3rd-party-example
<https://github.com/apache/metron/tree/master/metron-stellar/stellar-3rd-party-example>
gives a toy example, also checkout some of the source for the more interesting stellar functions
in https://github.com/apache/metron/tree/master/metron-analytics/metron-statistics <https://github.com/apache/metron/tree/master/metron-analytics/metron-statistics>.


If your algorithms tend more towards the traditional ML approach, using for example Spark,
python, or R, then the Model as a Service extension points might be more useful. This allows
you to run arbitrary micro-service type model inference, or scoring, and plug that into he
Metron real-time stream (https://github.com/apache/metron/tree/master/metron-analytics/metron-maas-service
<https://github.com/apache/metron/tree/master/metron-analytics/metron-maas-service>
provides more information and a worked example of how you would plug in an example python
based model).  

I would also suggest taking a look at some of the recent custom use-cases we have included
in the project to get some starters: https://github.com/apache/metron/tree/master/use-cases
<https://github.com/apache/metron/tree/master/use-cases>.

I hope that helps, and wish you the best of luck with your project. Also, do let the community
know what you’re working on, and I’m sure we will be more than happy to provide any help
and assistance we can. Looking forward to seeing what you come up with, and welcome to Metron.


Simon

> On 2 Feb 2018, at 12:11, Helder Reia <helder.reia@gmail.com> wrote:
> 
> Hello,
> I am a student currently finishing my master degree and for my final work I am proposing
to make a security analytics tool. I will want to make it on Apache Metron framework but I
have some questions:
> - Is it possible to implement my own functions ? ( I will want to have clustering and
classification algorithms )
> - If so, can you give me helpon how to implement those algorithms?
> 
> Thank you for the help !
> 
> -- 
> Helder Reia
> ALF-AL TM
> 
> 


Mime
View raw message