ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Алексей Платонов <aplaton...@gmail.com>
Subject Re: [ML] Distributed metrics computation
Date Tue, 10 Sep 2019 11:09:07 GMT
I mean metrics for model evaluation like Accuracy or Precision/Recall for
ML models. It isn't same as system metrics (like throughput). Such metrics
should be computed over a test set after model training. if it is
interesting for you, please, have a look at this material:
https://en.wikipedia.org/wiki/Precision_and_recall . It's just homonymy
between machine learning metrics and system metrics. We can't compute
ML-metrics via Zabbix for example.

Best regards,
Alexey Platonov

вт, 10 сент. 2019 г. в 13:52, Nikolay Izhikov <nizhikov@apache.org>:

> Hello, Alexey.
>
> Why do we need distributed metrics in the first place?
> It seems, there are many metric processing system out there: Prometheus,
> Zabbix, Splunk, etc.
>
> Each of then can aggregate metrics in many ways.
>
> I think, we should not use Ignite as an metrics aggregation system.
>
> What do you think?
>
> В Вт, 10/09/2019 в 13:08 +0300, Алексей Платонов пишет:
> > Hi Igniters!
> > I've been working on a prototype of distributed metrics computation for
> > ML-models. Unfortunately, we don't have an ability to compute metrics in
> a
> > distributed manner, so, it leads to gathering metric statistics to client
> > node via ScanQuery and all flow of vectors from partitions will be sent
> to
> > a client. I want to avoid such behavior and I propose the framework for
> > metrics computation using MapReduce approach based on an aggregation of
> > statistics for metrics.
> >
> > I prepared an issue in Apache Jira for this:
> > https://issues.apache.org/jira/browse/IGNITE-12155
> > Also, I prepared PR for it: https://github.com/apache/ignite/pull/6857
> > Currently, the work on this framework is still running but I'm going to
> > prepare full PR during this week.
> >
> > By this email, I want to start a discussion about this idea.
> >
> > Best regards,
> > Alexey Platonov
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message