commons-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Dunning <ted.dunn...@gmail.com>
Subject Re: [math] Re: running average of a rate
Date Mon, 14 Mar 2011 20:49:26 GMT
Phil,

A big part of the problem as I understand it is that these rates are needed
across a potentially quite large distributed system.

On Mon, Mar 14, 2011 at 1:42 PM, Phil Steitz <phil.steitz@gmail.com> wrote:

> On 3/14/11 12:34 PM, Luc Maisonobe wrote:
> > Le 14/03/2011 15:33, Benson Margulies a écrit :
> >> Please excuse the following ignorant question.
> >>
> >> I want to maintain summary statistics of a rate. At each 'event', I
> >> know the number of characters and the time it took to process them,
> >> and I want to maintain summary statistics for the rate of
> >> chars/second. I imagine that I'm missing something basic, but I don't
> >> see how to do this.
> > You should define some windows width, either in terms of a time span
> > (all events in the last n seconds) or in terms of number of events (last
> > n events).
> >
> > In [math], we do not provide (yet) anything for maintaining such a data
> > structure, you'll have to maintain the events in this slot by yourself,
> > with something similar to a FIFO.
>
> I am not sure I understand what the problem is exactly, but if what
> you need is simply "rolling" statistics, where a dataset of 0,...,n
> values are maintained with the newest values replacing the oldest,
> we do in fact support that in
> o.a.c.math.descriptive.DescriptiveStatistics.
>
> Phil
> > When you have your data available, each time a new event is added or
> > removed from the ones that belong to the window, you can fetch compute
> > the statistics you want on this data (min, max, mean, median, standard
> > deviation ...) and wait for next addition/removel to recompute it again.
> >
> > Another thing we discussed some months ago (but did not implement yet)
> > is a way to compute an approximation of percentiles in a flow of data
> > without storing them. There is an interesting algorithm for it that was
> > developed for the needs of telecommunication companies, I think it may
> > be of interest to you. This would provide results like : currently 95%
> > of the characters are processed in n milliseconds. would you be
> > interested in us implementing this feature ?
> >
> > Luc
> >
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> >> For additional commands, e-mail: user-help@commons.apache.org
> >>
> >>
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> > For additional commands, e-mail: user-help@commons.apache.org
> >
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> For additional commands, e-mail: user-help@commons.apache.org
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message