commons-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mark R. Diggory" <mdigg...@latte.harvard.edu>
Subject Re: [math][patch] New RollingUnivariateImpl
Date Fri, 16 May 2003 18:18:37 GMT
O'brien, Tim wrote:

>On Fri, 2003-05-16 at 11:04, Mark R. Diggory wrote:
>  
>
>>My implementation requires no such thing! ;-) 
>>    
>>
>
>My apologies.
>  
>
>>Look at it again, it "rolls" around the array simply replacing the last value in the
window 
>>with the new value in the window and increments the index representing 
>>the beginning of the array (when this index reaches the end of the array 
>>it gets mod'ed to send it back to the beginning of the array. No 
>>elements "move". The array doesn't change size, this is an extremely 
>>efficient design.
>>    
>>
>
>It is.
>

Thank you... :-)

>>Don't get me wrong, but doesn't your implementation actually produce an 
>>array that has an unused portion at the beginning?
>>    
>>
>
>See ContractableDoubleArray, storage is reclaimed when the internal
>storage exceed the number of elements by the factor defined in
>contractionCriteria.
>  
>

Ahhh, I missed that you were using Contractable, I thought you were 
using Expandable directly, I clearly see now that Contractable extends 
Expandable and provides those features, thats nice. I had missed that 
before and was wondering why expandable and contractable were separate.

>  
>
>>I don't see where storage is reclaimed at the beginning of the array 
>>because you always copy the array starting at 0. 
>>    
>>
>
>Again, check out ContractableDoubleArray - the contract function takes
>care of reclaiming unused space, only if the criteria are met.  If your
>contractionCriteria is 2.0, and you have 100 elements, you are
>gauranteed that the internal storage array will be between 101 and 200
>in length.  Then benefit here is that, memory is reclaimed only once
>every one hundred "rolls".
>  
>
this does clarify my misunderstanding,

>>I think it really may be wiser to separate the functionalities of 
>>Rolling and Expanding, At least until theres a clear methodology for the 
>>best implementation of each and a clear means of combining them.
>>    
>>
>
>Expandable and Contractable double arrays as a storage mechanism for
>Univariates provide a benefit in that they preserve the order of the
>values added to Univariate.
>  
>
What are the benifits of retaining this ordering in the array structure 
itself? Are there really any stats that would benifit from this 
ordering? Its also my understanding is that you would probibly need to 
apply sorting to a copy of the array to get the mode, median and quantiles.

>It might be useful to have a FixedDoubleArray which supports the
>rolling functionality described in your patch, but, in general, these
>objects provide usefulness outside of Univariate.  Essentially, I don't
>want an implementation of Univariate to have to worry about storage.
>

Hmm, I'm not sure of its value as a separate FixedDoubleArray, I'd 
rather explore it as a means of optimizing your implementations of 
DoubleArray further.

 From what I see, you've successfully implemented the Rolling 
functionality now and theres really no reason to use my separate 
RollingUnivariate Implementation because of this. As well, I see you've 
applied some of the recommendations for moving the window into to the 
UnivariateImpl as well.

I have another idea rolling around in my head, it involves the solution 
of the higher order moments like kurtosis and skew in such a way that 
they can be applied without any storage requirements. This is difficult 
in that I havn't seen any applications of this idea on the net or in 
research that I know of so far. Basically, I think it just involves 
calculating the sumcube and sumquad and applying them in the same 
strategy as a sum and sumsq. But, I havn't yet found the correct 
algorithm to capture this capability ( there are some details I may be 
missing). But if I did accomplish this feat, it would possibly alow you 
to pull the kurtosis and skew methods into the Univariate Interface. 
What do you think?

-Mark

p.s. Just a small criticism, while your free as an Apache committer to 
take what ever you want out of my patches to apply to your codebase. I 
hope I might retain some acknowledgement over the inclusion of ideas I 
may have produced or  helped to produce. Not having commit rights, I 
have no way of retaining authorship over ideas or prototyping efforts 
such as that included in my patches into Univariate/RollingUnivariate 
etc. I hope you might retain at least some comment or honorable mention 
for my efforts. :-)  thanks



---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org


Mime
View raw message