commons-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Romeo Palijan (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MATH-296) LoessInterpolator.smooth() not working correctly
Date Thu, 24 Sep 2009 18:23:16 GMT

    [ https://issues.apache.org/jira/browse/MATH-296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12759203#action_12759203
] 

Romeo Palijan commented on MATH-296:
------------------------------------

Eugene, that is great news. I already tried to fix it but did not get very far. 
First of all, please ignore the sample values that are supposed to be from loessFit in the
example above, they are not. I have reviewed my code and I can not reproduce those values.
Sorry for that. When I try the examples again I get hardly any changes except for the NaN
values (perhaps due to the fact that the values are not very close together?). The testcases
should be deleted/changed to reflect this  I have compiled a more appropriate testcase:

|xval|0.1|0.2|0.3|0.4|0.5|0.6|0.7|0.8|0.9|1.0|1.1|1.2|1.3|1.4|1.5|1.6|1.7|1.8|1.9|2.0|
|yval|0.47|0.48|0.55|0.56|-0.08|-0.04|-0.07|-0.07|-0.56|-0.46|-0.56|-0.52|-3.03|-3.08|-3.09|-3.04|3.54|3.46|3.36|3.35|
|result|0.46076343001050907|0.49997135715509317|0.5429801876574236|0.3191000440696382|0.17824135321363765|-0.03999999999999973|-0.0699999999999999|-0.21301261447145203|-0.3260295715150817|-0.45999999999999996|-0.560000000000002|-1.4961752602921834|-2.1317826106042093|-3.0799999999999965|-3.089999999999987|-0.6229499113916326|0.9970283360414918|3.450011252797152|3.3907474604481567|3.336774229999947|
|loessFit|0.4609008580907899|0.4985267031275769|0.5408291873645029|0.308077939113221|0.1754074466314782|-0.0419929152622151|-0.07205683451999362|-0.19661654144034335|-0.31073415047540565|-0.44615854234427305|-0.5567051673879394|-1.4972970376332615|-2.1330233520442317|-3.08|-3.0900000000000043|-0.6208045742447532|0.9823842010251099|3.449395357814414|3.389793487936696|3.3359749840089385|
|weight(see below)|1.0|1.0|1.0|1.0|1.0|1.0|1.0|1.0|1.0|1.0|1.0|1.0|0.0|0.0|1.0|1.0|0.0|0.0|1.0|1.0|
|loessFit with weights|0.4780117093745368|0.4924876444111407|0.48398021996388496|0.3201446090891406|0.17880078999161195|-0.003305611690841502|-0.08829472923805592|-0.20902549447499932|-0.3267558674057103|-0.45542226638134214|-0.518369970805548|-0.5365908808969353|-1.492628917733731|-2.1151751796193703|-3.09|-3.0400000000000005|-2.9899999999999998|0.15500000000000005|1.7524999999999986|3.3499999999999996|


The results do not differ very much and there are no NaN, if the values are close together.
Can this be explained by a slightly different algorithm in R?

For easy copy and paste:
double[] yval={0.47, 0.48, 0.55, 0.56,-0.08,-0.04,-0.07,-0.07,-0.56,-0.46,-0.56,-0.52,-3.03,-3.08,-3.09,-3.04,
3.54, 3.46, 3.36, 3.35};
double[] xval={0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 1.1, 1.2, 1.3, 1.4, 1.5,
1.6, 1.7, 1.8, 1.9, 2.0};
double[] weights = {1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 1, 1, 0, 0, 1, 1};
LoessInterpolator li = new LoessInterpolator(0.3,4);
result = li.smooth(xval, yval);


Another related thing. A very common use of the R-function is with weights (that is how we
use it as well): 
*loessFit(yValues, xValues, weights)*
where weights are the values 0 or 1, I dont know if some are using weights between 0 and 1

If a spot has weight 0 it still gets loessfitted but does not affect his neighbours. This
way you can specifiy more relevant points and have the complete data set interpolate around
these.

In your implementation you use robustnessWeights and set them initially to 1. Do I understand
this correctly that these are actually the same weights as the ones mentioned above?
Would it be possible to add a function with a third parameter and have the weights provided
by the user? I tried just passing them trough as a parameter instead of initializing with
1 but it did not do the trick.

> LoessInterpolator.smooth() not working correctly
> ------------------------------------------------
>
>                 Key: MATH-296
>                 URL: https://issues.apache.org/jira/browse/MATH-296
>             Project: Commons Math
>          Issue Type: Bug
>    Affects Versions: 2.0
>         Environment: Java 1.6 on Vista
>            Reporter: Romeo Palijan
>         Attachments: math-296-test.patch
>
>
> I have been comparing LoessInterpolator.smooth output with the loessFit output from R
(R-project.org, probably the most widely used loess implementation) and have had strangely
different numbers. I have created a small set to test the difference and something seems to
be wrong with the smooth method but I do no know what and I do not understand the code.
> *Example 1*
> |x-input: |1.5| 3.0| 6| 8| 12|13| 22| 24|28|31|
> |y-input: |3.1|6.1|3.1|2.1|1.4|5.1|5.1|6.1|7.1|7.2|
> |Output LoessInterpolator.smooth():|NaN|NaN|NaN|NaN|NaN|NaN|NaN|NaN|NaN|NaN|
> |Output from loessFit() from R: |3.191178027520974|3.0407201231474037|2.7089538903778636|2.7450823274490297|4.388011000549519|4.60078952381848|5.2988217587114805|5.867536388457898|6.7797794777879705|7.444888598397342|
> *Example 2 (same x-values, y-values just floored)*
> |x-input: |1.5| 3.0| 6| 8| 12|13| 22| 24|28|31|
> |y-input: |3|6|3|2|1|5|5|6|7|7|
> |Output LoessInterpolator.smooth(): |3|6|3|2|0.9999999999999005|5.0000000000001705|5|5.999999999999972|7|6.999999999999967|
> |Output from loessFit() from R: |3.091423927353068|2.9411521572524237|2.60967950675505|2.7421759322272248|4.382996912300442|4.646774316632562|5.225153658563424|5.768301917477015|6.637079139313073|7.270482144410326|
> As you see the output is practically the replicated y-input.
> At this point this funtionality is critical for us but I could not find any other suitable
java-implementation. Help. Maybe this strange behaviour gives someone a clue?

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message