On Tue, Jul 3, 2012 at 6:39 AM, Linyi Li wrote: > From Xuan Xuan's introduction in the beginning, I think the first data is > average value of the test results and the second data is the standard > deviation. > So why skip the numbers for the first round test? Isn't that what real users see, the first round? Sure, it will be slower as code is loaded into memory, files read from disk, etc. But the same thing happens for users. Also, I think the interesting 2nd number is the "standard error of the mean", which == std deviation / sqrt(count of measurements). That is what gives the error bars (confidence interval) on the measurement. For example, 95% confidence limits on a measurement would be: lower bound = mean - 1.96*standard_error upper bound = mean + 1.96*standard_error And easy "rule of thumb" is to compare the "before" and "after" measures and see if there is overlap in the intervals. For example: Before interval: (1.0, 2.0) After interval: (1.5, 2.5) Because the intervals overlap, there might not be a significant difference between the two. But: Before interval: (1.0, 2.0) After interval: (2.5, 3.5) In this case there is a clear difference, because the confidence intervals do not overlap. A t-test could also be used here, but the above approach works well in Calc if you use the "stock 2" type chart. This has series for high/low/close/open. So you could do something where the high and low values are 95% confidence intervals. This makes it easy to tell what is important from a glance. -Rob > > On Tue, Jul 3, 2012 at 5:36 PM, Ji Yan wrote: > >> I'm sure Yi Xuan will update her wiki page with detail test case and the >> meaning of report data >> >> 2012/7/3 Andre Fischer >> >> > On 03.07.2012 09:02, Herbert Duerr wrote: >> > >> >> |---------|-------------------**--------|---------------|-----** >> >>> ------------| >> >>> | Filter | odt Load Show | Plain | 0.72/ 0.03 | >> >>> | | | Complex | 1.13/ 0.03 | >> >>> |---------|-------------------**--------|---------------|-----** >> >>> ------------| >> >>> [...] >> >>> Any comment is welcomed! >> >>> >> >> >> >> Thanks for sharing this interesting data. >> >> I haven't found an explanation what these numbers mean though, so I have >> >> to guess: The first number is the average value and the second is the >> >> sigma value for running the test, right? >> >> >> > >> > It would be good to put any explanation/documentation on the wiki page or >> > else the information about the test parameters from the first mail -- (8 >> > runs, average over 5), what is plain or complex -- would be lost. >> > >> > -Andre >> > >> >> >> >> -- >> >> >> Thanks & Best Regards, Yan Ji >> > > > > -- > Best wishes. > Linyi Li