incubator-stdcxx-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark Brown <mark.g.br...@gmail.com>
Subject Re: [RFC] build result views
Date Fri, 23 Nov 2007 18:01:23 GMT
Have you guys seen the Boost regression test page?
http://engineering.meta-comm.com/boost-regression/1_34_1/user/summary_release.html
I think there are some good ideas there. I like the links to
the test sources and to the compiler errors for the ones that
error out.

--Mark

Martin Sebor wrote:
> Farid Zaripov wrote:
>>> -----Original Message-----
>>> From: Martin Sebor [mailto:msebor@gmail.com] On Behalf Of Martin Sebor
>>> Sent: Tuesday, November 13, 2007 6:18 AM
>>> To: stdcxx-dev@incubator.apache.org
>>> Subject: [RFC] build result views
>>>
>>> I'm looking for feedback on the two sets of nightly result pages we 
>>> currently publish:
>>>
>>>    http://people.apache.org/~sebor/stdcxx/results/
>>>
>>> and
>>>
>>>    http://people.apache.org/~sebor/stdcxx/results/builds
>>>
>>> Specifically, I'm wondering what would people think about replacing 
>>> the first page with the second, or something like it. Do you find 
>>> yourself using the first page more or do you prefer the second one, 
>>> and why?
>>
>>   The first page is useful to see the overall status of the library.
>> I'm using it just for sure that Windows builds are green :)
> 
> :) That's how I've been using it too. The danger of colorizing
> entire builds the way we do on that page is in making it easy
> to miss important failures occurring only in a small number of
> tests. I.e., you might see green even when a critical piece of
> the library is broken (recall the recent binary incompatibility
> with XLC++ exception).
> 
> Anyway, sounds like adding colorization either to builds.html
> or to the Logs and Columns table on each Multi-platform Test
> Result View is one enhancement you'd like to see, correct?
> 
> What about data? Do you use any of the data from the colorized
> page? E.g., the number of components (examples, tests, locales)
> vs the number of those that failed? I personally don't think
> they are terribly useful but adding them shouldn't be too hard.
> I do plan on adding the duration column (with a lot more detail,
> such as how long the library took to build in user, system, and
> wall clock time, and the same for all examples, the test driver,
> tests, and locales). Anything else?
> 
>>
>>   Of cource the second page is more convinient to see what
>> examples or tests are failed and how these fails dependent
>> from the build type.
>>
>>   I would like to have the possibility to merge the results of the
>> multiple platforms into the one page, i.e.:
>> - all windows builds (MSVC and ICC);
>> - all MSVC builds;
>> - all ICC/windows builds;
>> - all ICC builds (windows and linux);
>> - all MSVC 8.0 build + all gcc 4.2.0 builds;
> 
> This is something I'd like to be able to do as well, and I have
> in a small number of cases. It can easily be done by changing
> the genxviews script to generate whatever combination of builds
> you need (we should move the data out of the script to make it
> possible to do this without modifying the script itself).
> 
> After making the changes you just run your custom genxviews like
> this:
> 
>   $ genxviews > $HOME/public_html/stdcxx/results/builds.html
> 
> Of course, the ultimate implementation would let you do it on
> demand (e.g., as you suggest below).
> 
> One thing to keep in mind is that the more builds you squeeze on
> a page the harder it becomes to see them all at the same time. At
> a certain point it starts to defeat the purpose of the page because
> you end up scrolling it left and right to see the results for all
> the platforms.
> 
>> ...
>>
>>   It might be realized as the current page
>> (http://people.apache.org/~sebor/stdcxx/results/builds)
>> but with the checkboxes on the each line of the table and button
>> "Next" somewhere on the page.
> 
> That would be pretty cool. The only thing is that generating the
> pages takes quite a bit of time (you can see how long in the Time
> column), so you might have to wait a few minutes to get the results
> for a custom selection. We could probably optimize it to just a few
> seconds by pre-processing the individual logs so as not to make the
> script work so hard.
> 
> Martin
> 
>>
>>> If neither, how do you analyze build results and why do you find your 
>>> system preferable to what we have? What does your ideal result page 
>>> look like? What data should it show and how should it be presented?
>>
>> Farid.
>>
> 
> 


Mime
View raw message