Mailing-List: contact dev-help@stdcxx.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@stdcxx.apache.org
Received-SPF: pass (nike.apache.org: domain of msebor@gmail.com designates
 209.85.198.233 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=message-id:date:from:user-agent:mime-version:to:subject:references
         :in-reply-to:content-type:content-transfer-encoding;
        b=daHdRGIaUOwJGnyPFN+Xmd5knj9dUsqRn+73CPbvFPz6VoC8SaommTwytBOzChDSyz
         ZY5M3djZvya4VosCrrq/GgoXqUsmV+HytN8xHiNjfWwgOpAmBVoJVvExrPGG/jpfE6uD
         iGOxZ5BC6ZA9XHLiYbkMnSCD5pDTjt5kp1Sck=
Message-ID: <4A2C5059.1070101@gmail.com>
Date: Sun, 07 Jun 2009 17:42:17 -0600
From: Martin Sebor <msebor@gmail.com>
User-Agent: Thunderbird 2.0.0.21 (X11/20090320)
MIME-Version: 1.0
To: dev@stdcxx.apache.org
Subject: Re: Buildbot - Useful for nightly testing?
References: <0D16DE842D58469AA649EDD1DE9C5FEB@developer>
In-Reply-To: <0D16DE842D58469AA649EDD1DE9C5FEB@developer>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Gavin wrote:
> Hi All,

Hi Gavin!

> 
> I noticed from the last board report that you are looking for a
> replacement(s) for nightly testing.
> 
> We have a Buildbot instance at http://ci.apache.org/buildbot.html &&
> http://ci.apache.org/waterfall .
> 
> I have seen some of your test results at stdcxx.apache.org/builds/ and these
> look comprehensive. Whilst Buildbot may not be as thorough with producing
> results I hope that it can help in some way with your build testing.
> 
> Currently Buildbot is only available on Ubuntu. There are plans to expand
> the build farm and so Buildbot will be available on other platforms over the
> next few months.
> 
> Some questions if I can.
> 
> 1. Interested?

Definitely!

> 
> 2. Id like to test Stdcxx on Buildbot on our Ubuntu machine, can someone run
> through what simple tests we can run for now? - I'll about getting it on
> there over the next few days.

I'll be interested in looking at the results. I'll need some time
to better understand the current presentation format.

> 
> 3. I would like a summary of what you would need to replicate either your
> current build/test setup, or what you believe would be good to have at
> Apache for your needs - consider that no machine is likely to be dedicated
> to one project, but rather shared in a way to benefit all. Mention any
> features of post-build outcomes you have or desire also.

The stdcxx builds pages present the build and test results across
the set of tested configurations of the project on each platform
(a unique combination of compiler, operating system, and hardware
architecture).

For example, the page below consists of 5 tables showing the build
and test results of 16 configurations of stdcxx with gcc 4.1.1 on
Solaris 10/SPARC:

http://stdcxx.apache.org/builds/4.2.x/solaris-10-sparc-gcc-4.1.1.html

The first table, Logs and Columns (http://tinyurl.com/mvwufq#logs),
gives summary info about each build (the OS, hardware architecture,
compiler, the "build type" -- see the last table on the page, titled
Build Types), how old the build result is (WRT the date the page was
created), the svn revision the result corresponds to, the sizes of
various artifacts of the build process, and the number of diagnostics
issued during each stage of the build process.

The second table, Timings (http://tinyurl.com/mvwufq#timings) gives
timings of the stages of the build process.

The next three tables, titled "Results of %d %s from %d logs" show
the results of compiling, linking, and running each of stdcxx tests
(those are divided into tests of locales, unit tests, and example
programs supplied with the project). Every row in each of these
tables represents a single test. Every column corresponds to each
of the builds (configurations) of the project. The cell format,
including the color scheme, is documented in a table titled Codes
and Colors toward the bottom of the page (see
http://tinyurl.com/mvwufq#codes). Only rows with "interesting"
results (i.e., those indicating a problem of some kind) are shown.

The pages are generated from plain text logs created by the build
and test script distributed with stdcxx:
http://svn.apache.org/viewvc/stdcxx/branches/4.2.x/bin/buildntest

The script responsible for generating the pages is xbuildgen:
http://svn.apache.org/viewvc/stdcxx/branches/4.2.x/bin/xbuildgen

Martin

> 
> Whilst I couldn't guarantee anything, I would like to take this to
> infrastructure and use it in consideration when planning our purchasing
> requirements for the build farm expansion.
> 
> Thanks
> 
> Gav...
>