qpid-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From conflue...@apache.org
Subject [CONF] Apache Qpid > Continuous Integration
Date Wed, 02 Nov 2011 08:17:00 GMT
<html>
<head>
    <base href="https://cwiki.apache.org/confluence">
            <link rel="stylesheet" href="/confluence/s/2042/9/21/_/styles/combined.css?spaceKey=qpid&amp;forWysiwyg=true"
type="text/css">
    </head>
<body style="background: white;" bgcolor="white" class="email-body">
<div id="pageContent">
<div id="notificationFormat">
<div class="wiki-content">
<div class="email">
    <h2><a href="https://cwiki.apache.org/confluence/display/qpid/Continuous+Integration">Continuous
Integration</a></h2>
    <h4>Page <b>edited</b> by             <a href="https://cwiki.apache.org/confluence/display/~k-wall">keith
wall</a>
    </h4>
        <br/>
                         <h4>Changes (1)</h4>
                                 
    
<div id="page-diffs">
                    <table class="diff" cellpadding="0" cellspacing="0">
    
            <tr><td class="diff-snipped" >...<br></td></tr>
            <tr><td class="diff-unchanged" >* Publishing SNAPSHOT versions of
Maven artefacts to the Apache repository after a successful nightly build <br> <br></td></tr>
            <tr><td class="diff-deleted-lines" style="color:#999;background-color:#fdd;text-decoration:line-through;">h3.
Issues <br> <br>The initial testing of the CI configuration is being carried out
using the [qpid-java-build|https://builds.apache.org/hudson/job/qpid-java-build/configure]
job only, until any issues are sorted out. <br> <br>Currently, there seems to
be a problem with the exclusion of various tests. For example, {{AcknowledgeAfterFailoverOnMessageTest}}
should not be running under the _default_ test profile. <br> <br>* [https://builds.apache.org/hudson/job/Qpid%20Java%20Build/ws/trunk/qpid/java/build/results/systests/TEST-org.apache.qpid.test.unit.ack.AcknowledgeAfterFailoverOnMessageTest.testAutoAck.err]
<br> <br>This appears to be caused by spaces in the job name. A copy of the original
job named *qpid-java-build* works as expected. <br> <br>All JMX based tests are
failing on Solaris instances. The operational profiles have been locked to _Ubuntu_ Hudson
instances to try and fix this. <br> <br>* [https://builds.apache.org/hudson/view/M-R/view/Qpid/job/qpid-java-build/3/#showFailuresLink]
<br> <br>{noformat} <br>java.rmi.server.ExportException: internal error:
ObjID already in use <br>	at sun.rmi.transport.ObjectTable.putTarget(ObjectTable.java:169)
<br>	at sun.rmi.transport.Transport.exportObject(Transport.java:74) <br>	at sun.rmi.transport.tcp.TCPTransport.exportObject(TCPTransport.java:229)
<br>	at sun.rmi.transport.tcp.TCPEndpoint.exportObject(TCPEndpoint.java:393) <br>
at sun.rmi.transport.LiveRef.exportObject(LiveRef.java:129) <br>	at sun.rmi.server.UnicastServerRef.exportObject(UnicastServerRef.java:190)
<br>	at sun.rmi.registry.RegistryImpl.setup(RegistryImpl.java:92) <br>	at sun.rmi.registry.RegistryImpl.&lt;init&gt;(RegistryImpl.java:68)
<br>	at java.rmi.registry.LocateRegistry.createRegistry(LocateRegistry.java:222) <br>
at org.apache.qpid.server.management.JMXManagedObjectRegistry.start(JMXManagedObjectRegistry.java:218)
<br>	at org.apache.qpid.server.registry.ApplicationRegistry.initialise(ApplicationRegistry.java:294)
<br>	at org.apache.qpid.server.registry.ApplicationRegistry.initialise(ApplicationRegistry.java:154)
<br>	at org.apache.qpid.test.utils.QpidBrokerTestCase.startBroker(QpidBrokerTestCase.java:472)
<br>	at org.apache.qpid.test.utils.QpidBrokerTestCase.startBroker(QpidBrokerTestCase.java:403)
<br>	at org.apache.qpid.test.utils.QpidBrokerTestCase.setUp(QpidBrokerTestCase.java:301)
<br>	at org.apache.qpid.server.security.acl.AbstractACLTestCase.setUp(AbstractACLTestCase.java:120)
<br>	at org.apache.qpid.server.security.acl.ExternalACLJMXTest.setUp(ExternalACLJMXTest.java:57)
<br>	at org.apache.qpid.test.utils.QpidBrokerTestCase.runBare(QpidBrokerTestCase.java:234)
<br>	at org.apache.qpid.test.utils.QpidTestCase.run(QpidTestCase.java:120) <br>{noformat}
<br> <br></td></tr>
            <tr><td class="diff-unchanged" >h3. Monitoring <br> <br></td></tr>
            <tr><td class="diff-snipped" >...<br></td></tr>
    
            </table>
    </div>                            <h4>Full Content</h4>
                    <div class="notificationGreySide">
        <p>Qpid uses the Apache Infrastructure CI service, which is a <del>Hudson</del><a
href="http://jenkins-ci.org/" class="external-link" rel="nofollow">Jenkins</a> installation.
More <a href="http://wiki.apache.org/general/Hudson" class="external-link" rel="nofollow">details</a>
are available from the Apache wiki. Builds can run on any operating system instance available,
or be locked to a particular type. It may be necessary to restrict some build types to Unix
instances only, but using Hudson will give access to Solaris and Windows Server build environments
which will be useful for interoperability testing. If a build tool such as Maven is used,
even tighter integration with the build process is available, including better test access
and the ability to use the Apache Maven repository for publishing artefacts.</p>

<h2><a name="ContinuousIntegration-Implementation"></a>Implementation</h2>

<p>The implementation of CI for Qpid is being tracked as <a href="https://issues.apache.org/jira/browse/QPID-3149"
class="external-link" rel="nofollow">QPID-3149</a> and will be completed during the
0.11 development stream.</p>

<h2><a name="ContinuousIntegration-Operation"></a>Operation</h2>

<p>All <a href="https://builds.apache.org/hudson/view/M-R/view/Qpid/" class="external-link"
rel="nofollow">builds</a> are prefixed with <em>qpid-</em> to allow grouping
together, the <em>Qpid</em> view shows all jobs matching this pattern. The proposed
build entries are as follows:</p>

<ul>
	<li><b>qpid&#45;java&#45;build</b> - triggered by check-ins to
subversion</li>
	<li><b>qpid&#45;java&#45;system&#45;tests</b> - nightly</li>
	<li><b>qpid&#45;java&#45;network&#45;build</b> - nightly, from
branch with network layer changes</li>
	<li><b>qpid&#45;cpp&#45;build</b> - triggered by check-ins to subversion</li>
	<li><b>qpid&#45;cpp&#45;system&#45;tests</b> - nightly</li>
	<li><b>qpid&#45;python&#45;tests</b> - nightly, full run of python
test kit</li>
	<li><b>qpid&#45;java&#45;<em>VERSION</em>&#45;build</b>
- triggered by check-ins to subversion, <em>VERSION</em> is current release branch</li>
	<li><b>qpid&#45;cpp&#45;<em>VERSION</em>&#45;build</b>
- triggered by check-ins to subversion, <em>VERSION</em> is current release branch</li>
	<li><b>qpid&#45;java&#45;release</b> - nightly or weekly, full
java release with poms</li>
	<li><b>qpid&#45;cpp&#45;release</b> - nightly or weekly, full cpp
release</li>
	<li><b>qpid&#45;interop&#45;tests</b> - nightly or weekly</li>
</ul>


<p>Note that it is possible to add builds from branches, to give increased visibility
to proposed changes, and greater confidence in their reliability due to increased testing.</p>

<p>At present, these tests run a single profile, such as <em>default</em>
or <em>java.0.10</em>. As the number of profiles increases, it may be better to
use the Hudson <a href="http://wiki.hudson-ci.org/display/HUDSON/Aboutncysa" class="external-link"
rel="nofollow">matrix project</a> feature to paramaterise the system test build around
the test profile names. Additionally, merging of the cpp and java jobs is probably desirable,
as is chaining build and test jobs together.</p>

<p>A timeout of 60 minutes is set for all jobs.</p>

<h4><a name="ContinuousIntegration-JavaConfiguration"></a>Java Configuration</h4>

<p>The Java builds use the latest 1.6 release of the JDK, and the latest available Ant
version, which are both dependant on the versions available on the Hudson instance running
the job.</p>

<p>Only JUnit test results are reported on currently.</p>

<p>The release job archives the following files:</p>

<ul>
	<li>&#42;&#42;/release/&#42;.zip</li>
	<li>&#42;&#42;/release/&#42;.tar.gz</li>
	<li>&#42;&#42;/release/maven/&#42;&#42;/&#42;.&#42;</li>
</ul>


<h4><a name="ContinuousIntegration-CppConfiguration"></a>Cpp Configuration</h4>

<p>The following commands each build Qpid:</p>

<ul>
	<li><tt>cd trunk/qpid/cpp &amp;&amp; cmake CMakeLists.txt &amp;&amp;
make &amp;&amp; make test</tt></li>
	<li><tt>cd trunk/qpid/cpp &amp;&amp; ./bootstrap &amp;&amp; ./configure
&amp;&amp; make &amp;&amp; make test</tt></li>
</ul>


<p>The C++ builds cannot use <tt>cmake</tt> currently, as it is not installed.
The <a href="http://wiki.hudson-ci.org/display/HUDSON/cmakebuilder+Plugin" class="external-link"
rel="nofollow">cmakebuilder Plugin</a> also needs to be installed. The autoconf build
also fails, due to missing header files: <a href="/confluence/download/attachments/25204999/qpid-cpp-build.txt?version=1&amp;modificationDate=1300835561000">console
output</a>. Additional tools like <tt>swig</tt> also need to be installed,
as well as the packages required for clustering.</p>

<p>Currently the C++ builds are disabled.</p>

<h3><a name="ContinuousIntegration-Status"></a>Status</h3>

<p>The following builds are currently operational, as of 22 March 2011:</p>

<ul>
	<li><a href="https://builds.apache.org/hudson/job/qpid-cpp-build/" class="external-link"
rel="nofollow">qpid-cpp-build</a> <b>disabled</b></li>
	<li><a href="https://builds.apache.org/hudson/job/qpid-cpp-0.10-build/" class="external-link"
rel="nofollow">qpid-cpp-0.10-build</a> <b>disabled</b></li>
	<li><a href="https://builds.apache.org/hudson/job/qpid-java-build/" class="external-link"
rel="nofollow">qpid-java-build</a></li>
	<li><a href="https://builds.apache.org/hudson/job/qpid-java-0.10-build/" class="external-link"
rel="nofollow">qpid-java-0.10-build</a></li>
</ul>


<p>These are triggered by subversion check-ins, checked every 15 minutes.</p>

<ul>
	<li><a href="https://builds.apache.org/hudson/job/qpid-java-release/" class="external-link"
rel="nofollow">qpid-java-release</a></li>
	<li><a href="https://builds.apache.org/hudson/job/qpid-java-0.10-release/" class="external-link"
rel="nofollow">qpid-java-0.10-release</a></li>
</ul>


<p>These are run daily at 00h00.</p>

<h3><a name="ContinuousIntegration-Amazon"></a>Amazon</h3>

<p>Due to the difficulty configuring the ASF Hudson instances to perform a C++ build,
particularly if we intend to use clustering, it will be easier to use an externally hosted
server. <a href="http://www.cloudsoftcorp.com/" class="external-link" rel="nofollow">Cloudsoft</a>
have generously agreed to provide some Amazon EC2 instances and storage to run a Hudson CI
server. This is currently being tested at the following URL:</p>

<ul>
	<li><a href="http://ec2-79-125-76-99.eu-west-1.compute.amazonaws.com:8080/job/qpid-cpp-build/"
class="external-link" rel="nofollow">http://ec2-79-125-76-99.eu-west-1.compute.amazonaws.com:8080/job/qpid-cpp-build/</a></li>
</ul>


<p>This server is an EC2 <em>c1.medium</em> instance, running Ubuntu with
an initial Hudson configuration from <a href="http://www.ciinabox.com/" class="external-link"
rel="nofollow">http://www.ciinabox.com/</a> and has had the following packages installed
via <tt>apt-get install</tt>:</p>

<ul>
	<li>build&#45;essential</li>
	<li>autoconf</li>
	<li>libtool</li>
	<li>ruby</li>
	<li>swig</li>
	<li>libboost.&#42;1.42-dev</li>
	<li>uuid&#45;dev</li>
	<li>libsasl.&#42;&#45;dev</li>
	<li>sasl2&#45;bin</li>
	<li>cmake</li>
	<li>libcorosync&#45;dev</li>
	<li>libopenais&#45;dev</li>
	<li>ant</li>
	<li>.&#42;jdk.&#42;</li>
	<li>maven2</li>
</ul>


<ul>
	<li>ln /etc/init.d/sonar /usr/bin/sonar</li>
	<li>mvn clean install sonar:sonar</li>
</ul>


<p>It is currently building the C++ code, then the Java code, running all the Java tests
on the <em>default</em> profile, then the system tests only on the <em>cpp</em>
profile, and generating Sonar output for metrics from the release-bin target output. We could
also run some other profiles, and I have configured the build to use an external shared workspace
at <tt>/home/qpid/workspace</tt> to simplify this.</p>

<h3><a name="ContinuousIntegration-Future"></a>Future</h3>

<p>Hudson can be configured to provide many more features with its built-in plugins.
These include:</p>

<ul>
	<li>FIXME/TODO task reporting</li>
	<li>Javadoc publishing</li>
	<li>Release artifact publishing</li>
	<li>Coverage reports</li>
</ul>


<p>The use of these plugins will be investigated once the basic build and test cycle
is satisfactorily configured. Future functionality propsed includes:</p>

<ul>
	<li>Publishing SNAPSHOT versions of Maven artefacts to the Apache repository after
a successful nightly build</li>
</ul>


<h3><a name="ContinuousIntegration-Monitoring"></a>Monitoring</h3>

<p>I propose creation of a new mailing list, <em>qpid-build@apache.org</em>
which would receive a copy of all output from every CI build. Additionally, the results of
failing builds should be mailed to committers that checked in the breaking changes, and a
nightly (or weekly) summary from the release and/or interop test builds should be sent to
the <em>qpid-dev</em> list.</p>

<p>Currently <a href="mailto:grkvlt@apache.org" class="external-link" rel="nofollow">Andrew</a>,
<a href="mailto:rajith@apache.org" class="external-link" rel="nofollow">Rajith</a>
and <a href="mailto:robbie@apache.org" class="external-link" rel="nofollow">Robbie</a>
have access to the build configurations.</p>
    </div>
        <div id="commentsSection" class="wiki-content pageSection">
        <div style="float: right;">
            <a href="https://cwiki.apache.org/confluence/users/viewnotifications.action"
class="grey">Change Notification Preferences</a>
        </div>
        <a href="https://cwiki.apache.org/confluence/display/qpid/Continuous+Integration">View
Online</a>
        |
        <a href="https://cwiki.apache.org/confluence/pages/diffpagesbyversion.action?pageId=25204999&revisedVersion=24&originalVersion=23">View
Changes</a>
                |
        <a href="https://cwiki.apache.org/confluence/display/qpid/Continuous+Integration?showComments=true&amp;showCommentArea=true#addcomment">Add
Comment</a>
            </div>
</div>
</div>
</div>
</div>
</body>
</html>

---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project:      http://qpid.apache.org
Use/Interact: mailto:commits-subscribe@qpid.apache.org


Mime
View raw message