qpid-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From conflue...@apache.org
Subject [CONF] Apache Qpid > Proton Release Steps
Date Wed, 23 Jan 2013 17:27:00 GMT
<html>
<head>
    <base href="https://cwiki.apache.org/confluence">
            <link rel="stylesheet" href="/confluence/s/2042/9/21/_/styles/combined.css?spaceKey=qpid&amp;forWysiwyg=true"
type="text/css">
    </head>
<body style="background: white;" bgcolor="white" class="email-body">
<div id="pageContent">
<div id="notificationFormat">
<div class="wiki-content">
<div class="email">
    <h2><a href="https://cwiki.apache.org/confluence/display/qpid/Proton+Release+Steps">Proton
Release Steps</a></h2>
    <h4>Page <b>edited</b> by             <a href="https://cwiki.apache.org/confluence/display/~rhs">Rafael
H. Schloming</a>
    </h4>
        <br/>
                         <h4>Changes (6)</h4>
                                 
    
<div id="page-diffs">
                    <table class="diff" cellpadding="0" cellspacing="0">
    
            <tr><td class="diff-unchanged" >h1. Producing Source Artifacts <br>
<br></td></tr>
            <tr><td class="diff-changed-lines" >The actual exercise of producing
the release tarballs right now is quite simple. It <span class="diff-added-words"style="background-color:
#dfd;">can be done by running {{bin/release.sh -v &lt;VERSION&gt;}}. It</span>
is little more than an svn export, however there are a few key things the release script does
beyond just <span class="diff-changed-words">this<span class="diff-added-chars"style="background-color:
#dfd;">, and they are worth understanding should it be necessary to modify the release
script</span>.</span> It starts by querying the head revision and storing it in
a variable so that all subsequent exports can be made from a specific revision. This serves
two purposes: <br></td></tr>
            <tr><td class="diff-unchanged" > <br># It guarantees that separate
exports will be of the same revision. Just doing svn export proton-c; svn export proton-j
could result in different svn revisions if there were intervening commits. <br># It
allows each release artifact to record the specific svn revision (along with repo and branch
info) from which the export was made. This guarantees that any release artifact can be mapped
back to a precise point in the history of our tree, even if it was made from just a random
snapshot of an arbitrary feature branch. <br> <br></td></tr>
            <tr><td class="diff-deleted-lines" style="color:#999;background-color:#fdd;text-decoration:line-through;">Another
extremely important factor here is that the release script itself should never have local
changes when producing a release. If it does, then this means the logic that generates the
release tarballs is not captured in the svn tree that the release artifacts themselves refer
to. The only time I ever run the release script with local changes is to test those changes
and I immediately throw away the artifacts generated from them. <br></td></tr>
            <tr><td class="diff-added-lines" style="background-color: #dfd;">Another
extremely important factor here is that the release script itself should never have local
changes when producing a release. If it does, then this means the logic that generates the
release tarballs is not captured in the svn tree that the release artifacts themselves refer
to. The only time the release script should ever have local changes is when testing those
changes themselves, and any artifacts generated by those test runs should be thrown away and
those changes checked in before producing proper artifacts. <br></td></tr>
            <tr><td class="diff-unchanged" > <br></td></tr>
            <tr><td class="diff-changed-lines" >The above measures are important
in ensuring that the releases are repeatable. Beyond this there is one structural change to
the source that the release script makes. The top level test directory is merged into both
the proton-c and proton-j tarballs. Also, for the <span class="diff-changed-words"><span
class="diff-deleted-chars"style="color:#999;background-color:#fdd;text-decoration:line-through;">j</span><span
class="diff-added-chars"style="background-color: #dfd;">J</span>ava</span>
release, this magic incantation is run: <br></td></tr>
            <tr><td class="diff-unchanged" > <br>{noformat} <br></td></tr>
            <tr><td class="diff-snipped" >...<br></td></tr>
            <tr><td class="diff-unchanged" >{noformat} <br> <br></td></tr>
            <tr><td class="diff-changed-lines" >Beyond this there is signing and
checksums and what not, but that is all documented as part of the standard apache release
<span class="diff-changed-words">process<span class="diff-deleted-chars"style="color:#999;background-color:#fdd;text-decoration:line-through;">,
so I won&#39;t mention anymore here</span>.</span> <br></td></tr>
            <tr><td class="diff-unchanged" > <br></td></tr>
            <tr><td class="diff-added-lines" style="background-color: #dfd;">Note
that it really doesn&#39;t matter what branching policy is used on the svn tree, e.g.
release candidates could be made from a feature branch, from trunk, or from a release branch
and because the artifacts capture the precise point in svn history they were made from, we
can always retroactively branch and tag from that point should the release candidate prove
worthy. <br> <br></td></tr>
            <tr><td class="diff-unchanged" >h1. Producing Binary Artifacts <br>
<br></td></tr>
            <tr><td class="diff-snipped" >...<br></td></tr>
    
            </table>
    </div>                            <h4>Full Content</h4>
                    <div class="notificationGreySide">
        <h1><a name="ProtonReleaseSteps-ProducingSourceArtifacts"></a>Producing
Source Artifacts</h1>

<p>The actual exercise of producing the release tarballs right now is quite simple.
It can be done by running <tt>bin/release.sh -v &lt;VERSION&gt;</tt>.
It is little more than an svn export, however there are a few key things the release script
does beyond just this, and they are worth understanding should it be necessary to modify the
release script. It starts by querying the head revision and storing it in a variable so that
all subsequent exports can be made from a specific revision. This serves two purposes:</p>

<ol>
	<li>It guarantees that separate exports will be of the same revision. Just doing svn
export proton-c; svn export proton-j could result in different svn revisions if there were
intervening commits.</li>
	<li>It allows each release artifact to record the specific svn revision (along with
repo and branch info) from which the export was made. This guarantees that any release artifact
can be mapped back to a precise point in the history of our tree, even if it was made from
just a random snapshot of an arbitrary feature branch.</li>
</ol>


<p>Another extremely important factor here is that the release script itself should
never have local changes when producing a release. If it does, then this means the logic that
generates the release tarballs is not captured in the svn tree that the release artifacts
themselves refer to. The only time the release script should ever have local changes is when
testing those changes themselves, and any artifacts generated by those test runs should be
thrown away and those changes checked in before producing proper artifacts.</p>

<p>The above measures are important in ensuring that the releases are repeatable. Beyond
this there is one structural change to the source that the release script makes. The top level
test directory is merged into both the proton-c and proton-j tarballs. Also, for the Java
release, this magic incantation is run:</p>

<div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent
panelContent">
<pre>mvn org.codehaus.mojo:versions-maven-plugin:1.2:set org.codehaus.mojo:versions-maven-plugin:1.2:commit
-DnewVersion="${VERSION}" -f ${WORKDIR}/${rootname}/pom.xml
</pre>
</div></div>

<p>Beyond this there is signing and checksums and what not, but that is all documented
as part of the standard apache release process.</p>

<p>Note that it really doesn't matter what branching policy is used on the svn tree,
e.g. release candidates could be made from a feature branch, from trunk, or from a release
branch and because the artifacts capture the precise point in svn history they were made from,
we can always retroactively branch and tag from that point should the release candidate prove
worthy.</p>

<h1><a name="ProtonReleaseSteps-ProducingBinaryArtifacts"></a>Producing
Binary Artifacts</h1>

<p>For the C build we simply don't do this. For the Java build there are a bunch of
steps that I don't really understand but have learned to monkey through:</p>

<ol>
	<li>untar and cd into the proton-j source tarball</li>
	<li>run "<tt>mvn -P apache-release deploy</tt>" (note that for this step
to succeed you need to do mysterious and annoying things like putting your apache password
in cleartext inside your .m2/settings.xml file)</li>
	<li>go to <a href="https://repository.apache.org/index.html" class="external-link"
rel="nofollow">https://repository.apache.org/index.html</a>, login and click on one
of the Staging links (Repositories I think) and find the thing that maven just uploaded (it
should say open or opened or something like that and have a recent date)</li>
	<li>poke at the clumsy ui until you figure out how to "close" it, and then record the
URL it gives you for the repo</li>
</ol>


<h1><a name="ProtonReleaseSteps-TestingtheCartifacts"></a>Testing the C
artifacts</h1>

<p>The challenge here is the variety of different environmental variables to factor
in. For proton-c itself we have different architectures to contend with as well as different
cmake versions and different versions of the few key libraries we depend on (e.g. openssl).
On top of this, for each binding we have not only the language version, but also the version
of swig available, and even if those are the same, the configuration of the interpreter may
vary depending on which OS we're building on.</p>

<p>All of this of course makes a myriad of combinations, and we don't really have much
in the way of CI coverage to help us yet, and even if we did, many of the factors are actually
related to install testing and won't actually show up unless you do a full make install as
root and try to access the proton library from each binding as an end user would. While not
impossible to automate this level of testing is of course much more difficult to automate,
so for the short to medium term at least, testing the C artifacts requires a reasonable amount
of manual checking.</p>

<p>The full check for any given configuration/environment would involve installing all
the dependencies required for any optional pieces (omitting any that are unavailable of course)
doing a full build and a make install (as root), running the python test suite (this checks
both the C library and the python binding), and then for each of the non-python bindings doing
at least a minimal check that the binding works and running any binding tests available.</p>

<p>During this process of course some steps may fail, e.g. the specific version of a
given language may not build, or the openssl library might not be recent enough, and some
level of that is ok of course as long as we document what did/did not work for the given configuration/environment.</p>

<h1><a name="ProtonReleaseSteps-Notes"></a>Notes</h1>

<p>The only stage here that really needs to be done by one person is producing the release
tarballs. I expect as we wish to offer more binary artifacts and producing them becomes more
complex (e.g. requiring a windows box) we will need to distribute the responsibility for producing
binaries, e.g. have a java developer produce the java binaries, have a windows developer produce
windows binaries, have a ruby developer produces gems, and so forth.</p>
    </div>
        <div id="commentsSection" class="wiki-content pageSection">
        <div style="float: right;">
            <a href="https://cwiki.apache.org/confluence/users/viewnotifications.action"
class="grey">Change Notification Preferences</a>
        </div>
        <a href="https://cwiki.apache.org/confluence/display/qpid/Proton+Release+Steps">View
Online</a>
        |
        <a href="https://cwiki.apache.org/confluence/pages/diffpagesbyversion.action?pageId=30750034&revisedVersion=2&originalVersion=1">View
Changes</a>
                |
        <a href="https://cwiki.apache.org/confluence/display/qpid/Proton+Release+Steps?showComments=true&amp;showCommentArea=true#addcomment">Add
Comment</a>
            </div>
</div>
</div>
</div>
</div>
</body>
</html>

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@qpid.apache.org
For additional commands, e-mail: commits-help@qpid.apache.org


Mime
View raw message