Mailing-List: contact continuum-dev-help@maven.apache.org; run by ezmlm
Precedence: bulk
Reply-To: continuum-dev@maven.apache.org
Received-SPF: neutral (asf.osuosl.org: local policy)
Mime-Version: 1.0 (Apple Message framework v752.2)
In-Reply-To: 
 <OF737EFFC3.2E482661-ON872571D5.004E1C89-872571D5.00649893@sungardhe.com>
References: 
 <OF737EFFC3.2E482661-ON872571D5.004E1C89-872571D5.00649893@sungardhe.com>
Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed
Message-Id: <FEA1062B-3097-420E-859F-96F19BE3D7EA@apache.org>
Content-Transfer-Encoding: 7bit
From: Brett Porter <brett@apache.org>
Subject: Re: Release Management for Maven/Continuum
Date: Sat, 26 Aug 2006 19:07:01 +1000
To: continuum-dev@maven.apache.org

At the risk of oversimplifying, you are saying you need two things:
- a descriptor that lists all the files and their versions that make  
up the release, independent of SCM tag
- a descriptor that describes how the software was built (versions of  
everything)

The first is basically an SCM independent tagging mechanism, right?

The second is going to be required whether you use the above or you  
use a "real" tag. Is that correct?

The second has been on our "to do" list for some time surrounding the  
release plugin. There was some prior support but it didn't work  
perfectly. It's definitely needed for reproducible builds,  
independently of what Continuum needs.

The first sounds like an interesting thing to add to Maven SCM and  
utilise from the release process as you've described (as a  
preliminary, recorded "tag" that can be applied to the SCM later).

I think both of these are possibilities, but since they are generally  
useful they should be implemented elsewhere first and integrated into  
this feature. They'll fit in with the design that's being put  
together here.

Cheers,
Brett

On 26/08/2006, at 4:18 AM, Jan Nielsen wrote:

> Hi Brett,
>
> (Sorry, this got longer than I thought it would...)
>
> Yes; the "build labeling" would be a nice addition.
>
> I may be missing an important point in the release management  
> process that
> is being discussed so if you will endulge me, let me try to clarify my
> understanding, so you can help me understand the discussion.
>
> I think there are two important aspects which defines a "release":
> persistent meta-information identifying /what/ defines the release,  
> and a
> reproducible process defining /how/ to produce the release. With this
> information, it is possible to (re-)generate an arbitrary release,  
> and to
> branch an arbitrary release producing a new release related to that
> previous release (assuming the files and tools used for the release  
> are
> still available).
>
> As you note, there are two different broad tactics for generating a
> release: "tag-pull-build-publish", and "pull-build-publish[-tag]". The
> "tag" is an SCM operation which defines the set of files to operate on
> ("build labeling"); the "pull" is an SCM operation to get files  
> from SCM;
> the "build" is a build server operation to generate binaries from the
> source from SCM; and "publish" is distribution of the resultant binary
> (which may include test cycles, internal publication, etc.).
>
> The "tag-pull-build-publish" scheme has the benefit of having the SCM
> define the /"what"/ of the release; it generally does not define the
> /"how"/ until you combine the process with something like Maven and
> Continuum which has a defined process for the /"how"/. The drawback is
> that the "tag" process can be lengthy and possibly very broad.
>
> The "pull-build-publish[-tag]" scheme has the benefit of being the  
> same
> process that we go through as for any other build with the  
> exception that
> the /"what"/ is now not simply what is in the SCM system right now,  
> but
> rather is something very specific, e.g., "a/b/c.java#34, a/b/d.java#2,
> etc.", as specified in the "release descriptor". Since the release
> descriptor defines exactly what is in the release, we can "tag" at any
> later time without impacting the development cycle.
>
> For example, project "A" has two files a.java and a.xml. The  
> project has
> been building for a while producing builds 12, 13, etc. The build 12
> contains a.java#4 and a.xml#23, along with its meta data files, to  
> wit:
>
>   A#12: a.java#4, a.xml#23 pom.xml#22 profile.xml#3 settings.xml#5
>
> (this reads build 12 of project "A" contains "a.java" version 4 and
> "a.xml" version 23) and subsequent builds:
>
>   A#13: a.java#5, a.xml#23 pom.xml#22 profile.xml#3 settings.xml#5
>   A#14: a.java#6, a.xml#24 pom.xml#22 profile.xml#3 settings.xml#5
>   A#15: a.java#6, a.xml#25 pom.xml#22 profile.xml#3 settings.xml#5
>
> In general, the "tag-pull-build-publish" crowd would do:
>
>   tag "1.0" to all A#14 files
>   pull all "1.0" files
>   build
>   publish
>
> In general, the "pull-build-publish[-tag]" crowd would do:
>
>   pull all A#14 files
>   build
>   publish
>   optionally tag all A#14 files with "1.0"
>
> One way to implement this is to add a "release" button here:
>
>   http://people.apache.org/~epunzalan/continuum-white-site/index.html
>
> so you would have "Build Now | Release Now | Build and Release | Build
> History | Working Copy | Delete".
>
> By pressing the "Release", Continuum would ask the release engineer
> provide a "release tag", and then do the "pull-build-publish[-tag]"
> operations. By pressing the "Tag and Release", Continuum would ask the
> release engineer provide a "release tag", and then do the
> "tag-pull-build-publish" operations. In both modes, Continuum is
> rebuilding the build that the CI cycle already built. This process can
> probably be made more elegant.
>
> The "A#14" is the "release descriptor" but it needs to fleshed out,  
> as was
> mentioned. It should include full descriptions of the tools  
> required to
> reproduce the build, e.g., Maven version numbers, Continuum version
> numbers, Ant version numbers, JDK version numbers, OS version numbers,
> etc.
>
> Thoughts?
>
>
> -Jan
>
>
> Jan Nielsen * System Architect * SunGard Higher Education * Tel +1  
> 801 257
> 4155 * Fax +1 801 485 6606 * jan.nielsen@sungardhe.com *  
> www.sungardhe.com
> * 90 South 400 West, Suite 500, Salt Lake City, UT USA
>
> CONFIDENTIALITY: This email (including any attachments) may contain
> confidential, proprietary and privileged information, and unauthorized
> disclosure or use is prohibited.  If you received this email in error,
> please notify the sender and delete this email from your system. Thank
> you.
>
>
>
> Brett Porter <brett@apache.org>
> 08/24/2006 09:41 PM
> Please respond to
> continuum-dev@maven.apache.org
>
>
> To
> continuum-dev@maven.apache.org
> cc
>
> Subject
> Re: Release Management for Maven/Continuum
>
>
>
>
>
>
> Hi Jan,
>
> The release is indeed modifying the pom, tagging the code, and doing
> a full build and deploy cycle of that code to produce a formal  
> release.
>
> I think there is definitely room for another feature which I've
> always referred to as "build labeling" where you take the existing
> continuum build, label it as a certain milestone, and then if it
> passes into the final release stage goes through the build/tag
> procedure.
>
> Some organisations like to do the opposite and tag every build and
> then its just a matter of picking which is final. So the two would
> need to cooperate.
>
> - Brett
>
> On 25/08/2006, at 1:41 AM, Jan Nielsen wrote:
>
>> Sorry for jumping in here but I'm not entirely sure I understand the
>> big-picture process you are describing. Is the only difference
>> between a
>> "build" and a "release" the presence of meta-data describing what you
>> built the release from (and possibly the life-time of that meta- 
>> data)?
>>
>> If so, one way to implement a "release" is to "tag" everything with
>> some
>> release moniker, e.g., "1.0", and then retrieve all these tagged
>> files,
>> then build it, and then publish it as your "release 1.0". This has  
>> the
>> obvious benefit of using the SCM system to track release numbers,
>> but the
>> drawback that some SCM systems take along time to perform the tag
>> operation and it requires a two step build procedure, tag-and-
>> build, which
>> is not the normal build cycle. For releases spanning multiple
>> modules this
>> scheme become complex and very broad. (Is the intent of the
>> "prepare" step
>> to perform the "tag" SCM operation?) To recover a past release
>> build, you
>> then use the SCM to retrieve the "tagged" files - easy enough.
>>
>> Another way is denote a release is to capture the meta-data of
>> files in
>> the build, and dependent modules, and then check-in this release
>> descriptor with the release tag. The drawback to this scheme is
>> obviously
>> that the SCM store no longer directly identifies the set of files
>> with an
>> explicit release number. But the benefit is that no single "tag"
>> operation
>> needs to be done on the entire file set before the build and the
>> built set
>> of files is collected after the build operation which can be now be
>> done
>> for each build. The release descriptor will then contain the uniquely
>> identifying information for all files built, collected after the SCM
>> update operation, and all release descriptors of dependent  
>> modules. To
>> recover a past build, you then use the SCM to get the release
>> descriptor
>> and from the release descriptor pull the appropriate POM and files
>> from
>> the SCM - this is now a two step procedure, but these two steps
>> could be
>> automated.
>>
>> As you may have guessed, each of our builds is a "release" which
>> enables
>> us after QC completes to define build 12345 as "Release 1.0" (and
>> once the
>> release has been defined, you could go back and "tag" each file in  
>> the
>> build with the release moniker if you really want the SCM to hold  
>> this
>> information explicitedly).
>>
>> I hope I didn't stir the water. Thoughts?
>>
>> -Jan
>>
>>
>> Jan Nielsen * System Architect * SunGard Higher Education * Tel +1
>> 801 257
>> 4155 * Fax +1 801 485 6606 * jan.nielsen@sungardhe.com *
>> www.sungardhe.com
>> * 90 South 400 West, Suite 500, Salt Lake City, UT USA
>>
>> CONFIDENTIALITY: This email (including any attachments) may contain
>> confidential, proprietary and privileged information, and  
>> unauthorized
>> disclosure or use is prohibited.  If you received this email in  
>> error,
>> please notify the sender and delete this email from your system.  
>> Thank
>> you.
>>
>>
>>
>> Brett Porter <brett@apache.org>
>> 08/23/2006 10:37 PM
>> Please respond to
>> continuum-dev@maven.apache.org
>>
>>
>> To
>> continuum-dev@maven.apache.org
>> cc
>>
>> Subject
>> Re: Release Management for Maven/Continuum
>>
>>
>>
>>
>>
>>
>>> The summary page shows only a few of the release parameters. So the
>>> "Edit" link is there to direct the user to the more detailed
>>> release configuration page.  But since we'll be releasing projects
>>> one at a time, I guess I can incorporate what you mean into the new
>>> white-site.
>>
>> Just to be clear - if the single project that is released has
>> modules, there will be multiple entries on this page.
>>
>> I think it should be a big long form, though, because it would be
>> tedious to change individual values project per project when you need
>> to edit them like that (unless we get all ajax, but that might be a
>> separate UI initiaive)
>>
>>>>
>>> ok, so this means continuum should remember prepared releases.
>>> Should there be a separate release working directory for this?
>>> Because a prepared release may get lost after a scheduled build.
>>
>> A prepared release is simply a tag in the SCM, I think (you might
>> want to double check that that is all release:perform reads back from
>> the release properties).
>>
>> Basically what wuld happen here, after fleshing out the model, is
>> that it would replace the configuration store in the release plugin
>> (so you could use Xpp3Reader/Writer to store release.xml instead of
>> release.properties), and the same thing could be used to store a
>> release's information in the database, along with the information
>> here.
>>
>> Anyway, maybe I'm going overboard on that, but its something to think
>> about.
>>
>> - Brett
>>
>