ant-ivy-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shawn Castrianni <Shawn.Castria...@halliburton.com>
Subject RE: Publishing C-module artifacts from different platforms
Date Thu, 13 Jan 2011 16:04:14 GMT
For that case, I have used a "local" publish.  All of my ivy settings files have a chain of
resolvers.  The very first one in the chain is always the local resolver which just publishes
to a local directory on the developer's computer.  When he runs dependencies, it will look
there first before trying to find the official one with all platform artifacts in the shared
repository for the company.  This local publishing specifies only the current platform configuration
so only the current platform artifacts that the dev just built are published to this other
place on his hard drive.  He could then go to another module sandbox that is dependent on
the one he just built and locally published, run dependencies, and the locally built artifacts
come in as a dependency allowing him to write code against changes he made in that other sandbox.
 Essentially, what this gives to the dev is the ability to develop in multiple sandbox modules
with dependencies between each other and test his changes across modules before committing
any code changes to any of their respective source repositories.

Here is the chain I use in production for all 500 of my developers:

		<!--Chain resolvers-->
		<chain name="published.repository.daily.chain" returnFirst="true">
			<resolver ref="published.local"/>
			<resolver ref="published.repository.daily"/>
			<resolver ref="thirdparty"/>
		</chain>

		<filesystem name="published.local" force="true" cache="published.local.cache">
			<ivy pattern="${env.PUBLISH_LOCAL_ROOT}/${ivy.pattern}"/>
			<artifact pattern="${env.PUBLISH_LOCAL_ROOT}/${artifact.pattern}"/>
		</filesystem>
		<filesystem name="published.repository.daily" force="${published.force}">
			<ivy pattern="${ivy.published.root}/daily/${env.RELEASE_NAME}/${ivy.pattern}"/>
			<artifact pattern="${ivy.published.root}/daily/${env.RELEASE_NAME}/${artifact.pattern}"/>
		</filesystem>
		<svn name="thirdparty" repositoryRoot="http://thirdparty.src.lgc.com/anonymous/thirdparty/trunk"
userName="renobuild">
			<ivy pattern="[organisation]/[module]/[revision]/[type]/[artifact](.[ext])"/>
			<artifact pattern="[organisation]/[module]/[revision]/[type]/[artifact](.[ext])"/>
		</svn>


You can see that for the local publish resolver, I use ${env.PUBLISH_LOCAL_ROOT} in the pattern.
 This gives the dev the choice of where he wants his local publish repository to be located
on his computer by simply setting the env variable PUBLISH_LOCAL_ROOT.  You can then see that
the main official repository from the official CM builds come from published.repository.daily.
 And then if the dependency is not found in either of those, then it looks in the SVN based
resolver called thirdparty where we store all of our thirdparty dependencies.  Sure, we could
have used biblio or ivyrep resolvers to grab thirdparty stuff from the Internet, but our legal
team is very strict on the use of thirdparty components.  Therefore, each one has to be approved.
 Once approved, I copy it into our own svn thirdparty repository.

---
Shawn Castrianni

-----Original Message-----
From: Felix Drueke [mailto:fdrueke@orga-systems.com] 
Sent: Thursday, January 13, 2011 1:23 AM
To: ivy-user@ant.apache.org
Subject: Re: Publishing C-module artifacts from different platforms

Shawn, thanks for that valuable input.

I assume we're going for a pretty similar procedure.
We're using Hudson to verify that the build still works on all platforms
and compilers.
So I guess it's also possible to have Hudson publish the artifacts right away
(didn't try that yet).

However there's still one reason why I think that artifacts for just one
platform should be publishable (at least for dev-snapshots).
That is if a developer wants to publish the results of his latest work
in his current workspace. In that case he'll just want to publish for
just one platform and that's sufficient in that situation.

Felix


Shawn Castrianni wrote, on 01/12/2011 05:19 PM:
> I can't answer your question, but I can tell you how I handle this.
>
> I felt it was a bad idea to publish a partial module such that one platform's artifacts
are present and other are not.  If this happens because of a compiler error on one platform
but not others, then you can get into a bad situation.  Such as, someone saying that my new
feature works on linux but not on windows, not knowing that it works on linux because the
new feature was compiled successfully on that platform and he is using the new published module.
 But when he tries on windows, he is picking up on old published module since a new one with
windows artifacts is not present due to the compile error.  Stuff like that.
>
> Therefore, what I do is just as you describe, using separate configuration for each platform.
 But then I use a good automated build system (QuickBuild 3.1) to wrap ivy commands.  So my
QB build workflow is:
>
> 0. randomly select a master build machine from the pool, let's call it MASTER
> 1. MASTER:  checkout source on any machine
> 2. MASTER:  tar up source
> 3. MASTER:  copy source to other platform build machines
> 4. EACH PLATFORM:  untar onto each of those machines
> 5. MASTER:  delete tar file
> 6. MASTER:  clean.all
> 7. MASTER:  ivy dependencies (specifying proper conf setting so that All platforms dependency
artifacts are downloaded)
> 8. MASTER:  perform java compile
> 9. MASTER:  copy dependencies and resulting jars from java compile to other platform
build machines
> 10: EACH PLATFORM:  perform native code build
> 11: EACH PLATFORM:  perform unit testing
> 12: EACH PLATFORM:  copy resulting artifacts from each platform machine to master machine
> 13: MASTER:  publish all platform artifacts at once
>
> By using a good automated build system, you can easily handle this multi platform build
workflow and the hand off of control from one node to another.  Then the publishing ONLY happens
if ALL platforms are successful.  Therefore, I do not need the ability to publish a partial
module.
>
> I am sure there are other ways to do this and there is probably debate as to how efficient
this is, but it works very well in a production environment.  If this sounds like an advertisement
for QuickBuild, I apologize.  I do not work for them.  I am just a passionate happy customer.
>
> ---
> Shawn Castrianni
>
> -----Original Message-----
> From: Felix Drueke [mailto:fdrueke@orga-systems.com]
> Sent: Wednesday, January 12, 2011 4:59 AM
> To: ivy-user@ant.apache.org
> Subject: Publishing C-module artifacts from different platforms
>
> Hi,
>
> I'd like to manage the dependencies of a set of C-modules with ivy.
> Each of these modules needs to be compiled on different platforms (i.e.
> Solaris, Linux, HP-UX and others).
>
> This requires that the same sources are compiled in separate workspaces
> (on the different hosts).
>
> In a first attempt I setup one ivy.xml with a set of configurations - one for
> each platform on that I have to build&publish artifacts.
>
> Being unexperienced with Ivy I didnd't find a way to just publish the
> artifact for the one configuration that I built in a workspace.
> For example I built module XYZ on Solaris and of course I can only
> publish the Solaris-artifact from that workspace. In another worlspace
> I got the artifact for Linux and of course I can only publish that.
>
> How do you accomplish to publish just the artifacts for one particular
> configuration?
>
> Thanks for any hint,
> Felix
>
> The information included in this e-mail and any files transmitted with it is strictly
confidential and may be privileged or otherwise protected from disclosure. If you are not
the intended recipient, please notify the sender immediately by e-mail and delete this e-mail
as well as any attachment from your system. If you are not the intended recipient you are
not authorized to use and/or copy this message and/or attachment and/or disclose the contents
to any other person.
>
> ----------------------------------------------------------------------
> This e-mail, including any attached files, may contain confidential and privileged information
for the sole use of the intended recipient.  Any review, use, distribution, or disclosure
by others is strictly prohibited.  If you are not the intended recipient (or authorized to
receive information for the intended recipient), please contact the sender by reply e-mail
and delete all copies of this message.
>

The information included in this e-mail and any files transmitted with it is strictly confidential
and may be privileged or otherwise protected from disclosure. If you are not the intended
recipient, please notify the sender immediately by e-mail and delete this e-mail as well as
any attachment from your system. If you are not the intended recipient you are not authorized
to use and/or copy this message and/or attachment and/or disclose the contents to any other
person.

Mime
View raw message