geronimo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kevan Miller <kevan.mil...@gmail.com>
Subject Re: Speeding up G3.0 build time
Date Wed, 22 Jun 2011 17:02:53 GMT

On Jun 22, 2011, at 12:09 PM, Russell E Glaue wrote:

> I am wanting to update the G3 dev wiki to help first-time builders get started.
> How would you suggest a build of G3 could become faster?
> 
> The only tip I am aware of for speeding up trunk (G3) builds is to install a
> maven repository proxy like nexus. This is eluded to in the wiki already.
> Are there any other tips?

You can build without tests. E.g.: 'mvn clean install -DskipTests=true'. If I'm simply running
a build, I'll frequently turn off tests ('-DskipTests=true' I think '-DskipTests' is equivalent).
If I'm making code changes, I run with tests.

You can also build in an offline mode (e.g. 'mvn -o clean install -DskipTests'). This avoids
searching of maven repos, all together. So, you do need one full online build, before this
will work. Due to the current development work going on in OpenWebBeans, OpenEJB, Aries, and
Geronimo (I think these are the most frequently changing projects, ATM), you run the risk
of getting out-of-date between the projects.

FYI, we are running Continuous Integration builds. E.g. -- http://ci.apache.org/builders/geronimo-server-trunk/builds/106


I encourage people to build from source. However, if you really want pre-built binaries of
unreleased versions of Geronimo, assemblies from our CI builds are uploaded to the apache
snapshot repository. So, it is possible to download SNAPSHOT builds from the SNAPSHOT repository.
For example: https://repository.apache.org/content/repositories/snapshots/org/apache/geronimo/assemblies/geronimo-tomcat7-javaee6/3.0-SNAPSHOT/

> 
> 
> I have been able to successfully build G3 from trunk several times now but with
> differing build time results.
> 
> The best build time I have had is 13 hours.
> The worst build time I have had is 39 hours. (this is from my latest build)

Ouch. My build times are nowhere near this long. Are you running with an empty local maven
repository?

> 
> I am using a very basic build procedure.
> 1. checkout G3 trunk
> 2. JAVA_HOME="/path/to/jdk/1.6.0_22"
> 3. download and install a fresh maven 2.2.1
> 4. MAVEN_OPTS="-Xmx968m -XX:MaxPermSize=500m -XX:ReservedCodeCacheSize=64m"

My settings are similar. I don't specify '-XX:ReservedCodeCacheSize=64m'.

I run with the following. I know there are others running with larger max heaps. I haven't
:

MAVEN_OPTS="-Xmx1024m -XX:MaxPermSize=512m -XX:+HeapDumpOnOutOfMemoryError"

> 5. mvn clean install
> 6. 13 to 39 hours later == BUILD SUCCESS
> 
> 
> From observing the build process, it would seem the most used (seemingly paused)
> time is right before the build process moves on to the next plugin or assembly.
> 
> This "seemingly paused time" after the assembly I assume is the packaging
> process, because the resulting output is a huge collection.
> 
> [INFO] Geronimo Assemblies ............................... SUCCESS [3.995s]
> [INFO] Geronimo Assemblies :: Minimal + Jetty8 ........... SUCCESS [8:15.210s]
> [INFO] Geronimo Assemblies :: Java EE 6 Web Profile + Jetty8  SUCCESS [1:57:17.311s]
> [INFO] Geronimo Assemblies :: Java EE 6 + Jetty8 ......... SUCCESS [1:02:47.374s]
> [INFO] Geronimo Assemblies :: Minimal + Tomcat7 .......... SUCCESS [8:08.838s]
> [INFO] Geronimo Assemblies :: Java EE 6 Web Profile + Tomcat7  SUCCESS
> [1:59:16.619s]
> [INFO] Geronimo Assemblies :: Java EE 6 + Tomcat7 ........ SUCCESS [1:08:13.854s]
> 
> That is ~1.5 hours for the "Java EE 6 Web Profile + Jetty8" and "Java EE 6 Web
> Profile + Tomcat7" assemblies.

Here are my build times. This is running without tests. Which would affect the overall build
time. Assembly times would be unaffected -- we don't run any tests when building assemblies.

[INFO] Geronimo Assemblies ............................... SUCCESS [0.163s]
[INFO] Geronimo Assemblies :: Minimal + Jetty8 ........... SUCCESS [42.253s]
[INFO] Geronimo Assemblies :: Java EE 6 Web Profile + Jetty8  SUCCESS [1:04.295s]
[INFO] Geronimo Assemblies :: Java EE 6 + Jetty8 ......... SUCCESS [1:33.097s]
[INFO] Geronimo Assemblies :: Minimal + Tomcat7 .......... SUCCESS [41.587s]
[INFO] Geronimo Assemblies :: Java EE 6 Web Profile + Tomcat7  SUCCESS [1:14.409s]
[INFO] Geronimo Assemblies :: Java EE 6 + Tomcat7 ........ SUCCESS [1:46.333s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 28:15.199s
[INFO] Finished at: Wed Jun 22 12:38:10 EDT 2011
[INFO] Final Memory: 633M/1011M
[INFO] ------------------------------------------------------------------------


> 
> 
> For the plugins, I would assume the same, except that I notice that, of course,
> the latest snapshot dependencies are being downloaded, and the poms in the maven
> SNAPSHOT repositories are continually being accessed.
> 
> I realize that because trunk (G3) largely depends on 3rd-party artifact
> SNAPSHOTS that these repositories are checked during every build, and new
> SNAPSHOT libraries are always being downloaded (as is the nature of snapshot
> releases).

Correct. A local maven proxy (e.g. Nexus) can reduce the overhead.

> 
> As it is not a simple process for a user to setup a maven repo proxy, and could
> be a deterrent to someone getting started if it were necessary to install a
> proxy, I am looking at other methods and tips that might increase the speed of
> building G3.

I've tried running with a local maven proxy. However, I've found it to be too much trouble
to maintain/configure.

> 
> Is this feasible? Or would we just say, that because of the nature of SNAPSHOT
> builds and their dependencies on 3rd-party SNAPSHOT artifacts, there is no real
> means to improving build speed unless a proxy is utilized?
> If the later is true, I would like to add this note in the wiki's build
> documentation. And also warn those who choose to build without a proxy may
> experience a long build time.

While I'm thinking about it -- what is your network environment? The focus on proxies imply
that your slow build times are predominantly  network issue. Are you certain that is the case?

--kevan
Mime
View raw message