geronimo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Russell E Glaue <rgl...@cait.org>
Subject Re: Speeding up G3.0 build time
Date Wed, 22 Jun 2011 17:27:34 GMT
On 06/22/2011 12:02 PM, Kevan Miller wrote:
> 
> On Jun 22, 2011, at 12:09 PM, Russell E Glaue wrote:
> 
>> I am wanting to update the G3 dev wiki to help first-time builders get started.
>> How would you suggest a build of G3 could become faster?
>>
>> The only tip I am aware of for speeding up trunk (G3) builds is to install a
>> maven repository proxy like nexus. This is eluded to in the wiki already.
>> Are there any other tips?
> 
> You can build without tests. E.g.: 'mvn clean install -DskipTests=true'. If I'm simply
running a build, I'll frequently turn off tests ('-DskipTests=true' I think '-DskipTests'
is equivalent). If I'm making code changes, I run with tests.
> 
> You can also build in an offline mode (e.g. 'mvn -o clean install -DskipTests'). This
avoids searching of maven repos, all together. So, you do need one full online build, before
this will work. Due to the current development work going on in OpenWebBeans, OpenEJB, Aries,
and Geronimo (I think these are the most frequently changing projects, ATM), you run the risk
of getting out-of-date between the projects.

Ah, yes very good tip.
And the note on the risk would have to be included if the tip were given.

> 
> FYI, we are running Continuous Integration builds. E.g. -- http://ci.apache.org/builders/geronimo-server-trunk/builds/106

> 
> I encourage people to build from source. However, if you really want pre-built binaries
of unreleased versions of Geronimo, assemblies from our CI builds are uploaded to the apache
snapshot repository. So, it is possible to download SNAPSHOT builds from the SNAPSHOT repository.
For example: https://repository.apache.org/content/repositories/snapshots/org/apache/geronimo/assemblies/geronimo-tomcat7-javaee6/3.0-SNAPSHOT/

Yes, I actually use these for testing use scenarios because they are always up
to date with trunk.
Building from source is for when testing code changes.

I think this should definitely be mentioned on the "Building Apache Geronimo"
page: "If you just want a build of the latest trunk, use can download from the
CI builds...."

> 
>>
>>
>> I have been able to successfully build G3 from trunk several times now but with
>> differing build time results.
>>
>> The best build time I have had is 13 hours.
>> The worst build time I have had is 39 hours. (this is from my latest build)
> 
> Ouch. My build times are nowhere near this long. Are you running with an empty local
maven repository?

No, I have built G3.0 from trunk about a dozen times since the last time I
emptied my local maven repository. Believe it or not, the 13 hours build time
was starting from an empty local maven repository. So I was very surprised my
latest build finished this morning after 39 hours... the worst yet.

> 
>>
>> I am using a very basic build procedure.
>> 1. checkout G3 trunk
>> 2. JAVA_HOME="/path/to/jdk/1.6.0_22"
>> 3. download and install a fresh maven 2.2.1
>> 4. MAVEN_OPTS="-Xmx968m -XX:MaxPermSize=500m -XX:ReservedCodeCacheSize=64m"
> 
> My settings are similar. I don't specify '-XX:ReservedCodeCacheSize=64m'.
> 
> I run with the following. I know there are others running with larger max heaps. I haven't
:
> 
> MAVEN_OPTS="-Xmx1024m -XX:MaxPermSize=512m -XX:+HeapDumpOnOutOfMemoryError"

The server I am build on has 1024MB of memory total, and has a dual dual-core
Intel(R) XEON(TM) CPU 2.00GHz with 512KB cache.

I have tried many JVM configurations, and the one like mine and yours is the
best one I have found. For me, a JVM configuration with less than 400m for
MaxPermSize always ended with an OutOfMemory error on PermGen space.

The Geronimo dev wiki suggests 200m for MaxPermSize, which never worked for me
for trunk. I always assumed it was because of all the snapshot downloading the
build process has to do. Even after the 2nd or 3rd build, 200m was never enough.

> 
>> 5. mvn clean install
>> 6. 13 to 39 hours later == BUILD SUCCESS
>>
>>
>> From observing the build process, it would seem the most used (seemingly paused)
>> time is right before the build process moves on to the next plugin or assembly.
>>
>> This "seemingly paused time" after the assembly I assume is the packaging
>> process, because the resulting output is a huge collection.
>>
>> [INFO] Geronimo Assemblies ............................... SUCCESS [3.995s]
>> [INFO] Geronimo Assemblies :: Minimal + Jetty8 ........... SUCCESS [8:15.210s]
>> [INFO] Geronimo Assemblies :: Java EE 6 Web Profile + Jetty8  SUCCESS [1:57:17.311s]
>> [INFO] Geronimo Assemblies :: Java EE 6 + Jetty8 ......... SUCCESS [1:02:47.374s]
>> [INFO] Geronimo Assemblies :: Minimal + Tomcat7 .......... SUCCESS [8:08.838s]
>> [INFO] Geronimo Assemblies :: Java EE 6 Web Profile + Tomcat7  SUCCESS
>> [1:59:16.619s]
>> [INFO] Geronimo Assemblies :: Java EE 6 + Tomcat7 ........ SUCCESS [1:08:13.854s]
>>
>> That is ~1.5 hours for the "Java EE 6 Web Profile + Jetty8" and "Java EE 6 Web
>> Profile + Tomcat7" assemblies.
> 
> Here are my build times. This is running without tests. Which would affect the overall
build time. Assembly times would be unaffected -- we don't run any tests when building assemblies.
> 
> [INFO] Geronimo Assemblies ............................... SUCCESS [0.163s]
> [INFO] Geronimo Assemblies :: Minimal + Jetty8 ........... SUCCESS [42.253s]
> [INFO] Geronimo Assemblies :: Java EE 6 Web Profile + Jetty8  SUCCESS [1:04.295s]
> [INFO] Geronimo Assemblies :: Java EE 6 + Jetty8 ......... SUCCESS [1:33.097s]
> [INFO] Geronimo Assemblies :: Minimal + Tomcat7 .......... SUCCESS [41.587s]
> [INFO] Geronimo Assemblies :: Java EE 6 Web Profile + Tomcat7  SUCCESS [1:14.409s]
> [INFO] Geronimo Assemblies :: Java EE 6 + Tomcat7 ........ SUCCESS [1:46.333s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 28:15.199s
> [INFO] Finished at: Wed Jun 22 12:38:10 EDT 2011
> [INFO] Final Memory: 633M/1011M
> [INFO] ------------------------------------------------------------------------

Yours is about 1/3 faster than mine. But you might have a faster processor too.
Obviously, this part of the build is not the time consumer for me. It is nice to
know this part of the build is supposed to take several hours - that helps me
narrow down the issues.

> 
> 
>>
>>
>> For the plugins, I would assume the same, except that I notice that, of course,
>> the latest snapshot dependencies are being downloaded, and the poms in the maven
>> SNAPSHOT repositories are continually being accessed.
>>
>> I realize that because trunk (G3) largely depends on 3rd-party artifact
>> SNAPSHOTS that these repositories are checked during every build, and new
>> SNAPSHOT libraries are always being downloaded (as is the nature of snapshot
>> releases).
> 
> Correct. A local maven proxy (e.g. Nexus) can reduce the overhead.

So it is worth noting as it already is now.

> 
>>
>> As it is not a simple process for a user to setup a maven repo proxy, and could
>> be a deterrent to someone getting started if it were necessary to install a
>> proxy, I am looking at other methods and tips that might increase the speed of
>> building G3.
> 
> I've tried running with a local maven proxy. However, I've found it to be too much trouble
to maintain/configure.

Agreed. At least it is too much trouble if you are not continuously building the
entire trunk every day on the larger scale. So too much to ask a G3 beginner to do.

> 
>>
>> Is this feasible? Or would we just say, that because of the nature of SNAPSHOT
>> builds and their dependencies on 3rd-party SNAPSHOT artifacts, there is no real
>> means to improving build speed unless a proxy is utilized?
>> If the later is true, I would like to add this note in the wiki's build
>> documentation. And also warn those who choose to build without a proxy may
>> experience a long build time.
> 
> While I'm thinking about it -- what is your network environment? The focus on proxies
imply that your slow build times are predominantly  network issue. Are you certain that is
the case?

We have about ~14Mb/s speed to the Internet. Our backplane is Gigabit Ethernet.
We are connected to a major Internet backbone provider.
I have been wondering if the problem is responsiveness of the maven repo sites.
I have been seeing on occasion a refusal to download an artifact from the site,
where as I was able to download it within a browser from the same URL.
So I wonder if the repo sites are either metering the usage on a per user or
session basis.

> 
> --kevan

Thanks for the excellent tips!
I'll try some things out, and look at how to reflect the tips in the wiki
without degrading any affirmation of Best Practices (i.e. running the test cases
is good).

-RG


Mime
View raw message