geronimo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kevan Miller <kevan.mil...@gmail.com>
Subject Re: Speeding up G3.0 build time
Date Wed, 22 Jun 2011 18:54:13 GMT

On Jun 22, 2011, at 1:27 PM, Russell E Glaue wrote:

> On 06/22/2011 12:02 PM, Kevan Miller wrote:
>> 
>> On Jun 22, 2011, at 12:09 PM, Russell E Glaue wrote:

<snip>

>> 
>>> 
>>> I am using a very basic build procedure.
>>> 1. checkout G3 trunk
>>> 2. JAVA_HOME="/path/to/jdk/1.6.0_22"
>>> 3. download and install a fresh maven 2.2.1
>>> 4. MAVEN_OPTS="-Xmx968m -XX:MaxPermSize=500m -XX:ReservedCodeCacheSize=64m"
>> 
>> My settings are similar. I don't specify '-XX:ReservedCodeCacheSize=64m'.
>> 
>> I run with the following. I know there are others running with larger max heaps.
I haven't :
>> 
>> MAVEN_OPTS="-Xmx1024m -XX:MaxPermSize=512m -XX:+HeapDumpOnOutOfMemoryError"
> 
> The server I am build on has 1024MB of memory total, and has a dual dual-core
> Intel(R) XEON(TM) CPU 2.00GHz with 512KB cache.
> 
> I have tried many JVM configurations, and the one like mine and yours is the
> best one I have found. For me, a JVM configuration with less than 400m for
> MaxPermSize always ended with an OutOfMemory error on PermGen space.
> 
> The Geronimo dev wiki suggests 200m for MaxPermSize, which never worked for me
> for trunk. I always assumed it was because of all the snapshot downloading the
> build process has to do. Even after the 2nd or 3rd build, 200m was never enough.

Right. So, MaxPermSize=200m is definitely out-of-date. There is/may be some bugs that are
increasing this value. However, not really worth our while to pursue at the moment. I don't
think MaxPermSize=500m is too prohibitive. More important things to do, ATM. 

That said -- I'm building on a machine with 8 gigs of real memory. So, 1024MB of real memory
seems very small to me. And I suspect that this is the source of your problem... A max heap
of 1024MB and MaxPermSize of 512m means your system may be spending a lot of time swapping
virtual memory. 

> 
>> 
>>> 5. mvn clean install
>>> 6. 13 to 39 hours later == BUILD SUCCESS
>>> 
>>> 
>>> From observing the build process, it would seem the most used (seemingly paused)
>>> time is right before the build process moves on to the next plugin or assembly.
>>> 
>>> This "seemingly paused time" after the assembly I assume is the packaging
>>> process, because the resulting output is a huge collection.
>>> 
>>> [INFO] Geronimo Assemblies ............................... SUCCESS [3.995s]
>>> [INFO] Geronimo Assemblies :: Minimal + Jetty8 ........... SUCCESS [8:15.210s]
>>> [INFO] Geronimo Assemblies :: Java EE 6 Web Profile + Jetty8  SUCCESS [1:57:17.311s]
>>> [INFO] Geronimo Assemblies :: Java EE 6 + Jetty8 ......... SUCCESS [1:02:47.374s]
>>> [INFO] Geronimo Assemblies :: Minimal + Tomcat7 .......... SUCCESS [8:08.838s]
>>> [INFO] Geronimo Assemblies :: Java EE 6 Web Profile + Tomcat7  SUCCESS
>>> [1:59:16.619s]
>>> [INFO] Geronimo Assemblies :: Java EE 6 + Tomcat7 ........ SUCCESS [1:08:13.854s]
>>> 
>>> That is ~1.5 hours for the "Java EE 6 Web Profile + Jetty8" and "Java EE 6 Web
>>> Profile + Tomcat7" assemblies.
>> 
>> Here are my build times. This is running without tests. Which would affect the overall
build time. Assembly times would be unaffected -- we don't run any tests when building assemblies.
>> 
>> [INFO] Geronimo Assemblies ............................... SUCCESS [0.163s]
>> [INFO] Geronimo Assemblies :: Minimal + Jetty8 ........... SUCCESS [42.253s]
>> [INFO] Geronimo Assemblies :: Java EE 6 Web Profile + Jetty8  SUCCESS [1:04.295s]
>> [INFO] Geronimo Assemblies :: Java EE 6 + Jetty8 ......... SUCCESS [1:33.097s]
>> [INFO] Geronimo Assemblies :: Minimal + Tomcat7 .......... SUCCESS [41.587s]
>> [INFO] Geronimo Assemblies :: Java EE 6 Web Profile + Tomcat7  SUCCESS [1:14.409s]
>> [INFO] Geronimo Assemblies :: Java EE 6 + Tomcat7 ........ SUCCESS [1:46.333s]
>> [INFO] ------------------------------------------------------------------------
>> [INFO] BUILD SUCCESS
>> [INFO] ------------------------------------------------------------------------
>> [INFO] Total time: 28:15.199s
>> [INFO] Finished at: Wed Jun 22 12:38:10 EDT 2011
>> [INFO] Final Memory: 633M/1011M
>> [INFO] ------------------------------------------------------------------------
> 
> Yours is about 1/3 faster than mine.

I think you've missed the metric. My times are an order of magnitude (hours vs minutes) better
than yours. 

>>> [INFO] Geronimo Assemblies :: Java EE 6 + Tomcat7 ........ SUCCESS [1:08:13.854s]

and

>> [INFO] Geronimo Assemblies :: Java EE 6 + Tomcat7 ........ SUCCESS [1:46.333s]

That's 68 minutes vs 1 minute 46 seconds. My *total* build time was 28 minutes.

> But you might have a faster processor too.
> Obviously, this part of the build is not the time consumer for me. It is nice to
> know this part of the build is supposed to take several hours - that helps me
> narrow down the issues.
> 
>> 
>> 
>>> 
>>> 
>>> For the plugins, I would assume the same, except that I notice that, of course,
>>> the latest snapshot dependencies are being downloaded, and the poms in the maven
>>> SNAPSHOT repositories are continually being accessed.
>>> 
>>> I realize that because trunk (G3) largely depends on 3rd-party artifact
>>> SNAPSHOTS that these repositories are checked during every build, and new
>>> SNAPSHOT libraries are always being downloaded (as is the nature of snapshot
>>> releases).
>> 
>> Correct. A local maven proxy (e.g. Nexus) can reduce the overhead.
> 
> So it is worth noting as it already is now.
> 
>> 
>>> 
>>> As it is not a simple process for a user to setup a maven repo proxy, and could
>>> be a deterrent to someone getting started if it were necessary to install a
>>> proxy, I am looking at other methods and tips that might increase the speed of
>>> building G3.
>> 
>> I've tried running with a local maven proxy. However, I've found it to be too much
trouble to maintain/configure.
> 
> Agreed. At least it is too much trouble if you are not continuously building the
> entire trunk every day on the larger scale. So too much to ask a G3 beginner to do.

Too much to ask of a new maven user -- I agree.

> 
>> 
>>> 
>>> Is this feasible? Or would we just say, that because of the nature of SNAPSHOT
>>> builds and their dependencies on 3rd-party SNAPSHOT artifacts, there is no real
>>> means to improving build speed unless a proxy is utilized?
>>> If the later is true, I would like to add this note in the wiki's build
>>> documentation. And also warn those who choose to build without a proxy may
>>> experience a long build time.
>> 
>> While I'm thinking about it -- what is your network environment? The focus on proxies
imply that your slow build times are predominantly  network issue. Are you certain that is
the case?
> 
> We have about ~14Mb/s speed to the Internet. Our backplane is Gigabit Ethernet.
> We are connected to a major Internet backbone provider.
> I have been wondering if the problem is responsiveness of the maven repo sites.
> I have been seeing on occasion a refusal to download an artifact from the site,
> where as I was able to download it within a browser from the same URL.
> So I wonder if the repo sites are either metering the usage on a per user or
> session basis.

I'm building on my home broadband connection. Probably similar speeds. I think we should identify
the source of your extremely slow build time. 

If remote repository lookups are the cause of your problems, 'mvn -o clean install' should
result in a much faster build time. However, I suspect that it won't make much difference
at all...

> 
> Thanks for the excellent tips!
> I'll try some things out, and look at how to reflect the tips in the wiki
> without degrading any affirmation of Best Practices (i.e. running the test cases
> is good).

No problem. Many thanks for digging into this. Definitely an area we tend to overlook...

--kevan


Mime
View raw message