hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Daryn Sharp <da...@yahoo-inc.com>
Subject Re: .23 compile times
Date Wed, 15 Feb 2012 21:02:52 GMT
Hi Sriram,

You may want to try adding your source tree to Spotlight's privacy list, otherwise it's busy
trying to index files as fast as the build is writing them.  If you are (un)fortunate enough
to be running anti-virus sw, then it too is scurrying along behind your build checking every
file.  That's enough to turn a normally silent MBP into a hair dryer.

If you don't want to omit the entire project, because perhaps you want to use mdfind, then
you can rename dirs to end with ".noindex".  The trick/hack is you can symlink the original
dir name to the renamed dir.noindex.

But yes, I think the build is way to slow to be productive when used as intended.  The fact
that people have to use custom tricks to build & test in a "reasonable" amount of time
is a symptom of a problem...

Daryn


On Feb 15, 2012, at 12:47 PM, Todd Lipcon wrote:

> Hi Sriram,
> 
> I also do -Dmaven.skip.javadoc=true usually and that cuts out a couple
> minutes. Unless you need a tarball, you can also drop -Ptar.
> 
> When I'm just making quick changes confined to the java code, I
> usually just use "mvn -Dskiptests install" from within the hdfs
> project, then manually cp the resulting
> target/hadoop-hdfs-0.24.0-SNAPSHOT.jar into my install
> dir/share/hadoop/hdfs. Much faster than the full rebuild.
> 
> -Todd
> 
> On Wed, Feb 15, 2012 at 8:52 AM, Harsh J <harsh@cloudera.com> wrote:
>> Sriram,
>> 
>> My package command didn't take as long with the same command -- where
>> do you see it getting stuck at most of the times? Which particular
>> target (right term?) seems to take the highest time to complete?
>> 
>> On Wed, Feb 15, 2012 at 12:31 AM, Sriram Rao <sriramsrao@gmail.com> wrote:
>>> Folks,
>>> 
>>> The build takes forever:
>>> 
>>>  mvn package -Pdist -DskipTests -Dtar
>>> 
>>> ....
>>> 
>>> [INFO]
>>> ------------------------------------------------------------------------
>>> [INFO] BUILD SUCCESS
>>> [INFO]
>>> ------------------------------------------------------------------------
>>> [INFO] Total time: 6:25.399s
>>> [INFO] Finished at: Tue Feb 14 10:54:18 PST 2012
>>> [INFO] Final Memory: 56M/123M
>>> [INFO]
>>> ------------------------------------------------------------------------
>>> 
>>> This is on a mac with 8GB RAM.  (The above case involved 0-lines of code
>>> change; just making sure I had everything built!).
>>> 
>>> Is there a faster way to get the thing to build?
>>> 
>>> Sriram
>> 
>> 
>> 
>> --
>> Harsh J
>> Customer Ops. Engineer
>> Cloudera | http://tiny.cloudera.com/about
> 
> 
> 
> -- 
> Todd Lipcon
> Software Engineer, Cloudera


Mime
View raw message