maven-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Russell Gold <r...@gold-family.us>
Subject Re: Any reason ant should compile faster?
Date Thu, 25 Jul 2013 02:28:45 GMT

On Jul 24, 2013, at 8:36 PM, Ron Wheeler <rwheeler@artifact-software.com> wrote:

> 
> On 24/07/2013 4:40 PM, Russell Gold wrote:
>> Oh, you underestimate the coupling of this code. Yes, there are packages, but the
classes in those packages reference one another willy-nilly. It's not a problem with recompiling
- it's that there don't appear to be any good, sensible subset of classes in this group of
10,000 or so which can be compiled on their own. This set of 4000 appears to be the minimum
of what's left.
>> 
>> And worse, it appears that we published the main classes responsible for doing the
coupling - and our customers use them in their own scripts.
>> 
> This does not mean  that they require recompiling  in a single Maven project.

What am I missing? If I compile the classes I want, I get the others because of compile dependencies.
If I compile some of the others instead, I still get them all. How do I not compile them in
a single maven project?

Essentially, there is a backbone of classes, all of which refer to a certain few classes -
and those few classes refer to all the rest.

The comment about customers using them was just an explanation of why we can't just refactor
out the dependencies. The public classes are the ones that contain the dependencies.

> 
>> The architect responsible for this mess left the company ten years ago, shortly after
putting it in place over the objections of most of the senior staff, and we haven't been able
to remove it.
>> 
> You could start by breaking the whole mess into packages that live in a single project.
> If you  get down to 80-100 Maven projects, you will at least have a start on a cleanup
and a way to reduce the need to compile 4000 classes at once.

We do use multiple projects. This blob (which is actually what we call it) represents about
half the system, and a third of those participate in the tight coupling. Beyond that, we have
about 100 separate project that are currently in ant, and are due to convert.

> 
> Our biggest application was made up of 90 Maven projects but most major revisions only
required changes to 10-20 Maven projects and we edited, compiled and tested each project individually
even if it depended on other jars.
> We had designed it from the start as a layered stack so our initial packaging into Maven
projects made sense but we did many refactoring over the years as we went through major revisions
and changed our views about how to structure the application.
> 
> Maven is designed to support multi-project applications.
> 
> Ron
>> Russ
>> 
>> On Jul 24, 2013, at 3:28 PM, Ron Wheeler <rwheeler@artifact-software.com <mailto:rwheeler@artifact-software.com>>
wrote:
>> 
>>> On 24/07/2013 12:35 PM, Russell Gold wrote:
>>>> I wish I could break it up. Among the problems with this code is the extreme
bit of coupling. The scripts list 300 files to compile because that's all that we would have
wanted, but they pull in other files, which pull in yet others. Trying to compile this fairly
small subsystem winds up building about a third of the entire project.
>>> Pulling in classes does not mean recompiling them.
>>> Once you have compiled the dependencies and aggregate them into jars, they can
be referenced without recompiling them.
>>> This will speed things up tremendously.
>>> 
>>> Do you at least have packages to use as a starting point?
>>> 
>>> Ron
>>>> 
>>>> But I will definitely look at the options and see what I can learn from them;
thanks for the suggestions.
>>>> 
>>>> Russ
>>>> 
>>>> On Jul 24, 2013, at 12:31 PM, Wayne Fay <waynefay@gmail.com <mailto:waynefay@gmail.com>>
wrote:
>>>> 
>>>>>> The case I'm wondering about now is one where we have over 10,000
>>>>>> source files in a directory, and compile them bit-by-bit. One step
specifies
>>>>>> about 300 files, and thanks to javac dependency logic, actually compiles
>>>>>> about 4000 files in ant.
>>>>> Is there any way you can break that into 10 projects with 1k files
>>>>> each, or 20 projects with 500 files each, etc? I have found dramatic
>>>>> improvements in (Java compiling) speed as a result of breaking huge
>>>>> projects like this into a series of much smaller projects with
>>>>> dependencies between the projects etc as needed.
>>>>> 
>>>>> Having said that, 20 seconds for 4k files sounds pretty quick! Sorry
>>>>> but I'm not sure how to explain the 20 sec vs 90 sec results you're
>>>>> experiencing. I'd check versions of everything and flags sent to
>>>>> compilers etc via -X.
>>>>> 
>>>>> Wayne
>>>>> 
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org <mailto:users-unsubscribe@maven.apache.org>
>>>>> For additional commands, e-mail: users-help@maven.apache.org <mailto:users-help@maven.apache.org>
>>>>> 
>>>> -----------------
>>>> Come read my webnovel, Take a Lemon <http://www.takealemon.com>,
>>>> and listen to the Misfile radio play <http://www.gold-family.us/audio/misfile.html>!
>>>> 
>>>> 
>>>> 
>>>> 
>>>> 
>>> 
>>> 
>>> -- 
>>> Ron Wheeler
>>> President
>>> Artifact Software Inc
>>> email: rwheeler@artifact-software.com <mailto:rwheeler@artifact-software.com>
>>> skype: ronaldmwheeler
>>> phone: 866-970-2435, ext 102
>>> 
>>> 
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org <mailto:users-unsubscribe@maven.apache.org>
>>> For additional commands, e-mail: users-help@maven.apache.org <mailto:users-help@maven.apache.org>
>>> 
>> 
>> -----------------
>> Come read my webnovel,/Take a Lemon/<http://www.takealemon.com>,
>> and listen to the Misfile radio play <http://www.gold-family.us/audio/misfile.html>!
>> 
>> 
>> 
>> 
> 
> 
> -- 
> Ron Wheeler
> President
> Artifact Software Inc
> email: rwheeler@artifact-software.com
> skype: ronaldmwheeler
> phone: 866-970-2435, ext 102
> 

-----------------
Come read my webnovel, Take a Lemon <http://www.takealemon.com>, 
and listen to the Misfile radio play <http://www.gold-family.us/audio/misfile.html>!





Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message