maven-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark Struberg <strub...@yahoo.de>
Subject Re: the BuildContext
Date Sat, 01 Sep 2012 20:21:33 GMT
> Hence we haven't really published anything.
Well, not quite true. The examples I gave were taken from the released project which is used
in maven already.

But you are right, incremental builds is a hard nut and hard to crack for a single person.
The API you defined is a starting point, but I fear it only covers 60% of all cases. There
is one big fish in it. The set of candidates and algorithms is perfect for the compile step
but that's not the whole deal.  And even the compile step is broken and just works in m2eclipse
because
 Eclipse itself does the build upfront as I demonstrated to you 
yesterday.

In fact each plugin has it's own set of input files and creates another set of output files.
These input files might be on the classpath but also might be somewhere else! Each plugin
must determine itself if it should do some work or not. We can just make this easy by supplying
a set of utilities for the detection.

I also thought about introducing a marker @incremental (similar to what we have with @threadsafe)
to warn people that incremental builds will not work. Fact is that we need to fix or at least
verify each single plugin if it behaves well. There are 2 simple rules:

1.) If an input file/data changes, then the plugin must re-create a reliable result

2.) If no input file/data changes, then the plugin must not touch/change any result

> As Anders pointed there is no dependency on m2e. 

Technically not I agree, but I do see no reason to have an com.sonatype API in maven core
which exists just to notify the m2eclipse plugin. And that is the ONLY reason why it exists.
It would be better to have that API in maven (with a simple default impl) and you can exchange
that component via simple plexus/sisu/jsr-330 rules.

> And if we do take code from 
> m2e, which we will because it's the most advance/proven build avoidance code 
> there is, then we would extract it.
As I demonstrated yesterday on IRC even m2e fails to detect this properly. This just works
because the Eclipse JDT triggers a recompile upfront. If you turn out this feature in Eclipse
(disable 'Project-> Build Automatically') then you will see the bugs noted in the Wiki.
Of course m2e has very valuable knowledge around this area, but it doesn't atm have a solution
which works 100%.



LieGrue,
strub



----- Original Message -----
> From: Jason van Zyl <jason@tesla.io>
> To: Maven Developers List <dev@maven.apache.org>
> Cc: 
> Sent: Saturday, September 1, 2012 2:40 PM
> Subject: Re: the BuildContext
> 
> 
> On Sep 1, 2012, at 6:21 AM, Mark Struberg wrote:
> 
>>      Hi!
>> 
>> 
>>  While doing further research on the incremental build I came accross the 
>>  org.sonatype.plexus.build.incremental.BuildContext.
>> 
>>  There are a few things around this class which caught my interrest.
>> 
>>  1.) This class has a package org.sonatype, the artifact name groupId 
> org.codehaus.plexus and the project in git is sisu-build-api. 
>>  Quite  confusing...  
> 
> Hence we haven't really published anything. As you will discover incremental 
> builds, or build avoidance as we've been calling it, when taking into 
> consideration all tools and contexts is hard. The most advanced form of build 
> avoidance exists in m2e has much of the implementation bits required. But 
> obviously we don't want any tools depending on m2e. Like Maven depending on 
> m2e isn't acceptable. The latest of our work is here:
> 
> https://github.com/etesla/tesla-build-avoidance
> 
> It works but it's not complete, the API is not where we want it and there 
> are a couple branches there too. We have proven we can make plugins just work in 
> m2e which is a primary goal. The current changes that Igor has experimented with 
> in the core are here:
> 
> https://github.com/etesla/maven-3/tree/build-context-injection
> 
> And the m2e work is here:
> 
> https://github.com/etesla/m2e-core
> 
> But he doesn't particularly like it. So we don't even like all the ideas 
> or implementations but it's been there for a long time. Initially started by 
> Benjamin, Igor picked up the work and now Igor and I are looking at it again in 
> the context of continuous delivery. But it is confusing because we're still 
> not entirely clear on how we would like it to work but currently we feel using 
> the concept of something like an Eclipse workspace. This idea can work in many 
> tools. I'm not only interested in Maven. 
> 
> I also don't want to go puttering around in the core because I believe that 
> with some very tiny changes experiments can be done in a simple say. One, maybe 
> two injection sites and a custom scope and anything is possible.
> 
>> 
>>  2.) This replaced our internal scanning. What wonders me a bit is that the 
> DefaultBuildContext always returns true for hasDelta(). Not sure about the 
> original impl, but thiw might need to get changed if we like to support 
> incremental builds. The problematic code can be found in 
> DefaultMavenResourcesFiltering. Not sure if this got used somewhere else as 
> well.
>> 
>>  3.) Having the BuildContext API in a m2eclipse related package is not 
> exactly wise imo. This makes m2eclipse depending on maven and maven depending on 
> m2eclipse. 
> 
> As Anders pointed there is no dependency on m2e. And if we do take code from 
> m2e, which we will because it's the most advance/proven build avoidance code 
> there is, then we would extract it.
> 
>> 
>> 
>>  Wdyt? How to proceed?
> 
> I suggest just looking around the code. Once the JSR-330 branch is merged then 
> it might be easier to play around with. I have no doubt it can be implemented 
> because it has been effectively done in m2e for years. But making it accessible, 
> and making an API that will stand the test of time is what I'm interested 
> in, and I'm also keen on trying to make this available to other systems. By 
> this I don't mean Tesla but other tools like SBT, Gradle, Leiningen, Hudson, 
> and Jenkins because they all have the similar requirements for build avoidance.
> 
>>   
>> 
>> 
>>  LieGrue,
>>  strub
>> 
>> 
>>  ---------------------------------------------------------------------
>>  To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
>>  For additional commands, e-mail: dev-help@maven.apache.org
>> 
> 
> Thanks,
> 
> Jason
> 
> ----------------------------------------------------------
> Jason van Zyl
> Founder & CTO, Sonatype
> Founder,  Apache Maven
> http://twitter.com/jvanzyl
> ---------------------------------------------------------
> 
> I never make the mistake of arguing with people for whose opinions I have no 
> respect.
> 
> -- Edward Gibbon
> 

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org


Mime
View raw message