maven-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alexandre Porcelli Bakos <aba...@redhat.com>
Subject Re: [Proposal] When Old Meets New: Turning Maven into a High Scalable, Resource Efficient, Cloud Ready Microservice
Date Fri, 02 Nov 2018 13:45:17 GMT
Hi Robert,

 I'm Alex Porcelli and I work with Max. I can also create some jiras
about things that we couldn't do.

 Here's an example (likely my first JIRA):
 In our use case, all sources are stored in bare Git repositories and
we manipulate it using NIO2 API [1], if Maven accepted NIO2 Path
interface for example, instead of File, we didn't need to create a
temp folder to build. This would also require split the input
parameter to build, one for sources (that likely would just use
directly Git) and another Path for the target folder.

 This is one example out of others that we found during our journey.

Regards,
---
Alexandre Porcelli
Principal Software Engineer
Red Hat Business Systems and Intelligence Group
On Fri, Nov 2, 2018 at 5:57 AM Robert Scholte <rfscholte@apache.org> wrote:
>
> On Wed, 08 Aug 2018 09:49:17 +0200, Massimiliano Dessì
> <massimiliano.dessi@gmail.com> wrote:
>
> > Hi Robert
> >
> > 2018-08-07 18:35 GMT+02:00 Robert Scholte <rfscholte@apache.org>:
> >
> >> Hi Max,
> >>
> >> regarding the contribution, I would like to propose the following:
> >> - At least create a JIRA issue for every feature.
>
> Can we start creating these JIRA issues? I'd like to work on a list of
> features we should embed in Maven 4 and some of these are worth adding.
> Would be nice to keep you as the reporter and that you can refer to the
> codechanges on your fork.
> All to make it easier to implement it in Maven Core.
>
> thanks,
> Robert
>
> >> - Due to the large amount of code I can imagine there are some
> >> dependencies between the features, i.e some features should be
> >> implemented
> >> first.
> >> - Based on that we can decide what we want to adopt, prioritize/order
> >> and
> >> start working on the implementations.
> >> - I assume CLA[1] is not an issue ;)
> >>
> >
> > +1
> >
> >
> >>
> >> Do you have statistics of the memory consumption?
> >>
> >
> > Nope,
> >
> >>
> >> It looks to me we need to introduce some sort of profiles, so people can
> >> choose the preferred setup.
> >> I see both the memory and concurrency as solutions that come with
> >> "costs"
> >> not everybody wants to pay.
> >> Did you solve this with a specific builder?
> >>
> >
> > I've created the entire "compiler" to use with only one instance or
> > multiple, but now if multiple running in the same time they working
> > correctly,
> > the two major problems arised during the concurrent builds was the random
> > build failed to changed order of the parameters and the output
> > interleaved
> > in the same output.
> >
> >
> >> Also, did you have to break APIs or are all current plugins and
> >> extensions
> >> still usable?
> >>
> >
> > No API breaks, two aspects was mandatory, easy (or at least with few
> > changes) update of Maven version
> > and the same behaviour compared to the MavenCLI,
> > for this reason the changes on Maven API are limited to few classes of
> > the
> > Maven embedder, some was suggested by the debugging experience,
> > everytime the major part of the time was spent on the creation of the
> > Plexus/Sisu Ioc Container and this suggest to reuse the Container and
> > some
> > internal objects if the pom isn't changed between builds,
> > because in our workbench the editing is on the Java files and on drools
> > rules mostly,
> > in this way after the first build, when the maven infrastructure is
> > created
> > and takari build all the buildable stuffs,
> > the others following builds are fast, the Maven infrastructure used by
> > the
> > embedder is ready and only the changes classes needs to be recompiled,
> > in our workbench we cache the compiler created in this way with the
> > project
> > associated, and this association is alive for the entire working session,
> > a step furter could be the changes of Maven internal but this mean breaks
> > the Maven API.
> > Then when we have writed some concurrent build test to see if the Maven
> > runtime was happy we have discovered some problems and fixed,
> > specific slf4j in memory appenders are used to write the specific build
> > output, since the embedder redirects the entire System.out to a single
> > file.
> >
> > Here the Mojo used to share the drools runtime objects with the client
> > caller, it's the culprit of all these stuffs :)
> > https://github.com/kiegroup/droolsjbpm-integration/blob/master/kie-maven-plugin/src/main/java/org/kie/maven/plugin/BuildMojo.java#L183
> >
> >
> >
> >>
> >> thanks,
> >> Robert
> >>
> >> [1] https://www.apache.org/dev/new-committers-guide
> >
> >
> > Best
> > Max
> >
> >
> >>
> >>
> >>
> >> On Mon, 06 Aug 2018 11:16:36 +0200, Massimiliano Dessì <
> >> massimiliano.dessi@gmail.com> wrote:
> >>
> >> Hi all,
> >>> as a part of my daily job in Red Hat
> >>> I've worked on a "customization" of Maven for our Kie Workbench used
> >>> with
> >>> Drools, JBPM and Optaplanner.
> >>> The starting point was exports Objects created by Drools inside our
> >>> Maven
> >>> plugin, but the features are growed a lot.
> >>>
> >>> A brief and not exhaustive list of the features:
> >>> -From static file producer to in memory producer and exporter
> >>> -Reusable internal components to optimize memory footprint and time of
> >>> execution
> >>> -Maven like a builder Daemon
> >>> -Configurable behaviour using a pipeline of decorators
> >>> -Stateless (when possible)
> >>> -Concurrent builds and concurrent logs
> >>> -Cloud enabled
> >>> -Local and remote executions
> >>> -Plugins turned from FileSystem based to in memory based
> >>> -Async API to consume build result
> >>> -Incremental builds
> >>>
> >>> The current version is based on the Maven 3.3.9 but could be easily
> >>> updated
> >>> and is in our plans in the next months.
> >>> Currently we have four modules plus other testing module and the
> >>> offprocess
> >>> module is under development.
> >>>
> >>> core
> >>> service
> >>> maven-plugins (plugins turned from FS to in memory)
> >>> distribution
> >>>
> >>> Preso with details:
> >>> https://www.slideshare.net/desmax74/when-old-meets-new-turni
> >>> ng-maven-into-a-high-scalable-resource-efficient-cloud-ready-microservice
> >>>
> >>> I'd like to contribute this code, but since isn't a simple patch I'd
> >>> like
> >>> discuss about how to contribute,
> >>> code plus tests is around 18k loc.
> >>>
> >>> Best
> >>> Max
> >>>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> >> For additional commands, e-mail: dev-help@maven.apache.org
> >>
> >>
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org


Mime
View raw message