beam-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Romain Manni-Bucau <rmannibu...@gmail.com>
Subject Re: Gradle migration fixit: April 3
Date Wed, 04 Apr 2018 15:35:37 GMT
Sounds good.

I have less than expected time these days but happy to help doing the
artifact diff when the build will have moved enough, just shout here when
needed ;)

Le 4 avr. 2018 17:33, "Reuven Lax" <relax@google.com> a écrit :

> Yesterday we made progress migrating Python to Gradle. We had PRs (some
> still pending) from people around the world (from the US to France to
> Poland) helping out.
>
> Unfortunately many of the participants yesterday were unfamiliar with any
> of the infrastructure, and needed to be taught how Jenkins, Maven, and
> Gradle worked. As a result, the people who did have expertise spent much of
> the day assisting and teaching instead of migrating, and so we got less
> than expected done.
>
> I 100% agree that we don't want to stall again for months. We here at
> Google plan to continue working on this fixit with a goal of finishing (we
> will be postponing other work we had planned in order to get this done). I
> would love to continue to see the entire community continue to help out
> here. Yesterday was slower than expected due to the learning curve ofr
> participants, but we should pick up the pace.
>
> Reuven
>
> On Wed, Apr 4, 2018 at 5:52 AM Romain Manni-Bucau <rmannibucau@gmail.com>
> wrote:
>
>> Hi guys,
>>
>> Anyone has global a view of where we are? When I left yesterday we were
>> far to be consummable by maven users and my diff was not that useful yet
>> but saw JB was on it.
>> Do we make any progress on that in the night?
>>
>> Concrete question is: when can we drop maven as the primarly build tool
>> for beam (assuming we don't have that dataflow blocker which is a detail
>> for the project)? What I don't want is we start again for 2-3 months of a 2
>> build tools cohexistance state. Did we reach that?
>>
>>
>> Technical side question (unrelated to the "conclusion"): do you also have
>> sym links issue with gradle? If i dont gradle clean maven doesnt build
>> anymore cause of it (ending up in OOME cause it loops on sym links).
>>
>>
>> Romain Manni-Bucau
>> @rmannibucau <https://twitter.com/rmannibucau> |  Blog
>> <https://rmannibucau.metawerx.net/> | Old Blog
>> <http://rmannibucau.wordpress.com> | Github
>> <https://github.com/rmannibucau> | LinkedIn
>> <https://www.linkedin.com/in/rmannibucau> | Book
>> <https://www.packtpub.com/application-development/java-ee-8-high-performance>
>>
>> 2018-04-03 18:45 GMT+02:00 Jean-Baptiste Onofré <jb@nanthrax.net>:
>>
>>> Indeed. +1
>>>
>>> More we are digging around gradle, more we have stuff to do to be
>>> equivalent to
>>> Maven.
>>>
>>> I'm also clearly doing a build time comparison to see if it's worth the
>>> effort
>>> to actually migrate. Currently, on my box, gradle doesn't seem super fast
>>> compared to Maven. Let me dig here.
>>>
>>> Regards
>>> JB
>>>
>>> On 04/03/2018 06:43 PM, Alexey Romanenko wrote:
>>> > In BEAM-3249 <https://issues.apache.org/jira/browse/BEAM-3249> there
>>> are several
>>> > subtasks related to documentation and source code update in favour of
>>> gradle
>>> > - BEAM-3520 <https://issues.apache.org/jira/browse/BEAM-3520>, BEAM-
>>> 3521
>>> > <https://issues.apache.org/jira/browse/BEAM-3521>, BEAM-3522
>>> > <https://issues.apache.org/jira/browse/BEAM-3522> and BEAM-3939
>>> > <https://issues.apache.org/jira/browse/BEAM-3939> . Mostly, it's
>>> config snippets
>>> > and CLI commands.
>>> >
>>> > It seems that we need to do these changes in the end of the all
>>> process of
>>> > migration since there are many things that are not well defined for
>>> now and how
>>> > they will work with gradle. In the same time, the website and other
>>> > documentation should be updated with gradle-related stuff along with
>>> complete
>>> > switch to gradle in building process to avoid a confusion for users.
>>> >
>>> > What do you think?
>>> >
>>> > WBR,
>>> > Alexey
>>> >
>>> >> On 3 Apr 2018, at 18:35, Jean-Baptiste Onofré <jb@nanthrax.net
>>> >> <mailto:jb@nanthrax.net>> wrote:
>>> >>
>>> >> Good idea, isolated in a dedicated file will be easy to switch off.
>>> >>
>>> >> Regards
>>> >> JB
>>> >>
>>> >> On 04/03/2018 06:32 PM, Romain Manni-Bucau wrote:
>>> >>> +1
>>> >>>
>>> >>> also ensuring each optional task has a switch off flag will be
>>> important
>>> >>> (currently i cant use my computer while building with gradle which
>>> is a big lost
>>> >>> of time in a day)
>>> >>>
>>> >>>
>>> >>> Romain Manni-Bucau
>>> >>> @rmannibucau <https://twitter.com/rmannibucau> |  Blog
>>> >>> <https://rmannibucau.metawerx.net/> | Old Blog
>>> >>> <http://rmannibucau.wordpress.com> | Github <https://github.com/
>>> rmannibucau> |
>>> >>> LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book
>>> >>> <https://www.packtpub.com/application-development/java-
>>> ee-8-high-performance>
>>> >>>
>>> >>> 2018-04-03 18:27 GMT+02:00 Jean-Baptiste Onofré <jb@nanthrax.net
>>> >>> <mailto:jb@nanthrax.net>
>>> >>> <mailto:jb@nanthrax.net>>:
>>> >>>
>>> >>>    Hi guys,
>>> >>>
>>> >>>    "Europe located guys" started to work on test and changes on
>>> Gradle.
>>> >>>
>>> >>>    I would like to propose a little change in the gradle structure.
>>> >>>
>>> >>>    Today, lot of tasks and plugins definition are describe in an
>>> unique file.
>>> >>> It's
>>> >>>    pretty hard to find quickly what we are looking for.
>>> >>>
>>> >>>    What do you think to describe each "high level" tasks in a
>>> dedicated file
>>> >>>    located in the gradle folder.
>>> >>>
>>> >>>    To illustrate this, I created the following PR:
>>> >>>
>>> >>>    https://github.com/apache/beam/pull/5004
>>> >>>    <https://github.com/apache/beam/pull/5004>
>>> >>>
>>> >>>    It's about publishing the artifacts on Nexus repositories
>>> (snapshot or
>>> >>> staging).
>>> >>>    The PR is not yet fully ready but you can see a publish.gradle
>>> dedicated
>>> >>> to this
>>> >>>    in the gradle folder:
>>> >>>
>>> >>>    https://github.com/apache/beam/pull/5004/files#diff-
>>> 2634489a13e9ba4c3db8f067716556a4
>>> >>>    <https://github.com/apache/beam/pull/5004/files#diff-
>>> 2634489a13e9ba4c3db8f067716556a4>
>>> >>>
>>> >>>    We can imagine to have:
>>> >>>
>>> >>>    - gradle/jacoco.gradle for code coverage
>>> >>>    - gradle/rat.gradle for RAT
>>> >>>    - gradle/dependencies.gradle for dependencies resolution
>>> >>>    ...
>>> >>>
>>> >>>    Thoughts ?
>>> >>>
>>> >>>    Regards
>>> >>>    JB
>>> >>>
>>> >>>    On 03/29/2018 04:46 AM, Reuven Lax wrote:
>>> >>>> Hi all,
>>> >>>>
>>> >>>> Last week we discussed having a "fixit" day for Gradle, and
I
>>> volunteered to
>>> >>>> organize it. A number of people volunteered to help, from multiple
>>> organization.
>>> >>>> I'd like to say that it's great to see such a diverse set of
people
>>> volunteering
>>> >>>> to help here - this is a great way to build community! Everyone
who
>>> explicitly
>>> >>>> volunteered is directly cced on this email, though we'd love
for
>>> more of the
>>> >>>> community to help.
>>> >>>>
>>> >>>> The agreed upon date is April 3. The top-level JIRA tracking
this
>>> work is
>>> >>>>
>>> >>>> ttps://issues.apache.org/jira/browse/BEAM-3249
>>> >>>    <http://issues.apache.org/jira/browse/BEAM-3249>
>>> >>>> <https://issues.apache.org/jira/browse/BEAM-3249
>>> >>>    <https://issues.apache.org/jira/browse/BEAM-3249>>,
and we
>>> currently have 26
>>> >>>> subtasks linked to it. I've created a Kanban board to track
these
>>> issues,
>>> >>>    which
>>> >>>> I'll share out soon. We will use Slack the day of the fixit
for
>>> collaboration
>>> >>>> and for questions.
>>> >>>>
>>> >>>>
>>> >>>> Two major goals for this fixit should be to 1. Remove Maven
runs
>>> from our
>>> >>>> Jenkins executors and 2. to migrate our release process fully
over
>>> to
>>> >>>    Gradle. A
>>> >>>> lot of work has already been done on 1., and we've made some
>>> progress on 2..
>>> >>>> Slightly longer-term the goal is to delete all of the pom files;
>>> I'm not sure
>>> >>>> we'll get as far as completely deleting Maven in one day, but
we
>>> should get
>>> >>>> within striking distance!
>>> >>>>
>>> >>>>
>>> >>>> Thanks in advance to everyone who's helping out!
>>> >>>>
>>> >>>>
>>> >>>> Reuven
>>> >>>>
>>> >>>
>>> >>>    --
>>> >>>    Jean-Baptiste Onofré
>>> >>>    jbonofre@apache.org <mailto:jbonofre@apache.org> <mailto:
>>> jbonofre@apache.org>
>>> >>>    http://blog.nanthrax.net
>>> >>>    Talend - http://www.talend.com
>>> >>>
>>> >>>
>>> >>
>>> >> --
>>> >> Jean-Baptiste Onofré
>>> >> jbonofre@apache.org <mailto:jbonofre@apache.org>
>>> >> http://blog.nanthrax.net
>>> >> Talend - http://www.talend.com
>>> >
>>>
>>> --
>>> Jean-Baptiste Onofré
>>> jbonofre@apache.org
>>> http://blog.nanthrax.net
>>> Talend - http://www.talend.com
>>>
>>
>>

Mime
View raw message