lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dawid Weiss <dawid.we...@gmail.com>
Subject Gradle build effort (respin, please read)
Date Mon, 02 Dec 2019 15:07:26 GMT
Hi folks.

For those who prefer the short version:
git clone https://gitbox.apache.org/repos/asf/lucene-solr.git -b gradle-master
gradlew :help
gradlew -p lucene test
gradlew -p solr/packaging assemble

A longer explanation follows.

I realize many of you probably got tired of the discussions and issues
concerning the gradle build. I don't know why this turned so awkward
instead of exciting (which it is) but I wanted to try to reflect on
the lessons learned and retry the effort, perhaps with a better
outcome this time.

Erick and I spent some time trying to revive what Mark did but we
failed -- there were just too many mixed concerns and work-in-progress
changes to reconcile. I wanted something that would be:

- usable from the first minute,
- simple (no custom gradle plugins, etc.),
- divisible into smaller logical chunks so that contributions could be
smaller and incremental; without the need to work on the whole thing
for months,
- backward compatible with ant (because we are forced to use it until
everything is ported).

I took some of the things from Cao and Mark but I essentially rewrote
the build file and split it into logical fragments that configure
different build aspects:

https://github.com/apache/lucene-solr/blob/gradle-master/build.gradle#L18-L48

A good example of the "smaller fragment" approach is a developer aid
to display the slowest tests at the end of the run -- it's
self-contained and gracefully separated from anything else in the
build:

https://github.com/apache/lucene-solr/blob/gradle-master/gradle/testing/slowest-tests-at-end.gradle

It was, in fact, possible to isolate all the sore points and problems
present with the current ant-based build into separate files that
provide "workarounds" or "hacks" so that gradle works on the same file
structure as ant. All of these files indicate a potential problem with
the build itself (dependencies on test classes, non-conventional
folder naming, etc.) but until we get rid of ant we can't easily fix
these:

https://github.com/apache/lucene-solr/tree/gradle-master/gradle/ant-compat

The gradle-master branch is essentially a fork from the master branch
with gradle build files layered on top. My (most controversial)
decision was to embrace transitive dependencies -- I agree with Mark
that they just can't be managed manually anymore, especially for Solr.
Gradle has vast possibilities of excluding and configuring
dependencies in any way we like without the need to enumerate each and
every package manually. When something is added for the first time or
upgraded we will have a safety trigger of the lock file and jar
checksums (I still need to add this particular section though).

The build works for me just fine: tests run, assembly works, solr
packaging works. IntelliJ imports the project as it is, without any
need for custom tuning.

The question is how do we proceed from here. I can merge master fairly
often but I think it would make most sense if we folded this in to
master _as soon as possible_ so that people start doing actual
development using gradle (and provide real life feedback). Please
shout out if you have something against it.

In the meantime, if you're familiar with gradle and would like to help
then here is a list of things that are missing from the build:
https://issues.apache.org/jira/browse/LUCENE-9077

Some of them are simple, some are more difficult. The list is very
likely not close to complete. All of these items require some
understanding of how gradle works. Gradle's documentation is very good
but my experience tells me gradle can be confusing as hell at the
beginning nonetheless - if you don't understand something in the
current build let me know, I'll try to help.

Silent consensus is great but we'd really love to hear from you. Try
it and let us know what you think.

(Erick &) Dawid

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message