lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dawid Weiss <dawid.we...@gmail.com>
Subject Re: The Lucene Solr Gradle Build Game plan
Date Fri, 04 Oct 2019 09:39:34 GMT
Hi Cassandra,

Apologies this took so long -- I wasn't familiar with these
site-generation tools and the whole ecosystem is rather... fragile :)
After a few attempts at using gradle plugins I eventually leaned
towards using asciidoctor and jekyll explicitly (so that we know which
versions are being used and don't have to rely on dependencies).

I got bare bone html checking working, PDF generation working and site
generation working although the final link check currently fail for me
with a bunch of errors. This works for me on Windows... on Linux I get
site-generation generate a strange error from within jekyll:

  Conversion error: Jekyll::AsciiDoc::Converter encountered an error
while converting 'about-filters.adoc':
                    Bad file descriptor - /usr/bin/python2

I could install python but I don't see why it'd need it. Perhaps there
is something in the docs that would avoid using python altogether but
I haven't had the time to look into it.

Please feel free to check out the jira/SOLR-13452_gradle_7_refguide
branch and try to run:

./gradlew -p solr/solr-ref-guide buildPdf buildSite

There is a lot of room for improvement -- from property substitution,
through how the "tools" are handled at the moment to task naming but I
left this for the future. The initial step would be probably to get
the site generation running on Linux/ Macs but I'd gladly hand it over
back to you -- I can help with Gradle but a the rest of those tools
are a mistery to me.

Dawid

On Fri, Sep 27, 2019 at 7:53 PM Dawid Weiss <dawid.weiss@gmail.com> wrote:
>
>
> No problem. I will get it to work entirely, but not before next week - I am away for
the weekend.
>
> Dawid
>
> On Fri, Sep 27, 2019, 16:17 Cassandra Targett <casstargett@gmail.com> wrote:
>>
>> Thanks Dawid for working on this! I’ve been a bit swamped the last couple of days
but will take a look today at what you’ve been able to do so far and see where we might
need to go from here.
>>
>> Cassandra
>> On Sep 26, 2019, 7:25 AM -0500, Dawid Weiss <dawid.weiss@gmail.com>, wrote:
>>
>> I agree. Although I also understand the concern of trying to merge the
>> changes while we're in the transition period... it'd be hell. I'd say
>> move as much stuff as possible with the current folder structure (and
>> ignore what cannot be ported easily) then switch as soon as possible
>> to gradle and hack the old cruft with a chainsaw...
>>
>> D.
>>
>> On Thu, Sep 26, 2019 at 2:13 PM Erick Erickson <erickerickson@gmail.com> wrote:
>>
>>
>> Of course I’ll completely defer to Dawid and Mark (well and anybody else actually,
you know, doing _work_), but just can’t resist chiming in ;).
>>
>> My vote would be to “do it the Gradle way”. Yes, it’s a PITA to learn new stuff
and I won’t like it. Tough. I see no reason to carry a bunch of cruft around because “that
the way we always did it”.
>>
>> If we lose functionality, that’s a different discussion, starting with “do we
need that functionality". But jumping through hoops and having to maintain that awkwardness
forever going forward just because we forced the Ant structure on Gradle strikes me as a poor
trade off.
>>
>> That said, I’m not doing the work so I really have no vote. But don’t strain
to do it the old way on my account ;)
>>
>> Erick
>>
>> P.S. Thanks Dawid for jumping in!
>>
>> On Sep 26, 2019, at 3:57 AM, Dawid Weiss <dawid.weiss@gmail.com> wrote:
>>
>> I pushed it in to Lucene repo (it's on Cassandra's refguide branch
>> anyway, so shouldn't interfere with anything else); seems like it's in
>> better shape than previous code anyway (those questions I asked about
>> the nature of the gradle port still hold though).
>>
>> I got as far as building initial bare-bones HTML.
>>
>> .\gradlew -p solr\solr-ref-guide clean bareBonesHtmlValidation
>>
>> I don't know anything about the pipeline involved (asciidoctor, etc.)
>> so it's very likely some attributes will have to be corrected later
>> on.
>>
>> Dawid
>>
>> On Wed, Sep 25, 2019 at 9:14 PM Dawid Weiss <dawid.weiss@gmail.com> wrote:
>>
>>
>> I looked at the solr ref guide build and started converting it to
>> Gradle but have a question to Mark (because he coordinates the
>> effort).
>>
>> What immediately jumps into face is the decision problem -- do we want
>> to emulate what ant does at the moment or do we want to clean it up
>> (breaking file/ folder structure and causing incompatibility with ant
>> build).
>>
>> I went the "compatible" way and started porting ant tasks but it's
>> quite awkward. For example -- there are template properties that refer
>> to ivy version properties... we could emulate/ compute these but it's
>> a pain. The way the module is currently structured is also awkward -
>> it'd be more natural to have a separate java project with the "tools"
>> required to compile extra stuff and just reference it from the manual
>> build (and this would be a plain module, not a java module). This
>> would limit the need for customizing source sets, classpaths, etc.
>>
>> My few initial tasks syncing sources, setting up infrastructure to
>> filter templates and compiling the required tools are here:
>> https://github.com/apache/lucene-solr/compare/jira/SOLR-13452_gradle_7_refguide...dweiss:jira/SOLR-13452_gradle_7_refguide?expand=1
>>
>> I'll stop and wait for feedback (especially on the ivy versions issue)
>> before I resume.
>>
>> Dawid
>>
>> On Wed, Sep 25, 2019 at 6:20 PM Dawid Weiss <dawid.weiss@gmail.com> wrote:
>>
>>
>> Never mind, I've got it.
>>
>> D.
>>
>> On Wed, Sep 25, 2019 at 7:59 AM Dawid Weiss <dawid.weiss@gmail.com> wrote:
>>
>>
>> Hi Cassandra,
>>
>> I’m more than happy to share more details our current build so we can replicate
some of the above steps, but I’m stuck without a lot more basic Gradle skills that I don’t
have time to acquire with day-job/personal life commitments. I put it into a separate branch
so we could iterate a little easier, can anyone help?
>>
>>
>> Where is this branch you made changes on? If you can point me at the
>> corresponding ant code I'll try to help you out.
>>
>> Dawid
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: dev-help@lucene.apache.org
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: dev-help@lucene.apache.org
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: dev-help@lucene.apache.org
>>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message