From dev-return-367272-archive-asf-public=cust-asf.ponee.io@lucene.apache.org Fri Oct 4 09:40:25 2019 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [207.244.88.153]) by mx-eu-01.ponee.io (Postfix) with SMTP id 9618E180651 for ; Fri, 4 Oct 2019 11:40:25 +0200 (CEST) Received: (qmail 12425 invoked by uid 500); 4 Oct 2019 09:40:24 -0000 Mailing-List: contact dev-help@lucene.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@lucene.apache.org Delivered-To: mailing list dev@lucene.apache.org Received: (qmail 12397 invoked by uid 99); 4 Oct 2019 09:40:23 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 04 Oct 2019 09:40:23 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id EA5191A3257 for ; Fri, 4 Oct 2019 09:40:22 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 0.6 X-Spam-Level: X-Spam-Status: No, score=0.6 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, KAM_ASCII_DIVIDERS=0.8, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd2-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-he-de.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id uHloxdIl4fSN for ; Fri, 4 Oct 2019 09:40:06 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=2607:f8b0:4864:20::d2e; helo=mail-io1-xd2e.google.com; envelope-from=dawid.weiss@gmail.com; receiver= Received: from mail-io1-xd2e.google.com (mail-io1-xd2e.google.com [IPv6:2607:f8b0:4864:20::d2e]) by mx1-he-de.apache.org (ASF Mail Server at mx1-he-de.apache.org) with ESMTPS id B17B77DDF4 for ; Fri, 4 Oct 2019 09:40:05 +0000 (UTC) Received: by mail-io1-xd2e.google.com with SMTP id z19so12223620ior.0 for ; Fri, 04 Oct 2019 02:40:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :content-transfer-encoding; bh=0ApFnNUsWrtQYfc2+o9F4YWfakXcZuSZlwa3oBvYjCo=; b=V94G6a5mVzLdJcMuiz5r0k185cDDCIIr/IWbBmOZOD7SCqva8gbwnmQu3SYk+MPZFY ibW7/pZydv9DwH2WjtUBqQ/OXueLK9KrL94YR+YNcWQZsHQtPIyBQLvcjBkzLcTBBfqZ tYNUSVxXzbrIofg8qsJzY5J5EP4t5Bk0FensqUG4VGdh/2p7YrNbrkqzs0lBHPg0C3hU /gpFjWy1lUBaTq089V9/1at+7CzLCcWeu+bYSWuYhGpeWOq9hRxBmZ8l6kEd3pNWxMMJ 7cGVb1S4sw6bagdFhtjX5HelmST+QvzM/8jXqZTHSEkjfUcUR+rn1QMpKOiO/+YtiqK4 RfKg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:content-transfer-encoding; bh=0ApFnNUsWrtQYfc2+o9F4YWfakXcZuSZlwa3oBvYjCo=; b=ECpqLm/Wl4kqMi2HelSfUaWbkUj4p1kngu4Pjjs+6qn6K111cIDJGAoLly76FitPOD XV2BqFK/aGRLRLxaUzvEJuZaVAR7vah6W9K0FN6xqoh8H9lQBsrBfRQklKMzKmmlpadE JzQKe7ghHe5bFyKiUZ+SUT44UF+fTRNT45XQal4HcuVLu6vySdafTgrnYgbS0FYgOkSA KldAT8DAiAHRzTobAFb52aRdRSHOFvl4k09Pk6YyCrcJQnxLQuSO0+3+M9/qNnxTGTai Df+USCLJ3tCVFDkU1bPwdlbYWMmRDbxYYdhwcNg0MYbeh418hbIVVal1zXXjZYtdETlv NKbg== X-Gm-Message-State: APjAAAUmlUn1UxnTWTtJEp4maQMDQiLunPPGYJJBOBvr8YIM1BEYSoI0 QdSQRfW9haMn1StUC01hvjBlsVf3lPXXTopnye7FwlTq7Xs= X-Google-Smtp-Source: APXvYqy4gQzjoZrlA3V3tjKxDi2jqk9xZsMIf6g+qrs1Nvm6kSTpuEEi3wZHbOKeg7UInnEWjd49ylPQ6pYYC+dR6UE= X-Received: by 2002:a92:5fcb:: with SMTP id i72mr14724285ill.193.1570181998040; Fri, 04 Oct 2019 02:39:58 -0700 (PDT) MIME-Version: 1.0 References: <103f740a-c0a1-4601-b752-c83cbf0c93cc@Spark> <0B575103-F8EE-4A6C-B812-15D77FFB5C26@gmail.com> <2d0d348b-4a67-4c23-9c12-123ab3eb3622@Spark> In-Reply-To: From: Dawid Weiss Date: Fri, 4 Oct 2019 11:39:34 +0200 Message-ID: Subject: Re: The Lucene Solr Gradle Build Game plan To: Lucene Dev Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hi Cassandra, Apologies this took so long -- I wasn't familiar with these site-generation tools and the whole ecosystem is rather... fragile :) After a few attempts at using gradle plugins I eventually leaned towards using asciidoctor and jekyll explicitly (so that we know which versions are being used and don't have to rely on dependencies). I got bare bone html checking working, PDF generation working and site generation working although the final link check currently fail for me with a bunch of errors. This works for me on Windows... on Linux I get site-generation generate a strange error from within jekyll: Conversion error: Jekyll::AsciiDoc::Converter encountered an error while converting 'about-filters.adoc': Bad file descriptor - /usr/bin/python2 I could install python but I don't see why it'd need it. Perhaps there is something in the docs that would avoid using python altogether but I haven't had the time to look into it. Please feel free to check out the jira/SOLR-13452_gradle_7_refguide branch and try to run: ./gradlew -p solr/solr-ref-guide buildPdf buildSite There is a lot of room for improvement -- from property substitution, through how the "tools" are handled at the moment to task naming but I left this for the future. The initial step would be probably to get the site generation running on Linux/ Macs but I'd gladly hand it over back to you -- I can help with Gradle but a the rest of those tools are a mistery to me. Dawid On Fri, Sep 27, 2019 at 7:53 PM Dawid Weiss wrote: > > > No problem. I will get it to work entirely, but not before next week - I = am away for the weekend. > > Dawid > > On Fri, Sep 27, 2019, 16:17 Cassandra Targett wro= te: >> >> Thanks Dawid for working on this! I=E2=80=99ve been a bit swamped the la= st couple of days but will take a look today at what you=E2=80=99ve been ab= le to do so far and see where we might need to go from here. >> >> Cassandra >> On Sep 26, 2019, 7:25 AM -0500, Dawid Weiss , wro= te: >> >> I agree. Although I also understand the concern of trying to merge the >> changes while we're in the transition period... it'd be hell. I'd say >> move as much stuff as possible with the current folder structure (and >> ignore what cannot be ported easily) then switch as soon as possible >> to gradle and hack the old cruft with a chainsaw... >> >> D. >> >> On Thu, Sep 26, 2019 at 2:13 PM Erick Erickson = wrote: >> >> >> Of course I=E2=80=99ll completely defer to Dawid and Mark (well and anyb= ody else actually, you know, doing _work_), but just can=E2=80=99t resist c= himing in ;). >> >> My vote would be to =E2=80=9Cdo it the Gradle way=E2=80=9D. Yes, it=E2= =80=99s a PITA to learn new stuff and I won=E2=80=99t like it. Tough. I see= no reason to carry a bunch of cruft around because =E2=80=9Cthat the way w= e always did it=E2=80=9D. >> >> If we lose functionality, that=E2=80=99s a different discussion, startin= g with =E2=80=9Cdo we need that functionality". But jumping through hoops a= nd having to maintain that awkwardness forever going forward just because w= e forced the Ant structure on Gradle strikes me as a poor trade off. >> >> That said, I=E2=80=99m not doing the work so I really have no vote. But = don=E2=80=99t strain to do it the old way on my account ;) >> >> Erick >> >> P.S. Thanks Dawid for jumping in! >> >> On Sep 26, 2019, at 3:57 AM, Dawid Weiss wrote: >> >> I pushed it in to Lucene repo (it's on Cassandra's refguide branch >> anyway, so shouldn't interfere with anything else); seems like it's in >> better shape than previous code anyway (those questions I asked about >> the nature of the gradle port still hold though). >> >> I got as far as building initial bare-bones HTML. >> >> .\gradlew -p solr\solr-ref-guide clean bareBonesHtmlValidation >> >> I don't know anything about the pipeline involved (asciidoctor, etc.) >> so it's very likely some attributes will have to be corrected later >> on. >> >> Dawid >> >> On Wed, Sep 25, 2019 at 9:14 PM Dawid Weiss wrot= e: >> >> >> I looked at the solr ref guide build and started converting it to >> Gradle but have a question to Mark (because he coordinates the >> effort). >> >> What immediately jumps into face is the decision problem -- do we want >> to emulate what ant does at the moment or do we want to clean it up >> (breaking file/ folder structure and causing incompatibility with ant >> build). >> >> I went the "compatible" way and started porting ant tasks but it's >> quite awkward. For example -- there are template properties that refer >> to ivy version properties... we could emulate/ compute these but it's >> a pain. The way the module is currently structured is also awkward - >> it'd be more natural to have a separate java project with the "tools" >> required to compile extra stuff and just reference it from the manual >> build (and this would be a plain module, not a java module). This >> would limit the need for customizing source sets, classpaths, etc. >> >> My few initial tasks syncing sources, setting up infrastructure to >> filter templates and compiling the required tools are here: >> https://github.com/apache/lucene-solr/compare/jira/SOLR-13452_gradle_7_r= efguide...dweiss:jira/SOLR-13452_gradle_7_refguide?expand=3D1 >> >> I'll stop and wait for feedback (especially on the ivy versions issue) >> before I resume. >> >> Dawid >> >> On Wed, Sep 25, 2019 at 6:20 PM Dawid Weiss wrot= e: >> >> >> Never mind, I've got it. >> >> D. >> >> On Wed, Sep 25, 2019 at 7:59 AM Dawid Weiss wrot= e: >> >> >> Hi Cassandra, >> >> I=E2=80=99m more than happy to share more details our current build so w= e can replicate some of the above steps, but I=E2=80=99m stuck without a lo= t more basic Gradle skills that I don=E2=80=99t have time to acquire with d= ay-job/personal life commitments. I put it into a separate branch so we cou= ld iterate a little easier, can anyone help? >> >> >> Where is this branch you made changes on? If you can point me at the >> corresponding ant code I'll try to help you out. >> >> Dawid >> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org >> For additional commands, e-mail: dev-help@lucene.apache.org >> >> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org >> For additional commands, e-mail: dev-help@lucene.apache.org >> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org >> For additional commands, e-mail: dev-help@lucene.apache.org >> --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org For additional commands, e-mail: dev-help@lucene.apache.org