From dev-return-367273-archive-asf-public=cust-asf.ponee.io@lucene.apache.org Fri Oct 4 09:42:38 2019 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [207.244.88.153]) by mx-eu-01.ponee.io (Postfix) with SMTP id 21F8F180651 for ; Fri, 4 Oct 2019 11:42:38 +0200 (CEST) Received: (qmail 20223 invoked by uid 500); 4 Oct 2019 09:42:36 -0000 Mailing-List: contact dev-help@lucene.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@lucene.apache.org Delivered-To: mailing list dev@lucene.apache.org Received: (qmail 20212 invoked by uid 99); 4 Oct 2019 09:42:35 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 04 Oct 2019 09:42:35 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id 2377AC137F for ; Fri, 4 Oct 2019 09:42:35 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 0.599 X-Spam-Level: X-Spam-Status: No, score=0.599 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, KAM_ASCII_DIVIDERS=0.8, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd1-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-ec2-va.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id bdwSNpHHJzfa for ; Fri, 4 Oct 2019 09:42:32 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=209.85.166.51; helo=mail-io1-f51.google.com; envelope-from=dawid.weiss@gmail.com; receiver= Received: from mail-io1-f51.google.com (mail-io1-f51.google.com [209.85.166.51]) by mx1-ec2-va.apache.org (ASF Mail Server at mx1-ec2-va.apache.org) with ESMTPS id 9DE44BC5D3 for ; Fri, 4 Oct 2019 09:42:32 +0000 (UTC) Received: by mail-io1-f51.google.com with SMTP id q1so12211948ion.1 for ; Fri, 04 Oct 2019 02:42:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :content-transfer-encoding; bh=PUnHMG61PZnkjdGigJTeucHCC0D0/WaDwf2fxZcEuv4=; b=K308q1bi/TCRipHlrwkZvAcVAaf2IvkftZVd9W4p7cn69lB3yGro9VvnWUJ6c/5k95 cnARDQNHZtRGe1vaPLRr9V+ZrcLx8WfcwpYwIc6fqVLX9XxRc3Vfa+Wnmxdbw2GgTPeQ QpCh85cqWFvMaOGSF3aRdWQyQdyyOhms+d0rTfs+P07TxC0KCewMcz2+JbOY5BP7jqoG qTnxLSPUjpuD28KqMUB5z4j5JBGgDMXJumPV5+PojayBU3CLWuCiqX7BNT6wtL5zaWP4 L8DOGWJZvW+XmrVkL3eIXlbVmF+3bfH749HpMM3W4apOl4WG98jx1sPIUZCfykmQaowM rEoA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:content-transfer-encoding; bh=PUnHMG61PZnkjdGigJTeucHCC0D0/WaDwf2fxZcEuv4=; b=bhcv9qDcCmgSbBC2h+J0kViF/giP0IBTJStNqK6VlHgRDmQuW2WhAqjhqQr0tVlbe+ HhN+CnkTVW38Oht3lWltPdAqPq52+qsurdajx45Vzp+D8mz9Zm/PmE5NSziy4n3nFhlK qgwgUYyyyCKMOJEIG0VSCvDIU7ggBmCsg7lpOPx/vm3MRm9JfzXMVAQW5DbI1fQZBxsm QxPitb6H0a2362h6m5j+Vw7ofFKeTIQ/geCkroYqjOigQ+rfdQkT/wqyr/PR06DTZCJZ kgiogmGoPkdqaZU9ib55NyVnFP97yRBoTW2rIc5qj1TIEFeUI3ymUAn1O9J2u72so4KU 4jfg== X-Gm-Message-State: APjAAAXkPllNAWkOkCGyf45pziJ155lT7y06+wqz+8Ikl/j+kmk5tbuT 1z8l0XfzkM4gOeL0P+mYTFt37uxYOgkjhHQ4gbSU7m43kho= X-Google-Smtp-Source: APXvYqzPlur8Aqp0esdRI8+EkjInAIsqKQfnYcvUx1shVqTMvTeXxwOZ1Zl5m+layFm4eqghE3/Kfm9HC+jv3iRPtnc= X-Received: by 2002:a92:c682:: with SMTP id o2mr14690878ilg.51.1570182151562; Fri, 04 Oct 2019 02:42:31 -0700 (PDT) MIME-Version: 1.0 References: <103f740a-c0a1-4601-b752-c83cbf0c93cc@Spark> <0B575103-F8EE-4A6C-B812-15D77FFB5C26@gmail.com> <2d0d348b-4a67-4c23-9c12-123ab3eb3622@Spark> In-Reply-To: From: Dawid Weiss Date: Fri, 4 Oct 2019 11:42:08 +0200 Message-ID: Subject: Re: The Lucene Solr Gradle Build Game plan To: Lucene Dev Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Seems like pygments is to blame for the python requirement... I didn't check but there seem to be ruby-only highlighters for jekyll as well: https://jekyll-windows.juthilo.com/3-syntax-highlighting/ On Fri, Oct 4, 2019 at 11:39 AM Dawid Weiss wrote: > > Hi Cassandra, > > Apologies this took so long -- I wasn't familiar with these > site-generation tools and the whole ecosystem is rather... fragile :) > After a few attempts at using gradle plugins I eventually leaned > towards using asciidoctor and jekyll explicitly (so that we know which > versions are being used and don't have to rely on dependencies). > > I got bare bone html checking working, PDF generation working and site > generation working although the final link check currently fail for me > with a bunch of errors. This works for me on Windows... on Linux I get > site-generation generate a strange error from within jekyll: > > Conversion error: Jekyll::AsciiDoc::Converter encountered an error > while converting 'about-filters.adoc': > Bad file descriptor - /usr/bin/python2 > > I could install python but I don't see why it'd need it. Perhaps there > is something in the docs that would avoid using python altogether but > I haven't had the time to look into it. > > Please feel free to check out the jira/SOLR-13452_gradle_7_refguide > branch and try to run: > > ./gradlew -p solr/solr-ref-guide buildPdf buildSite > > There is a lot of room for improvement -- from property substitution, > through how the "tools" are handled at the moment to task naming but I > left this for the future. The initial step would be probably to get > the site generation running on Linux/ Macs but I'd gladly hand it over > back to you -- I can help with Gradle but a the rest of those tools > are a mistery to me. > > Dawid > > On Fri, Sep 27, 2019 at 7:53 PM Dawid Weiss wrote= : > > > > > > No problem. I will get it to work entirely, but not before next week - = I am away for the weekend. > > > > Dawid > > > > On Fri, Sep 27, 2019, 16:17 Cassandra Targett w= rote: > >> > >> Thanks Dawid for working on this! I=E2=80=99ve been a bit swamped the = last couple of days but will take a look today at what you=E2=80=99ve been = able to do so far and see where we might need to go from here. > >> > >> Cassandra > >> On Sep 26, 2019, 7:25 AM -0500, Dawid Weiss , w= rote: > >> > >> I agree. Although I also understand the concern of trying to merge the > >> changes while we're in the transition period... it'd be hell. I'd say > >> move as much stuff as possible with the current folder structure (and > >> ignore what cannot be ported easily) then switch as soon as possible > >> to gradle and hack the old cruft with a chainsaw... > >> > >> D. > >> > >> On Thu, Sep 26, 2019 at 2:13 PM Erick Erickson wrote: > >> > >> > >> Of course I=E2=80=99ll completely defer to Dawid and Mark (well and an= ybody else actually, you know, doing _work_), but just can=E2=80=99t resist= chiming in ;). > >> > >> My vote would be to =E2=80=9Cdo it the Gradle way=E2=80=9D. Yes, it=E2= =80=99s a PITA to learn new stuff and I won=E2=80=99t like it. Tough. I see= no reason to carry a bunch of cruft around because =E2=80=9Cthat the way w= e always did it=E2=80=9D. > >> > >> If we lose functionality, that=E2=80=99s a different discussion, start= ing with =E2=80=9Cdo we need that functionality". But jumping through hoops= and having to maintain that awkwardness forever going forward just because= we forced the Ant structure on Gradle strikes me as a poor trade off. > >> > >> That said, I=E2=80=99m not doing the work so I really have no vote. Bu= t don=E2=80=99t strain to do it the old way on my account ;) > >> > >> Erick > >> > >> P.S. Thanks Dawid for jumping in! > >> > >> On Sep 26, 2019, at 3:57 AM, Dawid Weiss wrote= : > >> > >> I pushed it in to Lucene repo (it's on Cassandra's refguide branch > >> anyway, so shouldn't interfere with anything else); seems like it's in > >> better shape than previous code anyway (those questions I asked about > >> the nature of the gradle port still hold though). > >> > >> I got as far as building initial bare-bones HTML. > >> > >> .\gradlew -p solr\solr-ref-guide clean bareBonesHtmlValidation > >> > >> I don't know anything about the pipeline involved (asciidoctor, etc.) > >> so it's very likely some attributes will have to be corrected later > >> on. > >> > >> Dawid > >> > >> On Wed, Sep 25, 2019 at 9:14 PM Dawid Weiss wr= ote: > >> > >> > >> I looked at the solr ref guide build and started converting it to > >> Gradle but have a question to Mark (because he coordinates the > >> effort). > >> > >> What immediately jumps into face is the decision problem -- do we want > >> to emulate what ant does at the moment or do we want to clean it up > >> (breaking file/ folder structure and causing incompatibility with ant > >> build). > >> > >> I went the "compatible" way and started porting ant tasks but it's > >> quite awkward. For example -- there are template properties that refer > >> to ivy version properties... we could emulate/ compute these but it's > >> a pain. The way the module is currently structured is also awkward - > >> it'd be more natural to have a separate java project with the "tools" > >> required to compile extra stuff and just reference it from the manual > >> build (and this would be a plain module, not a java module). This > >> would limit the need for customizing source sets, classpaths, etc. > >> > >> My few initial tasks syncing sources, setting up infrastructure to > >> filter templates and compiling the required tools are here: > >> https://github.com/apache/lucene-solr/compare/jira/SOLR-13452_gradle_7= _refguide...dweiss:jira/SOLR-13452_gradle_7_refguide?expand=3D1 > >> > >> I'll stop and wait for feedback (especially on the ivy versions issue) > >> before I resume. > >> > >> Dawid > >> > >> On Wed, Sep 25, 2019 at 6:20 PM Dawid Weiss wr= ote: > >> > >> > >> Never mind, I've got it. > >> > >> D. > >> > >> On Wed, Sep 25, 2019 at 7:59 AM Dawid Weiss wr= ote: > >> > >> > >> Hi Cassandra, > >> > >> I=E2=80=99m more than happy to share more details our current build so= we can replicate some of the above steps, but I=E2=80=99m stuck without a = lot more basic Gradle skills that I don=E2=80=99t have time to acquire with= day-job/personal life commitments. I put it into a separate branch so we c= ould iterate a little easier, can anyone help? > >> > >> > >> Where is this branch you made changes on? If you can point me at the > >> corresponding ant code I'll try to help you out. > >> > >> Dawid > >> > >> > >> --------------------------------------------------------------------- > >> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org > >> For additional commands, e-mail: dev-help@lucene.apache.org > >> > >> > >> > >> --------------------------------------------------------------------- > >> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org > >> For additional commands, e-mail: dev-help@lucene.apache.org > >> > >> > >> --------------------------------------------------------------------- > >> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org > >> For additional commands, e-mail: dev-help@lucene.apache.org > >> --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org For additional commands, e-mail: dev-help@lucene.apache.org