www-builds mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sean Busbey <bus...@cloudera.com>
Subject RE: JDK, Maven, Ant versions have been consolidated.
Date Wed, 17 Aug 2016 11:39:48 GMT
Isn't the wikipedia english corpus licensed either CC-BY-SA or GFDL? I
thought those licenses weren't okay to have in an ASF source repo?

-- 
Sean Busbey
I set this up and configured Jenkins to use the test-data checked out from
Subversion.

Uwe

-----
Uwe Schindler
uschindler@apache.org
ASF Member, Apache Lucene PMC / Committer
Bremen, Germany
http://lucene.apache.org/

> From: Steve Rowe [mailto:sarowe@gmail.com]
> Sent: Monday, August 15, 2016 8:40 PM
> To: dev@lucene.apache.org
> Subject: Re: JDK, Maven, Ant versions have been consolidated.
>
> Thanks Uwe for moving this process along.
>
> +1 to make a new SVN dir at https://svn.apache.org/repos/asf/lucene/test-
> data/ and put the enwiki data file there.
>
> --
> Steve
> www.lucidworks.com
>
> > On Aug 15, 2016, at 1:47 PM, Uwe Schindler <uschindler@apache.org>
> wrote:
> >
> > Hi,
> >
> > I thought about the wikipedia test files used by Jenkins: I think I
would
> commit them to SVN inside the Lucene/Solr project’s  folder. The
Nightly-Test
> Jenkins jobs that use them could simply check them out into a separate
dir of
> the workspace. Byt that it is easier for us to update them. If you also
think
> this is fine, we can leave it like that.
> >
> > The ~jenkins/lucene.build.properties file is (as said before)
> machine/hardware specific. I’d keep them in Jenkins’ home dir. We may
> create a puppet module out of it, but this would prevent us from
optimizing
> or changing them quickly per project requirements.
> >
> > Finally the “ant ivy-bootstrap” task was for now separated into a
Jenkins
> job, that can be manually run (https://builds.apache.org/job/Lucene-Ivy-
> Bootstrap/). In the future, we can add the “ivy-bootstrap” target as
> dependency to the “ant jenkins” and other targets which are solely
triggered
> by Jenkins, so separately bootstrapping is no longer required (please
keep in
> mind that bootstrapping places the ivy.jar file in ~/.ant/lib, so outside
> workspace – this is why it’s done separately!). I just have to fix the
Jenkins-
> specific targets in our build.xml in Git repo to only do the actual
> bootstrapping if Ivy is not installed locally on Developer’s/Jenkins’
machine.
> Then it’s a no-op on most builds. For now the separate Jenkins job is
enough
> to quickly do the bootstrap as workaround.
> >
> > If nobody stops me from committing the following file into SVN:
> > uschindler@lucene1-us-west:/x1/jenkins/lucene-data$ ls -lh
> > total 2.9G
> > -r--r--r-- 1 jenkins jenkins 2.9G May  9  2015 enwiki.random.lines.txt
> >
> > I will do this later this evening and change the Jenkins builds to do
an extra
> checkout into the workspace next to Git.
> > I’d suggest to place the folder contents of ~/lucene-data here:
> https://svn.apache.org/repos/asf/lucene/test-data
> >
> > Uwe
> >
> > -----
> > Uwe Schindler
> > uschindler@apache.org
> > ASF Member, Apache Lucene PMC / Committer
> > Bremen, Germany
> > http://lucene.apache.org/
> >
> > From: Uwe Schindler [mailto:uschindler@apache.org]
> > Sent: Monday, August 15, 2016 12:47 PM
> > To: dev@lucene.apache.org; gmcdonald@apache.org; 'Uwe Schindler'
> <uschindler@apache.org>
> > Cc: 'builds' <builds@apache.org>
> > Subject: RE: JDK, Maven, Ant versions have been consolidated.
> >
> > Hi,
> >
> > the test files could be a puppet module.
> >
> > The lucene.build.properties in the Jenkins home dir is a “hardware
specific”
> file, as it allows to configure the number of parallel runners and where
the
> special test files are – one would not be able to use it on any other
jenkins
> slave. If it is missing, defaults are used, which are not good for
optimal CPU
> use. We could add them also to the puppet module, but this would make it
> harder for use to change the file easily.
> >
> > All other files are unmodified.
> >
> > Uwe
> >
> > -----
> > Uwe Schindler
> > uschindler@apache.org
> > ASF Member, Apache Lucene PMC / Committer
> > Bremen, Germany
> > http://lucene.apache.org/
> >
> > From: Gav [mailto:gmcdonald@apache.org]
> > Sent: Monday, August 15, 2016 5:36 AM
> > To: Uwe Schindler <uschindler@apache.org>
> > Cc: builds <builds@apache.org>; dev@lucene.apache.org
> > Subject: Re: JDK, Maven, Ant versions have been consolidated.
> >
> > I'll make a copy of the jenkins home dir and get started.
> > It might be an idea to add those test files to puppet wdyt?
> > (I see nothing private about them)
> >
> > Gav...
> >
> > On Sat, Aug 13, 2016 at 3:13 AM, Uwe Schindler <uschindler@apache.org>
> wrote:
> >> Hi Gav,
> >>
> >> I disabled all auto-running Jenkins Jobs of Lucene/Solr as this spammed
> the mailing list with like 50 mails today. Can you send me a msg when you
> installed the required puppet modules?
> >>
> >> Uwe
> >>
> >> -----
> >> Uwe Schindler
> >> uschindler@apache.org
> >> ASF Member, Apache Lucene PMC / Committer
> >> Bremen, Germany
> >> http://lucene.apache.org/
> >>
> >> From: Gav [mailto:gmcdonald@apache.org]
> >> Sent: Friday, August 12, 2016 11:18 AM
> >> To: Uwe Schindler <uschindler@apache.org>
> >> Cc: builds <builds@apache.org>
> >> Subject: Re: JDK, Maven, Ant versions have been consolidated.
> >>
> >> Hi Uwe,
> >>
> >> Will get on with your vm tomorrow.
> >>
> >> Gav...
> >>
> >> On Fri, Aug 12, 2016 at 6:39 PM, Uwe Schindler <uschindler@apache.org>
> wrote:
> >>> Hi,
> >>>
> >>> on the lucene-jenkins machine, the packages are still missing. I
think you
> have to install the jenkins puppet module. Here the message:
> >>> FATAL: Cannot find executable from the chosen Ant installation "Ant
> 1.8.4"
> >>>
> >>> See my e-mail from last week about using the "official" jenkins puppet
> module (answer to your question). Unless our customizations in the home
> dir keep preserved, we are fine with that:
> >>>
> >>> My mail:
> >>>
> >>> ====================================
> >>> Hi Gav,
> >>>
> >>> sorry for responding late – I am currently on vacation!
> >>>
> >>> About your questions: In the past we required some extra modules (like
> older Subversion to be compatible with the older Jenkins Checkouts). As we
> are on Git now and the build scripts use JGit to fetch revision numbers
and
> check for clean checkouts, this is obsolete. So if you want to install the
> “Jenkins puppet module”, we are fine.
> >>>
> >>> There is one special case for Lucene:
> >>> We have a hardware-specific “lucene.build.properties” file in Jenkin’s
> root and also some additional “test files” (large wikiepdia checkouts)
that are
> used for nightly tests. If the puppet module does not delete those files
in
> current ~/jenkins, it is ok. We would still need access to the machine and
> sudo to jenkins.
> >>>
> >>> If this is all fine, go ahead!
> >>>
> >>> Uwe
> >>> =======================================
> >>>
> >>> Uwe
> >>>
> >>> -----
> >>> Uwe Schindler
> >>> H.-H.-Meier-Allee 63, D-28213 Bremen
> >>> http://www.thetaphi.de
> >>> eMail: uwe@thetaphi.de
> >>>
> >>> > -----Original Message-----
> >>> > From: Gav [mailto:gmcdonald@apache.org]
> >>> > Sent: Friday, August 12, 2016 6:43 AM
> >>> > To: builds <builds@apache.org>
> >>> > Subject: JDK, Maven, Ant versions have been consolidated.
> >>> >
> >>> > HI All,
> >>> >
> >>> > As per previous emails and as documented [1] [2] [3] we have now
> >>> > consolidated the JDK, Maven and Ant install versions.
> >>> >
> >>> > All versions are now installed/managed via puppet and any 'auto-
> install'
> >>> > versions have been removed.
> >>> >
> >>> > At the time of consolidation and removal of versions, over 200 jobs
> were
> >>> > still configured to use deprecated versions despite the emails I've
been
> >>> > sending. Those 200+ jobs I have migrated myself to supported
> versions.
> >>> >
> >>> > Any issues with your jobs please do create a new INFRA jira or email
> this
> >>> > list.
> >>> >
> >>> > Thanks,
> >>> >
> >>> > Gav...
> >>> >
> >>> > [1] -
> >>> >
> https://cwiki.apache.org/confluence/display/INFRA/JDK+Installation+Matrix
> >>> > [2] -
> >>> >
> https://cwiki.apache.org/confluence/display/INFRA/Ant+Installation+Matrix
> >>> > [3] -
> >>> >
> https://cwiki.apache.org/confluence/display/INFRA/Maven+Installation+Mat
> >>> > rix
> >>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message