www-builds mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gav <gmcdon...@apache.org>
Subject Re: JDK, Maven, Ant versions have been consolidated.
Date Thu, 18 Aug 2016 07:54:31 GMT
On Thu, Aug 18, 2016 at 5:29 PM, Uwe Schindler <uschindler@apache.org>
wrote:

> Ah,
>
>
>
> you can delete the backup folder of old jenkins home-dir.
>

Done.


> I nuked the workspaces inside already (after sudo jenkins, otherwise we
> would have gone out of disk space), but I cannot remove the folder on its
> own.
>
>
>
> The lucene.build.properties file is now the only “customization” of this
> machine, but this one is hardware-specific just for this node.
>
>
>

ok, I'm happy for you to manage that yourselves. It is possible to maintain
it in puppet but would be more lines of code than whats in that one file ;)

Gav...


> Uwe
>
>
>
> -----
>
> Uwe Schindler
>
> uschindler@apache.org
>
> ASF Member, Apache Lucene PMC / Committer
>
> Bremen, Germany
>
> http://lucene.apache.org/
>
>
>
> *From:* Gav [mailto:gmcdonald@apache.org]
> *Sent:* Thursday, August 18, 2016 6:47 AM
> *To:* Uwe Schindler <uschindler@apache.org>
> *Cc:* dev@lucene.apache.org; builds <builds@apache.org>
>
> *Subject:* Re: JDK, Maven, Ant versions have been consolidated.
>
>
>
> Its fine where it is.
>
> Is there any more to do here for infra Uwe or is the Lucene Jenkins node
> all good now ?
>
> Gav...
>
>
>
> On Wed, Aug 17, 2016 at 10:33 PM, Uwe Schindler <uschindler@apache.org>
> wrote:
>
> Hi,
>
>
>
> it is not part of any distribution or in any developer checkouts. We just
> placed it there for Jenkins access to run more extensive tests – we can
> also move it to a private subversion branch, not accessible from public? We
> could then add the Jenkins user to that folder.
>
>
>
> Uwe
>
>
>
> -----
>
> Uwe Schindler
>
> uschindler@apache.org
>
> ASF Member, Apache Lucene PMC / Committer
>
> Bremen, Germany
>
> http://lucene.apache.org/
>
>
>
> *From:* Sean Busbey [mailto:busbey@cloudera.com]
> *Sent:* Wednesday, August 17, 2016 1:40 PM
> *To:* builds <builds@apache.org>
> *Cc:* gmcdonald@apache.org; dev@lucene.apache.org
>
>
> *Subject:* RE: JDK, Maven, Ant versions have been consolidated.
>
>
>
> Isn't the wikipedia english corpus licensed either CC-BY-SA or GFDL? I
> thought those licenses weren't okay to have in an ASF source repo?
>
> --
> Sean Busbey
>
> I set this up and configured Jenkins to use the test-data checked out from
> Subversion.
>
> Uwe
>
> -----
> Uwe Schindler
> uschindler@apache.org
> ASF Member, Apache Lucene PMC / Committer
> Bremen, Germany
> http://lucene.apache.org/
>
> > From: Steve Rowe [mailto:sarowe@gmail.com]
> > Sent: Monday, August 15, 2016 8:40 PM
> > To: dev@lucene.apache.org
> > Subject: Re: JDK, Maven, Ant versions have been consolidated.
> >
> > Thanks Uwe for moving this process along.
> >
> > +1 to make a new SVN dir at https://svn.apache.org/repos/
> asf/lucene/test-
> > data/ and put the enwiki data file there.
> >
> > --
> > Steve
> > www.lucidworks.com
> >
> > > On Aug 15, 2016, at 1:47 PM, Uwe Schindler <uschindler@apache.org>
> > wrote:
> > >
> > > Hi,
> > >
> > > I thought about the wikipedia test files used by Jenkins: I think I
> would
> > commit them to SVN inside the Lucene/Solr project’s  folder. The
> Nightly-Test
> > Jenkins jobs that use them could simply check them out into a separate
> dir of
> > the workspace. Byt that it is easier for us to update them. If you also
> think
> > this is fine, we can leave it like that.
> > >
> > > The ~jenkins/lucene.build.properties file is (as said before)
> > machine/hardware specific. I’d keep them in Jenkins’ home dir. We may
> > create a puppet module out of it, but this would prevent us from
> optimizing
> > or changing them quickly per project requirements.
> > >
> > > Finally the “ant ivy-bootstrap” task was for now separated into a
> Jenkins
> > job, that can be manually run (https://builds.apache.org/job/Lucene-Ivy-
> > Bootstrap/). In the future, we can add the “ivy-bootstrap” target as
> > dependency to the “ant jenkins” and other targets which are solely
> triggered
> > by Jenkins, so separately bootstrapping is no longer required (please
> keep in
> > mind that bootstrapping places the ivy.jar file in ~/.ant/lib, so outside
> > workspace – this is why it’s done separately!). I just have to fix the
> Jenkins-
> > specific targets in our build.xml in Git repo to only do the actual
> > bootstrapping if Ivy is not installed locally on Developer’s/Jenkins’
> machine.
> > Then it’s a no-op on most builds. For now the separate Jenkins job is
> enough
> > to quickly do the bootstrap as workaround.
> > >
> > > If nobody stops me from committing the following file into SVN:
> > > uschindler@lucene1-us-west:/x1/jenkins/lucene-data$ ls -lh
> > > total 2.9G
> > > -r--r--r-- 1 jenkins jenkins 2.9G May  9  2015 enwiki.random.lines.txt
> > >
> > > I will do this later this evening and change the Jenkins builds to do
> an extra
> > checkout into the workspace next to Git.
> > > I’d suggest to place the folder contents of ~/lucene-data here:
> > https://svn.apache.org/repos/asf/lucene/test-data
> > >
> > > Uwe
> > >
> > > -----
> > > Uwe Schindler
> > > uschindler@apache.org
> > > ASF Member, Apache Lucene PMC / Committer
> > > Bremen, Germany
> > > http://lucene.apache.org/
> > >
> > > From: Uwe Schindler [mailto:uschindler@apache.org]
> > > Sent: Monday, August 15, 2016 12:47 PM
> > > To: dev@lucene.apache.org; gmcdonald@apache.org; 'Uwe Schindler'
> > <uschindler@apache.org>
> > > Cc: 'builds' <builds@apache.org>
> > > Subject: RE: JDK, Maven, Ant versions have been consolidated.
> > >
> > > Hi,
> > >
> > > the test files could be a puppet module.
> > >
> > > The lucene.build.properties in the Jenkins home dir is a “hardware
> specific”
> > file, as it allows to configure the number of parallel runners and where
> the
> > special test files are – one would not be able to use it on any other
> jenkins
> > slave. If it is missing, defaults are used, which are not good for
> optimal CPU
> > use. We could add them also to the puppet module, but this would make it
> > harder for use to change the file easily.
> > >
> > > All other files are unmodified.
> > >
> > > Uwe
> > >
> > > -----
> > > Uwe Schindler
> > > uschindler@apache.org
> > > ASF Member, Apache Lucene PMC / Committer
> > > Bremen, Germany
> > > http://lucene.apache.org/
> > >
> > > From: Gav [mailto:gmcdonald@apache.org]
> > > Sent: Monday, August 15, 2016 5:36 AM
> > > To: Uwe Schindler <uschindler@apache.org>
> > > Cc: builds <builds@apache.org>; dev@lucene.apache.org
> > > Subject: Re: JDK, Maven, Ant versions have been consolidated.
> > >
> > > I'll make a copy of the jenkins home dir and get started.
> > > It might be an idea to add those test files to puppet wdyt?
> > > (I see nothing private about them)
> > >
> > > Gav...
> > >
> > > On Sat, Aug 13, 2016 at 3:13 AM, Uwe Schindler <uschindler@apache.org>
> > wrote:
> > >> Hi Gav,
> > >>
> > >> I disabled all auto-running Jenkins Jobs of Lucene/Solr as this
> spammed
> > the mailing list with like 50 mails today. Can you send me a msg when you
> > installed the required puppet modules?
> > >>
> > >> Uwe
> > >>
> > >> -----
> > >> Uwe Schindler
> > >> uschindler@apache.org
> > >> ASF Member, Apache Lucene PMC / Committer
> > >> Bremen, Germany
> > >> http://lucene.apache.org/
> > >>
> > >> From: Gav [mailto:gmcdonald@apache.org]
> > >> Sent: Friday, August 12, 2016 11:18 AM
> > >> To: Uwe Schindler <uschindler@apache.org>
> > >> Cc: builds <builds@apache.org>
> > >> Subject: Re: JDK, Maven, Ant versions have been consolidated.
> > >>
> > >> Hi Uwe,
> > >>
> > >> Will get on with your vm tomorrow.
> > >>
> > >> Gav...
> > >>
> > >> On Fri, Aug 12, 2016 at 6:39 PM, Uwe Schindler <uschindler@apache.org
> >
> > wrote:
> > >>> Hi,
> > >>>
> > >>> on the lucene-jenkins machine, the packages are still missing. I
> think you
> > have to install the jenkins puppet module. Here the message:
> > >>> FATAL: Cannot find executable from the chosen Ant installation "Ant
> > 1.8.4"
> > >>>
> > >>> See my e-mail from last week about using the "official" jenkins
> puppet
> > module (answer to your question). Unless our customizations in the home
> > dir keep preserved, we are fine with that:
> > >>>
> > >>> My mail:
> > >>>
> > >>> ====================================
> > >>> Hi Gav,
> > >>>
> > >>> sorry for responding late – I am currently on vacation!
> > >>>
> > >>> About your questions: In the past we required some extra modules
> (like
> > older Subversion to be compatible with the older Jenkins Checkouts). As
> we
> > are on Git now and the build scripts use JGit to fetch revision numbers
> and
> > check for clean checkouts, this is obsolete. So if you want to install
> the
> > “Jenkins puppet module”, we are fine.
> > >>>
> > >>> There is one special case for Lucene:
> > >>> We have a hardware-specific “lucene.build.properties” file in
> Jenkin’s
> > root and also some additional “test files” (large wikiepdia checkouts)
> that are
> > used for nightly tests. If the puppet module does not delete those files
> in
> > current ~/jenkins, it is ok. We would still need access to the machine
> and
> > sudo to jenkins.
> > >>>
> > >>> If this is all fine, go ahead!
> > >>>
> > >>> Uwe
> > >>> =======================================
> > >>>
> > >>> Uwe
> > >>>
> > >>> -----
> > >>> Uwe Schindler
> > >>> H.-H.-Meier-Allee 63, D-28213 Bremen
> > >>> http://www.thetaphi.de
> > >>> eMail: uwe@thetaphi.de
> > >>>
> > >>> > -----Original Message-----
> > >>> > From: Gav [mailto:gmcdonald@apache.org]
> > >>> > Sent: Friday, August 12, 2016 6:43 AM
> > >>> > To: builds <builds@apache.org>
> > >>> > Subject: JDK, Maven, Ant versions have been consolidated.
> > >>> >
> > >>> > HI All,
> > >>> >
> > >>> > As per previous emails and as documented [1] [2] [3] we have now
> > >>> > consolidated the JDK, Maven and Ant install versions.
> > >>> >
> > >>> > All versions are now installed/managed via puppet and any 'auto-
> > install'
> > >>> > versions have been removed.
> > >>> >
> > >>> > At the time of consolidation and removal of versions, over 200
jobs
> > were
> > >>> > still configured to use deprecated versions despite the emails
> I've been
> > >>> > sending. Those 200+ jobs I have migrated myself to supported
> > versions.
> > >>> >
> > >>> > Any issues with your jobs please do create a new INFRA jira or
> email
> > this
> > >>> > list.
> > >>> >
> > >>> > Thanks,
> > >>> >
> > >>> > Gav...
> > >>> >
> > >>> > [1] -
> > >>> >
> > https://cwiki.apache.org/confluence/display/INFRA/JDK+
> Installation+Matrix
> > >>> > [2] -
> > >>> >
> > https://cwiki.apache.org/confluence/display/INFRA/Ant+
> Installation+Matrix
> > >>> > [3] -
> > >>> >
> > https://cwiki.apache.org/confluence/display/INFRA/Maven+Installation+Mat
> > >>> > rix
> > >>>
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: dev-help@lucene.apache.org
>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message