www-builds mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Uwe Schindler" <uschind...@apache.org>
Subject RE: JDK, Maven, Ant versions have been consolidated.
Date Mon, 15 Aug 2016 17:47:58 GMT
Hi,

 

I thought about the wikipedia test files used by Jenkins: I think I would commit them to SVN
inside the Lucene/Solr project’s  folder. The Nightly-Test Jenkins jobs that use them could
simply check them out into a separate dir of the workspace. Byt that it is easier for us to
update them. If you also think this is fine, we can leave it like that.

 

The ~jenkins/lucene.build.properties file is (as said before) machine/hardware specific. I’d
keep them in Jenkins’ home dir. We may create a puppet module out of it, but this would
prevent us from optimizing or changing them quickly per project requirements.

 

Finally the “ant ivy-bootstrap” task was for now separated into a Jenkins job, that can
be manually run (https://builds.apache.org/job/Lucene-Ivy-Bootstrap/). In the future, we can
add the “ivy-bootstrap” target as dependency to the “ant jenkins” and other targets
which are solely triggered by Jenkins, so separately bootstrapping is no longer required (please
keep in mind that bootstrapping places the ivy.jar file in ~/.ant/lib, so outside workspace
– this is why it’s done separately!). I just have to fix the Jenkins-specific targets
in our build.xml in Git repo to only do the actual bootstrapping if Ivy is not installed locally
on Developer’s/Jenkins’ machine. Then it’s a no-op on most builds. For now the separate
Jenkins job is enough to quickly do the bootstrap as workaround.

 

If nobody stops me from committing the following file into SVN:

uschindler@lucene1-us-west:/x1/jenkins/lucene-data$ ls -lh

total 2.9G

-r--r--r-- 1 jenkins jenkins 2.9G May  9  2015 enwiki.random.lines.txt

 

I will do this later this evening and change the Jenkins builds to do an extra checkout into
the workspace next to Git.

I’d suggest to place the folder contents of ~/lucene-data here: https://svn.apache.org/repos/asf/lucene/test-data

 

Uwe

 

-----

Uwe Schindler

uschindler@apache.org 

ASF Member, Apache Lucene PMC / Committer

Bremen, Germany

http://lucene.apache.org/

 

From: Uwe Schindler [mailto:uschindler@apache.org] 
Sent: Monday, August 15, 2016 12:47 PM
To: dev@lucene.apache.org; gmcdonald@apache.org; 'Uwe Schindler' <uschindler@apache.org>
Cc: 'builds' <builds@apache.org>
Subject: RE: JDK, Maven, Ant versions have been consolidated.

 

Hi,

 

the test files could be a puppet module.

 

The lucene.build.properties in the Jenkins home dir is a “hardware specific” file, as
it allows to configure the number of parallel runners and where the special test files are
– one would not be able to use it on any other jenkins slave. If it is missing, defaults
are used, which are not good for optimal CPU use. We could add them also to the puppet module,
but this would make it harder for use to change the file easily.

 

All other files are unmodified.

 

Uwe

 

-----

Uwe Schindler

uschindler@apache.org <mailto:uschindler@apache.org>  

ASF Member, Apache Lucene PMC / Committer

Bremen, Germany

 <http://lucene.apache.org/> http://lucene.apache.org/

 

From: Gav [ <mailto:gmcdonald@apache.org> mailto:gmcdonald@apache.org] 
Sent: Monday, August 15, 2016 5:36 AM
To: Uwe Schindler < <mailto:uschindler@apache.org> uschindler@apache.org>
Cc: builds < <mailto:builds@apache.org> builds@apache.org>;  <mailto:dev@lucene.apache.org>
dev@lucene.apache.org
Subject: Re: JDK, Maven, Ant versions have been consolidated.

 

I'll make a copy of the jenkins home dir and get started.

It might be an idea to add those test files to puppet wdyt? 

(I see nothing private about them)

Gav...

 

On Sat, Aug 13, 2016 at 3:13 AM, Uwe Schindler <uschindler@apache.org <mailto:uschindler@apache.org>
> wrote:

Hi Gav,

 

I disabled all auto-running Jenkins Jobs of Lucene/Solr as this spammed the mailing list with
like 50 mails today. Can you send me a msg when you installed the required puppet modules?

 

Uwe

 

-----

Uwe Schindler

uschindler@apache.org <mailto:uschindler@apache.org>  

ASF Member, Apache Lucene PMC / Committer

Bremen, Germany

http://lucene.apache.org/

 

From: Gav [mailto:gmcdonald@apache.org <mailto:gmcdonald@apache.org> ] 
Sent: Friday, August 12, 2016 11:18 AM
To: Uwe Schindler <uschindler@apache.org <mailto:uschindler@apache.org> >
Cc: builds <builds@apache.org <mailto:builds@apache.org> >
Subject: Re: JDK, Maven, Ant versions have been consolidated.

 

Hi Uwe,

Will get on with your vm tomorrow. 

Gav...

 

On Fri, Aug 12, 2016 at 6:39 PM, Uwe Schindler <uschindler@apache.org <mailto:uschindler@apache.org>
> wrote:

Hi,

on the lucene-jenkins machine, the packages are still missing. I think you have to install
the jenkins puppet module. Here the message:
FATAL: Cannot find executable from the chosen Ant installation "Ant 1.8.4"

See my e-mail from last week about using the "official" jenkins puppet module (answer to your
question). Unless our customizations in the home dir keep preserved, we are fine with that:

My mail:

====================================
Hi Gav,

sorry for responding late – I am currently on vacation!

About your questions: In the past we required some extra modules (like older Subversion to
be compatible with the older Jenkins Checkouts). As we are on Git now and the build scripts
use JGit to fetch revision numbers and check for clean checkouts, this is obsolete. So if
you want to install the “Jenkins puppet module”, we are fine.

There is one special case for Lucene:
We have a hardware-specific “lucene.build.properties” file in Jenkin’s root and also
some additional “test files” (large wikiepdia checkouts) that are used for nightly tests.
If the puppet module does not delete those files in current ~/jenkins, it is ok. We would
still need access to the machine and sudo to jenkins.

If this is all fine, go ahead!

Uwe
=======================================

Uwe

-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de <mailto:uwe@thetaphi.de> 


> -----Original Message-----
> From: Gav [mailto:gmcdonald@apache.org <mailto:gmcdonald@apache.org> ]
> Sent: Friday, August 12, 2016 6:43 AM
> To: builds <builds@apache.org <mailto:builds@apache.org> >
> Subject: JDK, Maven, Ant versions have been consolidated.
>
> HI All,
>
> As per previous emails and as documented [1] [2] [3] we have now
> consolidated the JDK, Maven and Ant install versions.
>
> All versions are now installed/managed via puppet and any 'auto-install'
> versions have been removed.
>
> At the time of consolidation and removal of versions, over 200 jobs were
> still configured to use deprecated versions despite the emails I've been
> sending. Those 200+ jobs I have migrated myself to supported versions.
>
> Any issues with your jobs please do create a new INFRA jira or email this
> list.
>
> Thanks,
>
> Gav...
>
> [1] -
> https://cwiki.apache.org/confluence/display/INFRA/JDK+Installation+Matrix
> [2] -
> https://cwiki.apache.org/confluence/display/INFRA/Ant+Installation+Matrix
> [3] -
> https://cwiki.apache.org/confluence/display/INFRA/Maven+Installation+Mat
> rix

 

 


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message