hadoop-common-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Wiki <wikidi...@apache.org>
Subject [Hadoop Wiki] Update of "HudsonBuildServer" by NigelDaley
Date Fri, 07 Mar 2008 06:34:47 GMT
Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Hadoop Wiki" for change notification.

The following page has been changed by NigelDaley:
http://wiki.apache.org/hadoop/HudsonBuildServer

The comment on the change is:
Updated for new hudson.zones.apache.org build machine

------------------------------------------------------------------------------
  
  We use [https://hudson.dev.java.net/ Hudson] as our continuous integration server:
  
-  * [http://lucene.zones.apache.org:8080/hudson/]
+  * [http://hudson.zones.apache.org/hudson/]
  
- Hudson performs nightly builds for Lucene, Solr, Nutch, and Hadoop.  In addition is runs
automated patch verification builds for Hadoop.
+ Hudson runs automated patch verification builds for Hadoop.
  
  = Administering Hudson =
  
- Hudson is run by the {{{hudson}}} user on {{{lucene.zones.apache.org}}}.  Any Lucene committer
may have an account on this machine.  To manage the Hudson server, log into this machine and
+ Hudson is run by the {{{hudson}}} user on {{{hudson.zones.apache.org}}}.  Any Apache committer
may have an account on this machine.  To manage the Hudson server, log into this machine and
  
  {{{sudo su - hudson}}}
  
  This will give you the proper shell environment to manage the Hudson server.  Hudson uses
a number of tools for itself and for the builds it runs.  All tools are located in {{{~hudson/tools}}}.
 Hudson stores all of it's config files and builds in {{{$HUDSON_HOME}}} which is set to {{{~hudson/hudson}}}.
  
- '''Note:''' Please notify others on the Hadoop IRC channel (#hadoop at irc.freenode.net)
before you start manipulating Hudson, to make sure that you're not stepping on someone else's
toes.  Any substantial configuration changes should be discussed on the developer mailing
list.
+ '''Note:''' Please notify others on the Hadoop IRC channel (#hadoop at irc.freenode.net)
before you start manipulating Hudson, to make sure that you're not stepping on someone else's
toes.  Any substantial configuration changes to Hadoop builds should be discussed on the developer
mailing list.
  
  == Tasks ==
  
- Most tasks, such as creating a new build or modifying an existing build, are performed through
the Hudson web interface.  Once you login to Hudson, you are provided with an expanded number
of administration options.  To create a login on Hudson, see below.  The following tasks must
be performed after logging into {{{lucene.zones.apache.org}}} and switching to the {{{hudson}}}
user.
+ Most tasks, such as creating a new build or modifying an existing build, are performed through
the Hudson web interface.  Once you login to Hudson, you are provided with an expanded number
of administration options.  To create a login on Hudson, see below.  The following tasks must
be performed after logging into {{{hudson.zones.apache.org}}} and switching to the {{{hudson}}}
user.
  
     1. '''Stopping a nightly build:'''
-       When logged into Hudson, a red X button is displayed next to each running build. 
Clicking the button will end the build as far as Hudson is concerned, but the underlying processes
sometimes don't get properly terminated.  A better approach is to kill the {{{build-test.sh}}}
process (used by some builds) or, if that doesn't exist, the {{{ant}}} process:
+       When logged into Hudson, a red X button is displayed next to each running build. 
Clicking the button will end the build as far as Hudson is concerned, but the underlying processes
sometimes don't get properly terminated.  A better approach is to kill the {{{hudsonBuildHadoopNightly.sh}}}
process or, if that doesn't exist, the {{{ant}}} process:
-          1. Login to {{{lucene.zones.apache.org}}}
+          1. Login to {{{hudson.zones.apache.org}}}
           1. Switch to the {{{hudson}}} user
-          1. {{{ps -ef | grep hudson}}} to get the {{{build-test.sh}}} or {{{ant}}} process
id
+          1. {{{ps -ef | grep hudson}}} to get the {{{hudsonBuildHadoopNightly.sh}}} or {{{ant}}}
process id
-          1. {{{kill [-9] <build-test.sh or ant process id>}}}
+          1. {{{kill [-9] <hudsonBuildHadoopNightly.sh or ant process id>}}}
-          1. {{{ps -ef | grep hudson}}} again to verify the build and test processes are
gone.  The only remaining java process should be the Tomcat process ({{{org.apache.catalina.startup.Bootstrap}}}).
+          1. {{{ps -ef | grep hudson}}} again to verify the build and test processes are
gone.
        When this process dies, Hudson will fail the build.
- 
-    1. '''Stopping a patch build:'''
-       Follow the same advice for stopping a nightly build (above) but include these additional
steps once the running patch build is terminated:
-          1. {{{cd ~hudson/hudson/jobs/Hadoop-Patch/}}}
-          1. {{{ls -al current}}} to see which directory the {{{current}}} link is pointing
to
-          1. {{{rm -rf <directory that current points to> current}}} to delete both
the directory and the {{{current}}} link
-       Once the {{{current}}} link is gone, a waiting {{{processEmail.sh}}} process will
create a new {{{current}}} link and start the next patch build.
  
     1. '''Killing a hung test:'''
        Sometimes it is desirable to just kill a hung test and allow the rest of the build
to continue. Although the build will still fail, it can sometimes provide useful information
if you can get a thread dump on the hung test and, if the build is in the first test pass,
the second pass might succeed as some tests hang only intermittently. You can tell a test
is hung by looking at the console log for the build and if the last entry is some time ago
(at least 15 minutes) it is safe to assume the test is hung.
-          1. Login to {{{lucene.zones.apache.org}}}
+          1. Login to {{{hudson.zones.apache.org}}}
           1. Switch to the {{{hudson}}} user
           1. Find the hung test: {{{ps -laxww | grep hudson | grep test.build.data | grep
-v grep'}}}
              For example:
@@ -81, +74 @@

           1. {{{kill -QUIT 17700}}} to try to get a thread dump. Although this does not always
work (because the test is just too locked up), if it does work, it can provide useful debugging
information.
           1. {{{kill -9 17700}}} to kill the hung test. If you are watching the console (and
you should be), you should see the console output resume.
  
-    1. '''Creating a login on lucene.zones.apache.org:'''
+    1. '''Creating a login on hudson.zones.apache.org and the Hudson web UI:'''
+       Committers may have a login on the hudson.zones.apache.org build machine.  Follow
[http://wiki.apache.org/general/Hudson these] directions.
-       Committers may have a login on the lucene.zones.apache.org build machine.  To create
a new user, follow these directions:
-          {{{cat /etc/passwd}}} and look for an unused userid (the 3rd column)
  
-          {{{sudo useradd -u <unique userid> -g other -d /export/home/<user name>
-m -s /usr/bin/bash <user name>}}}
- 
-          {{{sudo visudo}}} to give the new user sudo permission
- 
-          {{{sudo passwd <user name>}}} to set the new password that you will pass
along to the new user; they should change it of course
- 
-    1. '''Creating a login on Hudson:'''
-       Committers can create a Hudson login for themselves by adding an entry to the Tomcat
users list located in {{{~hudson/tools/tomcat/apache-tomcat-5.5.20/conf/tomcat-users.xml}}}.
 Simply copy and edit an existing entry.  Once your entry is added, Tomcat must be restarted
(see below for instructions to perform this task).  Once Tomcat is restarted, you can then
login to the web interface.
- 
-    1. '''Starting and shutting down Hudson (and Tomcat):'''
-       Hudson is a web app run inside of Tomcat.  '''Please only shutdown Hudson when there
are no builds in process.'''  Tomcat is started and shutdown with these scripts:
-          {{{~hudson/tools/tomcat/apache-tomcat-5.5.20/bin/startup.sh}}}
- 
-          {{{~hudson/tools/tomcat/apache-tomcat-5.5.20/bin/shutdown.sh}}}
- 
-    1. '''Installing a new version of Hudson:'''
-       The {{{hudson.war}}} file goes in {{{~hudson/tools/tomcat/apache-tomcat-5.5.20/webapp/}}}
-          1. Shutdown Hudson
-          1. Download the new {{{hudson.war}}} file
-          1. Save the old {{{hudson.war}}} file
-          1. Copy in the new {{{hudson.war}}} file into the {{{webapp}}} directory
-          1. Startup Hudson
- 
-    1. '''Installing a new plugin:'''
-       Hudson plugins extend the functionality of Hudson.  Plugins are used to collect and
present data for Findbugs and Checkstyle, to name a few.  These plugins are packaged as {{{.hpi}}}
files and are available from [https://hudson.dev.java.net/].  The plugin file goes in {{{~hudson/hudson/plugins/}}}.
 To install or upgrade a plugin, follow these steps:
-          1. Download the new plugin file
-          1. Save the old plugin file
-          1. Copy in the new plugin file into the {{{plugins}}} directory
-          1. Shutdown and startup Hudson
- 

Mime
View raw message