hadoop-common-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Wiki <wikidi...@apache.org>
Subject [Hadoop Wiki] Update of "HudsonBuildServer" by NigelDaley
Date Tue, 23 Sep 2008 17:53:55 GMT
Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Hadoop Wiki" for change notification.

The following page has been changed by NigelDaley:
http://wiki.apache.org/hadoop/HudsonBuildServer

The comment on the change is:
Hudson server info is now maintained elsewhere.  Attempting to add a redirect.

------------------------------------------------------------------------------
- = Introducing Hudson =
+ #REFRESH 0 /general/Hudson
  
- We use [https://hudson.dev.java.net/ Hudson] as our continuous integration server:
- 
-  * [http://hudson.zones.apache.org/hudson/]
- 
- Hudson runs automated patch verification builds for Hadoop.
- 
- = Administering Hudson =
- 
- Hudson is run by the {{{hudson}}} user on {{{hudson.zones.apache.org}}}.  Any Apache committer
may have an account on this machine.  To manage the Hudson server, log into this machine and
- 
- {{{sudo su - hudson}}}
- 
- This will give you the proper shell environment to manage the Hudson server.  Hudson uses
a number of tools for itself and for the builds it runs.  All tools are located in {{{~hudson/tools}}}.
 Hudson stores all of it's config files and builds in {{{$HUDSON_HOME}}} which is set to {{{~hudson/hudson}}}.
- 
- '''Note:''' Please notify others on the Hadoop IRC channel (#hadoop at irc.freenode.net)
before you start manipulating Hudson, to make sure that you're not stepping on someone else's
toes.  Any substantial configuration changes to Hadoop builds should be discussed on the developer
mailing list.
- 
- == Tasks ==
- 
- Most tasks, such as creating a new build or modifying an existing build, are performed through
the Hudson web interface.  Once you login to Hudson, you are provided with an expanded number
of administration options.  To create a login on Hudson, see below.  The following tasks must
be performed after logging into {{{hudson.zones.apache.org}}} and switching to the {{{hudson}}}
user.
- 
-    1. '''Stopping a nightly build:'''
-       When logged into Hudson, a red X button is displayed next to each running build. 
Clicking the button will end the build as far as Hudson is concerned, but the underlying processes
sometimes don't get properly terminated.  A better approach is to kill the {{{hudsonBuildHadoopNightly.sh}}}
process or, if that doesn't exist, the {{{ant}}} process:
-          1. Login to {{{hudson.zones.apache.org}}}
-          1. Switch to the {{{hudson}}} user
-          1. {{{ps -ef | grep hudson}}} to get the {{{hudsonBuildHadoopNightly.sh}}} or {{{ant}}}
process id
-          1. {{{kill [-9] <hudsonBuildHadoopNightly.sh or ant process id>}}}
-          1. {{{ps -ef | grep hudson}}} again to verify the build and test processes are
gone.
-       When this process dies, Hudson will fail the build.
- 
-    1. '''Restarting a hung patch testing queue:'''
-       A number of issues can cause the [http://hudson.zones.apache.org/hudson/view/Hadoop/job/Hadoop-Patch-Admin/lastSuccessfulBuild/artifact/HADOOP_PatchQueue.html
queue of pending patches] to hang or appear hung.  Note that the queue page is updated by
the [http://hudson.zones.apache.org/hudson/view/Hadoop/job/Hadoop-Patch-Admin/ Hadoop-Patch-Admin]
build every five minutes.  Sometimes, this build can't run for a long period of time because
other build are running on Hudson.
-          1. '''There are patches in the queue but [http://hudson.zones.apache.org/hudson/view/Hadoop/job/Hadoop-Patch/
Hadoop-Patch] build is not running'''
-             Try kicking off the [http://hudson.zones.apache.org/hudson/view/Hadoop/job/Hadoop-Patch-Admin/
Hadoop-Patch-Admin] manually.  If that doesn't start the [http://hudson.zones.apache.org/hudson/view/Hadoop/job/Hadoop-Patch/
Hadoop-Patch] build, then kick off the [http://hudson.zones.apache.org/hudson/view/Hadoop/job/Hadoop-Patch/
Hadoop-Patch] build manually.  That should be all that is necessary in this scenario.
-          1. '''The [http://hudson.zones.apache.org/hudson/view/Hadoop/job/Hadoop-Patch/
Hadoop-Patch] is running well over 2 hours'''
-             Some test has likely hung.  See "Killing a hung test" below. 
- 
-    1. '''Killing a hung test:'''
-       Sometimes it is desirable to just kill a hung test and allow the rest of the build
to continue. Although the build will still fail, it can sometimes provide useful information
if you can get a thread dump on the hung test and, if the build is in the first test pass,
the second pass might succeed as some tests hang only intermittently. You can tell a test
is hung by looking at the console log for the build and if the last entry is some time ago
(at least 15 minutes) it is safe to assume the test is hung.
-          1. {{{ssh hudson.zones.apache.org}}}
-          1. {{{sudo su - hudson}}}
-          1. Find the hung test: {{{ps -laxww | grep hudson | grep test.build.data | grep
-v grep'}}}
-             For example:
- 
-            {{{
-  0  1000 17700  4921  0  53 20150292 1032 ffffffffbbf4f456 S ?         0:00 /export/home/hudson/tools/java/jdk1.5.0_11/jre/bin/java
-Xmx256m -Dtest.build.data=/export/home/hudson/hudson/jobs/Hadoop-Patch/workspace/trunk/build/test/data
-Dtest.cache.data=/export/home/hudson/hudson/jobs/Hadoop-Patch/workspace/trunk/build/test/cache
-Dhadoop.log.dir=/export/home/hudson/hudson/jobs/Hadoop-Patch/workspace/trunk/build/test/logs
-Dtest.src.dir=/export/home/hudson/hudson/jobs/Hadoop-Patch/workspace/trunk/src/test -Djava.library.path=/export/home/hudson/hudson/jobs/Hadoop-Patch/workspace/trunk/build/native/
- SunOS-x86-32/lib:/export/home/hudson/hudson/jobs/Hadoop-Patch/workspace/trunk/lib/native/
- SunOS-x86-32 -Dinstall.c++.examples=/export/home/hudson/hudson/jobs/Hadoop-Patch/workspace/
- trunk/build/c++-examples/SunOS-x86-32 -classpath /export/home/hudson/hudson/jobs/Hadoop-Patch/
- workspace/trunk/build/test/classes:/export/home/hudson/hudson/jobs/Hadoop-Patch/workspace/trunk/
- src/test:/export/home/hudson/hudson/jobs/Hadoop-Patch/workspace/trunk/build:/export/home/hudson/
- hudson/jobs/Hadoop-Patch/workspace/trunk/build/examples:/export/home/hudson/hudson/jobs/
- Hadoop-Patch/workspace/trunk/build/classes:/export/home/hudson/hudson/jobs/Hadoop-Patch/
- workspace/trunk/lib/commons-cli-2.0-SNAPSHOT.jar:/export/home/hudson/hudson/jobs/Hadoop-Patch/
- workspace/trunk/lib/commons-codec-1.3.jar:/export/home/hudson/hudson/jobs/Hadoop-Patch/
- workspace/trunk/lib/commons-httpclient-3.0.1.jar:/export/home/hudson/hudson/jobs/Hadoop-Patch/
- workspace/trunk/lib/commons-logging-1.0.4.jar:/export/home/hudson/hudson/jobs/Hadoop-Patch/
- workspace/trunk/lib/commons-logging-api-1.0.4.jar:/export/home/hudson/hudson/jobs/Hadoop-Patch/
- workspace/trunk/lib/jets3t-0.5.0.jar:/export/home/hudson/hudson/jobs/Hadoop-Patch/workspace/trunk/
- lib/jetty-5.1.4.jar:/export/home/hudson/hudson/jobs/Hadoop-Patch/workspace/trunk/lib/jetty-ext/
- commons-el.jar:/export/home/hudson/hudson/jobs/Hadoop-Patch/workspace/trunk/lib/jetty-ext/
- jasper-compiler.jar:/export/home/hudson/hudson/jobs/Hadoop-Patch/workspace/trunk/lib/jetty-ext/
- jasper-runtime.jar:/export/home/hudson/hudson/jobs/Hadoop-Patch/workspace/trunk/lib/jetty-ext/
- jsp-api.jar:/export/home/hudson/hudson/jobs/Hadoop-Patch/workspace/trunk/lib/junit-3.8.1.jar:
- /export/home/hudson/hudson/jobs/Hadoop-Patch/workspace/trunk/lib/log4j-1.2.13.jar:/export/home/
- hudson/hudson/jobs/Hadoop-Patch/workspace/trunk/lib/servlet-api.jar:/export/home/hudson/hudson/
- jobs/Hadoop-Patch/workspace/trunk/lib/xmlenc-0.52.jar:/export/home/hudson/hudson/jobs/
- Hadoop-Patch/workspace/trunk/conf:/export/home/hudson/tools/ant/latest/lib/junit-3.8.1.jar:/export/
- home/hudson/tools/ant/apache-ant-1.6.5/lib/ant-launcher.jar:/export/home/hudson/tools/ant/latest/lib/
- ant.jar:/export/home/hudson/tools/ant/latest/lib/ant-junit.jar 
- org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner 
- org.apache.hadoop.mapred.TestMiniMRWithDFS filtertrace=true haltOnError=false haltOnFailure=false
formatter=org.apache.tools.ant.taskdefs.optional.junit.SummaryJUnitResultFormatter showoutput=true
formatter=org.apache.tools.ant.taskdefs.optional.junit.XMLJUnitResultFormatter,/export/home/hudson/
- hudson/jobs/Hadoop-Patch/workspace/trunk/build/test/
- TEST-org.apache.hadoop.mapred.TestMiniMRWithDFS.xml propsfile=/export/home/hudson/hudson/
- jobs/Hadoop-Patch/workspace/trunk/junit1559750177.properties
-             }}}
- 
-          1. {{{kill -QUIT 17700}}} to try to get a thread dump. Although this does not always
work (because the test is just too locked up), if it does work, it can provide useful debugging
information.
-          1. {{{kill -9 17700}}} to kill the hung test. If you are watching the console (and
you should be), you should see the console output resume.
- 
-    1. '''Creating a login on hudson.zones.apache.org and the Hudson web UI:'''
-       Committers may have a login on the hudson.zones.apache.org build machine.  Follow
[http://wiki.apache.org/general/Hudson these] directions.
- 

Mime
View raw message