continuum-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Peter Janes <continuum.apache....@peterjanes.ca>
Subject Re: Continuum queuing jobs but not performing builds
Date Fri, 05 Sep 2008 15:49:33 GMT
After having this happen a third time I took a closer look at the database 
and saw that one of the entries in the BUILDRESULT table had an END_TIME of 
0.  When I changed that field to a current time and restarted the server 
Continuum started to work again.

I was able to reproduce the problem by setting the END_TIME of the last 
BUILDRESULT to 0, starting the server, and waiting for that project to be 
queued.  It was resolved again when I reset END_TIME to its original value.

I've reported this as http://jira.codehaus.org/browse/CONTINUUM-1871 and 
attached the thread dump.

Thanks,
Peter J.

Brett Porter wrote:
> I certainly haven't seen something like this - perhaps something in the
> project change caused an infinite loop that might be a bug in the Maven
> project handling.
> Are you able to get a thread dump from Continuum when it occurs?
> 
> Thanks,
> Brett
> 
> 2008/9/3 Peter Janes <continuum.apache.org@peterjanes.ca>
> 
>> Since adding a set of projects to a Continuum 1.2 instance last Thursday,
>> the server has stopped performing any builds on any projects.  The server
>> had performed hundreds of builds without issue over a period of several
>> months.
>>
>> The initial problem that appeared was that projects would be queued but
>> never started; as soon as the first job entered the queue Continuum would
>> start using 100% CPU and never start building anything.  Even when I removed
>> all of the projects from the queue it remained at 100% CPU.  Restarting the
>> server didn't help: as soon as a job was queued the CPU would spike and no
>> further progress would be made.
>>
>> Thinking that somehow the (MySQL) database might be corrupt, I removed it
>> and started fresh.  I added the projects that appeared to cause the problem
>> and they built successfully, so I started re-adding the rest of the
>> projects.  Unfortunately, at some point while I was doing this builds
>> stopped being executed again: this time they get queued up and are
>> apparently started, but I don't see any build processes being invoked.  Even
>> the simplest pom-only job times out.  The timeouts are shown in the
>> Continuum log but aren't being recorded in the database as failures or
>> errors, and restarting doesn't make a difference, which suggests to me that
>> perhaps the database has been corrupted again.
>>
>> I don't see anything in Jira or the list archives; has anyone seen
>> behaviours like this?  More importantly, has anyone been able to solve them?
>>
>> Environment: RHEL 5, Continuum 1.2 (built in May), Java 1.6.0_03, MySQL
>> 5.0.45, Maven 2.0.9
>>
>> Peter J.
>> --
>> Sometimes the Universe needs a change of perspective.
>>  --J. Michael Straczynski
>>
> 
> 
> 


-- 
Sometimes the Universe needs a change of perspective.
   --J. Michael Straczynski

Mime
View raw message