ant-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Barry Pape" <bp...@netspend.com>
Subject RE: Exec Connection Not Closing
Date Wed, 24 Feb 2010 19:23:24 GMT
Antoine,

Thanks for looking at my issue and for all the advice you've given me.

It appears that we have resolved this issue by adding the -n switch to
the exec command:
<exec executable="ssh" logError="true">
                        <arg line="-n ${remote.host} ${sudo.cmd} -u
${username} /usr/local/qa_admin/deploy_build.sh ${deploy.param}" />
                </exec>


As far as how we have this setup, this task resides in the CruiseControl
build.xml file and is executed through an antpublisher in the
CruiseControl config file.

Thanks,
Barry


-----Original Message-----
From: Antoine Levy-Lambert [mailto:antoine@gmx.de] 
Sent: Monday, February 22, 2010 1:17 PM
To: Ant Users List
Subject: Re: Exec Connection Not Closing

Barry Pape wrote:
> I changed the ant task to use sshexec instead of exec with ssh as the
> executable.  I still have the seemingly random hung process.  I
executed
> the ant task from outside cruisecontrol and it completed successfully.
> The last time I found a hung process I grabbed a thread dump and am
> including it here.  Thanks for the suggestions and if there are any
> other ideas I'd love to hear them.
>   

Hello Barry,

Do I understand well that in your case CruiseControl is running ant
within its own process ?

If this is the case, there is a way to make CruiseControl start ant as
an external process, I think using a script attribute on the ant
builder.
Try this, it might help.

Also, there might be an underlying bug, either in CruiseControl or in
Ant. Opening a ticket either in CruiseControl's JIRA or in ant's
bugzilla depending on which application you think is misbehaving might
help.
If CruiseControl is starting ant within the CruiseControl process, I
would say it is primarily a CruiseControl bug.

Regards,

Antoine
> Thanks,
> Barry
>
> [cc]Feb-22 11:12:13 ScriptRunner  - 2010-02-22 11:12:13
> [cc]Feb-22 11:12:13 ScriptRunner  - Full thread dump Java HotSpot(TM)
> Server VM (14.2-b01 mixed mode):
> [cc]Feb-22 11:12:13 ScriptRunner  -
> [cc]Feb-22 11:12:13 ScriptRunner  - "Thread-3" prio=3 tid=0x08c5b400
> nid=0x12 sleeping[0xb667c000]
> [cc]Feb-22 11:12:13 ScriptRunner  -    java.lang.Thread.State:
> TIMED_WAITING (sleeping)
> [cc]Feb-22 11:12:13 ScriptRunner  -     at
java.lang.Thread.sleep(Native
> Method)
> [cc]Feb-22 11:12:13 ScriptRunner  -     at
>
org.apache.tools.ant.taskdefs.optional.ssh.SSHExec$1.run(SSHExec.java:15
> 8)
> [cc]Feb-22 11:12:13 ScriptRunner  -
> [cc]Feb-22 11:12:13 ScriptRunner  - "Connect thread
> testb-pub-web01.test.aus.netspend.net session" prio=3 tid=0x08830800
> nid=0x11 runnable [0xb65da000]
> [cc]Feb-22 11:12:13 ScriptRunner  -    java.lang.Thread.State:
RUNNABLE
> [cc]Feb-22 11:12:13 ScriptRunner  -     at
> java.net.SocketInputStream.socketRead0(Native Method)
> [cc]Feb-22 11:12:13 ScriptRunner  -     at
> java.net.SocketInputStream.read(SocketInputStream.java:129)
> [cc]Feb-22 11:12:13 ScriptRunner  -     at
> com.jcraft.jsch.IO.getByte(IO.java:82)
> [cc]Feb-22 11:12:13 ScriptRunner  -     at
> com.jcraft.jsch.Session.read(Session.java:811)
> [cc]Feb-22 11:12:13 ScriptRunner  -     at
> com.jcraft.jsch.Session.run(Session.java:1195)
> [cc]Feb-22 11:12:13 ScriptRunner  -     at
> java.lang.Thread.run(Thread.java:619)
> [cc]Feb-22 11:12:13 ScriptRunner  -
> [cc]Feb-22 11:12:13 ScriptRunner  - "Low Memory Detector" daemon
prio=3
> tid=0x08140000 nid=0xb runnable [0x00000000]
> [cc]Feb-22 11:12:13 ScriptRunner  -    java.lang.Thread.State:
RUNNABLE
> [cc]Feb-22 11:12:13 ScriptRunner  -
> [cc]Feb-22 11:12:13 ScriptRunner  - "CompilerThread1" daemon prio=3
> tid=0x0813d000 nid=0xa waiting on condition [0x00000000]
> [cc]Feb-22 11:12:13 ScriptRunner  -    java.lang.Thread.State:
RUNNABLE
> [cc]Feb-22 11:12:13 ScriptRunner  -
> [cc]Feb-22 11:12:13 ScriptRunner  - "CompilerThread0" daemon prio=3
> tid=0x0813a800 nid=0x9 waiting on condition [0x00000000]
> [cc]Feb-22 11:12:13 ScriptRunner  -    java.lang.Thread.State:
RUNNABLE
> [cc]Feb-22 11:12:13 ScriptRunner  -
> [cc]Feb-22 11:12:13 ScriptRunner  - "Signal Dispatcher" daemon prio=3
> tid=0x08139000 nid=0x8 waiting on condition [0x00000000][cc]Feb-22
> 11:12:13 ScriptRunner  -    java.lang.Thread.State: RUNNABLE
> [cc]Feb-22 11:12:13 ScriptRunner  -
> [cc]Feb-22 11:12:13 ScriptRunner  - "Finalizer" daemon prio=3
> tid=0x08124c00 nid=0x7 in Object.wait() [0xfb0ab000]
> [cc]Feb-22 11:12:13 ScriptRunner  -    java.lang.Thread.State: WAITING
> (on object monitor)
> [cc]Feb-22 11:12:13 ScriptRunner  -     at
java.lang.Object.wait(Native
> Method)
> [cc]Feb-22 11:12:13 ScriptRunner  -     - waiting on <0xbaf16bc0> (a
> java.lang.ref.ReferenceQueue$Lock)
> [cc]Feb-22 11:12:13 ScriptRunner  -     at
> java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:118)
> [cc]Feb-22 11:12:13 ScriptRunner  -     - locked <0xbaf16bc0> (a
> java.lang.ref.ReferenceQueue$Lock)
> [cc]Feb-22 11:12:13 ScriptRunner  -     at
> java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:134)
> [cc]Feb-22 11:12:13 ScriptRunner  -     at
> java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159)
> [cc]Feb-22 11:12:13 ScriptRunner  -
> [cc]Feb-22 11:12:13 ScriptRunner  - "Reference Handler" daemon prio=3
> tid=0x08123400 nid=0x6 in Object.wait() [0xfb0fc000]
> [cc]Feb-22 11:12:13 ScriptRunner  -    java.lang.Thread.State: WAITING
> (on object monitor)
> [cc]Feb-22 11:12:13 ScriptRunner  -     at
java.lang.Object.wait(Native
> Method)
> [cc]Feb-22 11:12:13 ScriptRunner  -     - waiting on <0xbb097080> (a
> java.lang.ref.Reference$Lock)
> [cc]Feb-22 11:12:13 ScriptRunner  -     at
> java.lang.Object.wait(Object.java:485)
> [cc]Feb-22 11:12:14 ScriptRunner  -     at
> java.lang.ref.Reference$ReferenceHandler.run(Reference.java:116)
> [cc]Feb-22 11:12:14 ScriptRunner  -     - locked <0xbb097080> (a
> java.lang.ref.Reference$Lock)
> [cc]Feb-22 11:12:14 ScriptRunner  -
> [cc]Feb-22 11:12:14 ScriptRunner  - "main" prio=3 tid=0x08070800
nid=0x2
> in Object.wait() [0xfe3dd000]
> [cc]Feb-22 11:12:14 ScriptRunner  -    java.lang.Thread.State: WAITING
> (on object monitor)
> [cc]Feb-22 11:12:14 ScriptRunner  -     at
java.lang.Object.wait(Native
> Method)
> [cc]Feb-22 11:12:14 ScriptRunner  -     - waiting on <0xf7f41278> (a
> org.apache.tools.ant.taskdefs.optional.ssh.SSHExec$1)
> [cc]Feb-22 11:12:14 ScriptRunner  -     at
> java.lang.Thread.join(Thread.java:1143)
> [cc]Feb-22 11:12:14 ScriptRunner  -     - locked <0xf7f41278> (a
> org.apache.tools.ant.taskdefs.optional.ssh.SSHExec$1)
> [cc]Feb-22 11:12:14 ScriptRunner  -     at
>
org.apache.tools.ant.taskdefs.optional.ssh.SSHExec.execute(SSHExec.java:
> 167)
> [cc]Feb-22 11:12:14 ScriptRunner  -     at
> org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:288)
> [cc]Feb-22 11:12:14 ScriptRunner  -     at
> sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
> [cc]Feb-22 11:12:14 ScriptRunner  -     at
>
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessor
> Impl.java:25)
> [cc]Feb-22 11:12:14 ScriptRunner  -     at
> java.lang.reflect.Method.invoke(Method.java:597)
> [cc]Feb-22 11:12:14 ScriptRunner  -     at
>
org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:1
> 05)
> [cc]Feb-22 11:12:14 ScriptRunner  -     at
> org.apache.tools.ant.Task.perform(Task.java:348)
> [cc]Feb-22 11:12:14 ScriptRunner  -     at
> org.apache.tools.ant.taskdefs.Sequential.execute(Sequential.java:62)
> [cc]Feb-22 11:12:14 ScriptRunner  -     at
> org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:288)
> [cc]Feb-22 11:12:14 ScriptRunner  -     at
> sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
> [cc]Feb-22 11:12:14 ScriptRunner  -     at
>
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessor
> Impl.java:25)
> [cc]Feb-22 11:12:14 ScriptRunner  -     at
> java.lang.reflect.Method.invoke(Method.java:597)
> [cc]Feb-22 11:12:14 ScriptRunner  -     at
>
org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:1
> 05)
> [cc]Feb-22 11:12:14 ScriptRunner  -     at
> org.apache.tools.ant.Task.perform(Task.java:348)
> [cc]Feb-22 11:12:14 ScriptRunner  -     at
>
org.apache.tools.ant.taskdefs.MacroInstance.execute(MacroInstance.java:3
> 91)
> [cc]Feb-22 11:12:14 ScriptRunner  -     at
> org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:288)
> [cc]Feb-22 11:12:14 ScriptRunner  -     at
> sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
> [cc]Feb-22 11:12:14 ScriptRunner  -     at
>
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessor
> Impl.java:25)
> [cc]Feb-22 11:12:14 ScriptRunner  -     at
> java.lang.reflect.Method.invoke(Method.java:597)
> [cc]Feb-22 11:12:14 ScriptRunner  -     at
>
org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:1
> 05)
> [cc]Feb-22 11:12:14 ScriptRunner  -     at
> org.apache.tools.ant.Task.perform(Task.java:348)
> [cc]Feb-22 11:12:14 ScriptRunner  -     at
> org.apache.tools.ant.taskdefs.Sequential.execute(Sequential.java:62)
> [cc]Feb-22 11:12:14 ScriptRunner  -     at
> net.sf.antcontrib.logic.IfTask.execute(IfTask.java:217)
> [cc]Feb-22 11:12:14 ScriptRunner  -     at
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> [cc]Feb-22 11:12:14 ScriptRunner  -     at
>
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.jav
> a:39)
> [cc]Feb-22 11:12:14 ScriptRunner  -     at
>
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessor
> Impl.java:25)
> [cc]Feb-22 11:12:14 ScriptRunner  -     at
> java.lang.reflect.Method.invoke(Method.java:597)
> [cc]Feb-22 11:12:14 ScriptRunner  -     at
>
org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:1
> 05)
> [cc]Feb-22 11:12:14 ScriptRunner  -     at
> org.apache.tools.ant.TaskAdapter.execute(TaskAdapter.java:134)
> [cc]Feb-22 11:12:14 ScriptRunner  -     at
> org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:288)
> [cc]Feb-22 11:12:14 ScriptRunner  -     at
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> [cc]Feb-22 11:12:14 ScriptRunner  -     at
>
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.jav
> a:39)
> [cc]Feb-22 11:12:14 ScriptRunner  -     at
>
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessor
> Impl.java:25)
> [cc]Feb-22 11:12:14 ScriptRunner  -     at
> java.lang.reflect.Method.invoke(Method.java:597)
> [cc]Feb-22 11:12:14 ScriptRunner  -     at
>
org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:1
> 05)
> [cc]Feb-22 11:12:14 ScriptRunner  -     at
> org.apache.tools.ant.Task.perform(Task.java:348)
> [cc]Feb-22 11:12:14 ScriptRunner  -     at
> org.apache.tools.ant.Target.execute(Target.java:357)
> [cc]Feb-22 11:12:14 ScriptRunner  -     at
> org.apache.tools.ant.Target.performTasks(Target.java:385)
> [cc]Feb-22 11:12:14 ScriptRunner  -     at
> org.apache.tools.ant.Project.executeSortedTargets(Project.java:1329)
> [cc]Feb-22 11:12:14 ScriptRunner  -     at
> org.apache.tools.ant.Project.executeTarget(Project.java:1298)
> [cc]Feb-22 11:12:14 ScriptRunner  -     at
>
org.apache.tools.ant.helper.DefaultExecutor.executeTargets(DefaultExecut
> or.java:41)[cc]Feb-22 11:12:14 ScriptRunner  -     at
> org.apache.tools.ant.Project.executeTargets(Project.java:1181)
> [cc]Feb-22 11:12:14 ScriptRunner  -     at
> org.apache.tools.ant.Main.runBuild(Main.java:698)
> [cc]Feb-22 11:12:14 ScriptRunner  -     at
> org.apache.tools.ant.Main.startAnt(Main.java:199)
> [cc]Feb-22 11:12:14 ScriptRunner  -     at
> org.apache.tools.ant.launch.Launcher.run(Launcher.java:257)
> [cc]Feb-22 11:12:14 ScriptRunner  -     at
> org.apache.tools.ant.launch.Launcher.main(Launcher.java:104)
> [cc]Feb-22 11:12:14 ScriptRunner  -
> [cc]Feb-22 11:12:14 ScriptRunner  - "VM Thread" prio=3 tid=0x0811f400
> nid=0x5 runnable
> [cc]Feb-22 11:12:14 ScriptRunner  -
> [cc]Feb-22 11:12:14 ScriptRunner  - "GC task thread#0 (ParallelGC)"
> prio=3 tid=0x08076c00 nid=0x3 runnable
> [cc]Feb-22 11:12:14 ScriptRunner  -
> [cc]Feb-22 11:12:14 ScriptRunner  - "GC task thread#1 (ParallelGC)"
> prio=3 tid=0x08078400 nid=0x4 runnable
> [cc]Feb-22 11:12:14 ScriptRunner  -
> [cc]Feb-22 11:12:14 ScriptRunner  - "VM Periodic Task Thread" prio=3
> tid=0x08141800 nid=0xc waiting on condition
> [cc]Feb-22 11:12:14 ScriptRunner  -
> [cc]Feb-22 11:12:14 ScriptRunner  - JNI global references: 1235
> [cc]Feb-22 11:12:14 ScriptRunner  -
> [cc]Feb-22 11:12:14 ScriptRunner  - Heap
> [cc]Feb-22 11:12:14 ScriptRunner  -  PSYoungGen      total 67136K,
used
> 55220K [0xf3e00000, 0xfb000000, 0xfb000000)
> [cc]Feb-22 11:12:14 ScriptRunner  -   eden space 64640K, 81% used
> [0xf3e00000,0xf71b52b8,0xf7d20000)
> [cc]Feb-22 11:12:14 ScriptRunner  -   from space 2496K, 91% used
> [0xf7d20000,0xf7f58010,0xf7f90000)
> [cc]Feb-22 11:12:14 ScriptRunner  -   to   space 2752K, 0% used
> [0xfad50000,0xfad50000,0xfb000000)
> [cc]Feb-22 11:12:14 ScriptRunner  -  PSOldGen        total 69632K,
used
> 2971K [0xbae00000, 0xbf200000, 0xf3e00000)
> [cc]Feb-22 11:12:14 ScriptRunner  -   object space 69632K, 4% used
> [0xbae00000,0xbb0e6d80,0xbf200000)
> [cc]Feb-22 11:12:14 ScriptRunner  -  PSPermGen       total 16384K,
used
> 7839K [0xb6e00000, 0xb7e00000, 0xbae00000)
> [cc]Feb-22 11:12:14 ScriptRunner  -   object space 16384K, 47% used
> [0xb6e00000,0xb75a7d08,0xb7e00000)
> [cc]Feb-22 11:12:14 ScriptRunner  -
>
>
> -----Original Message-----
> From: Antoine Levy Lambert [mailto:antoine@gmx.de] 
> Sent: Saturday, February 13, 2010 4:36 PM
> To: Ant Users List
> Subject: Re: Exec Connection Not Closing
>
> Barry Pape wrote:
>   
>> Hi Everyone,
>>
>>  
>>
>> I seem to have an issue with an exec task not closing the ssh
>>     
> connection
>   
>> it opens.  Basically we use CruiseControl and Ant to build and deploy
>> our code.  In the CruiseControl build.xml I have an ant task that
>>     
> ssh's
>   
>> to a remote server and starts a shell script.  Once the script
>>     
> completes
>   
>> CruiseControl continues along its merry way...until recently.  Now at
>> seemingly random times, one of these ssh connections will not close
>>     
> thus
>   
>> causing CruiseControl to hang.  If I kill the process, CruiseControl
>> continues like normal.
>>
>>  
>>   
>>     
> I am using CruiseControl and never saw CruiseControl hanging because a

> build is hanging. However if CruiseControl is configured with only
> one thread, only one build can run at one time.
>   
>> The ant task is:
>>
>>                 <exec executable="ssh" logError="true">
>>
>>                         <arg line="${remote.host} ${sudo.cmd} -u
>> ${username} /usr/local/qa/deploy_build.sh ${deploy.param}" />
>>
>>                 </exec>
>>
>>   
>>     
> You could use the <sshexec/> ant task instead which is shipping with 
> ant, and requires jsch.
> Maybe the underlying problem is that sudo is now configured to request
a
>
> password ? or your remote user is expired ?
> You should try to run this ssh command by hand to see what is wrong.
>   
>>  
>>
>> This had been working fine for the past year.  Now we are having
>>     
> issues.
>   
>> It started when we migrated to a new build server.  It has a more
>>     
> recent
>   
>> OS (solaris 2.10 instead of 2.9).
>>
>>  
>>
>> I tried adding the timeout attribute to the exec task, but all that
>> seemed to do was hide the stuck process.
>>
>>  
>>
>> Any thoughts or suggestions are much appreciated!
>>
>>  
>>
>> Thanks,
>>
>> Barry
>>
>>   
>>     


---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@ant.apache.org
For additional commands, e-mail: user-help@ant.apache.org

Confidentiality Notice! This electronic transmission and any attached documents or other writings
are confidential and are for the sole use of the intended recipient(s) identified above. This
message may contain information that is privileged, confidential or otherwise protected from
disclosure under applicable law. If the receiver of this information is not the intended recipient,
or the employee, or agent responsible for delivering the information to the intended recipient,
you are hereby notified that any use, reading, dissemination, distribution, copying or storage
of this information is strictly prohibited. If you have received this information in error,
please notify the sender by return email and delete the electronic transmission, including
all attachments from your system.


---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@ant.apache.org
For additional commands, e-mail: user-help@ant.apache.org


Mime
View raw message