commons-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Martin Gainty <mgai...@hotmail.com>
Subject Re: Proper use of Executors
Date Sun, 28 May 2017 21:05:23 GMT

MG>below


________________________________
From: Siegfried Goeschl <siegfried.goeschl@it20one.com>
Sent: Sunday, May 28, 2017 3:41 PM
To: Commons Users List
Cc: cgamache@gmail.com
Subject: Re: Proper use of Executors

Hi Chris,

there are couple of things to consider

* You are using a PumpStreamHander but the STDERR is not consumed. Each process has an internal
buffer (size depends on the OS) and when the buffer is full any write to STDERR is blocked
* That could happen if the process being executed actually writes some error messages :-)
* Are you 100% sure that the processes will terminate? See ExecuteWatchdog
* You might habe a look at ProcessDestroyer to cleanup during shutdown
MG>correct ThreadMonitor needs a watchdog for timeout e.g.

ExecuteWatchdog watchDog = new ExecuteWatchdog(60L * 60L * 1000L);

MG>once timeout is achieved you will need ProcessDestroyer to kill the process

 ShutdownHookProcessDestroyer processDestroyer = new ShutdownHookProcessDestroyer();

MG>Executor needs to identify its watchdog and ProcessDestroyer

executor.setStreamHandler(streamHandler); //PumpStreamHandler
executor.setWorkingDirectory(new File(getExecutionDirectory())); //which directory to write
to
executor.setProcessDestroyer(processDestroyer); //what is processDestroyer
executor.setWatchdog(watchDog); //identify watchdog


 // Async Executor  doesn't block.
    DefaultExecuteResultHandler resultHandler = new DefaultExecuteResultHandler();

//Executor HashMap

// Inherit the current process's environment variables and add the user-defined ones
    @SuppressWarnings("unchecked")
    Map<String, String> newEnvironment = EnvironmentUtils.getProcEnvironment();
    newEnvironment.putAll(this.environment);


//place to stuff the results

DefaultExecuteResultHandler resultHandler = new DefaultExecuteResultHandler();

//use executor to execute commandLine, environment HashHap

executor.execute(command, newEnvironment, resultHandler);

//return results

return resultHandler;

http://www.programcreek.com/java-api-examples/index.php?api=org.apache.commons.exec.DefaultExecuteResultHandler
Java Code Example org.apache.commons.exec ...<http://www.programcreek.com/java-api-examples/index.php?api=org.apache.commons.exec.DefaultExecuteResultHandler>
www.programcreek.com
This page provides Java code examples for org.apache.commons.exec.DefaultExecuteResultHandler.
The examples are extracted from open source Java projects from GitHub.




Thanks in advance,

Siegfried Goeschl


> On 27 May 2017, at 14:27, Chris Gamache <cgamache@gmail.com> wrote:
>
> Hi all,
>
> I'm using org.apache.commons:commons-exec:1.3 on Java 8.
>
> I'm having an issue where my Tomcat server is bleeding out hundreds of
> threads and all of the memory in the form of Executors that I'm running but
> don't seem to be closing down ... When the server finally grinds to a halt
> I have to restart. When I do it looks like this at shutdown time:
>
> <snip>
>
> 27-May-2017 07:56:21.631 WARNING [localhost-startStop-11]
> org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The
> web application [ROOT##000252] appears to have started a thread named [Exec
> Default Executor] but has failed to stop it. This is very likely to create
> a memory leak. Stack trace of thread:
> java.lang.Object.wait(Native Method)
> java.lang.Object.wait(Object.java:502)
> java.lang.UNIXProcess.waitFor(UNIXProcess.java:396)
> org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:364)
> org.apache.commons.exec.DefaultExecutor.access$200(DefaultExecutor.java:48)
> org.apache.commons.exec.DefaultExecutor$1.run(DefaultExecutor.java:200)
> java.lang.Thread.run(Thread.java:745)
>
> 27-May-2017 07:56:21.633 WARNING [localhost-startStop-11]
> org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The
> web application [ROOT##000252] appears to have started a thread named [Exec
> Stream Pumper] but has failed to stop it. This is very likely to create a
> memory leak. Stack trace of thread:
> java.io.FileInputStream.readBytes(Native Method)
> java.io.FileInputStream.read(FileInputStream.java:255)
> java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
> java.io.BufferedInputStream.read1(BufferedInputStream.java:286)
> java.io.BufferedInputStream.read(BufferedInputStream.java:345)
> java.io.FilterInputStream.read(FilterInputStream.java:107)
> org.apache.commons.exec.StreamPumper.run(StreamPumper.java:107)
> java.lang.Thread.run(Thread.java:745)
>
> </snip>
>
> And my thread dump is a mile long.
>
> I am certainly willing to concede I'm Doing It Wrong(tm) ... Here's the
> relevant code. It is called from a regular method in a regular class,
> nothing fancy:
>
> CommandLine cmdLine = CommandLine.parse(command.toString());
> DefaultExecutor executor = new DefaultExecutor();
> PumpStreamHandler esh = new PumpStreamHandler(os,null,is);
> executor.setStreamHandler(esh);
> executor.execute(cmdLine);
>
> `is` and `os` are passed in on the constructor. Their opens and closes are
> managed well and cleaned up on the outside of this class...
> Are there further steps I'm missing to ensure the threads I'm creating are
> getting shut down properly and the resources they are using are being
> returned?
>
> Any help is much appreciated.


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message