hc-httpclient-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From <kenneth.s.bro...@chase.com>
Subject Infinite loop in ChunkedInputStream.close
Date Thu, 19 Nov 2009 03:30:04 GMT
We are using httpclient 3.1 to send a post and get back serialized java
objects (think of something analogous to Spring Remoting).
We deploy this client code to many client machines talking thru
loadbalancers communicating with Weblogic 10MP1 servers.
The clients in this test are on windows, running jdk5 thru mercury
loadrunner (performance testing software).

We've been testing anywhere between 45 and 90tps and the vast majority
of the transactions are successful.

About 1 transaction every half hour ends up with a
java.net.SocketException: Connection reset.
The loadbalancer will close the socket after 60 seconds.
The reason we get that exception (and not a different one) is that we
have the client timeout set up higher than 60 seconds (just for trying
to expose this issue) and the loadbalancer has a 60 second timeout.

Here is what we have learned so far.

We get the response body as stream:
            InputStream responseBody =
postMethod.getResponseBodyAsStream();

We then send it thru a custom deserializer, we know that the object in
the response is being read because the log right after it is being
printed out:
However we have a close on that input stream in the finally. I'm
probably going to remove that close (since I read elsewhere that its
taken care of by the postMethod.releaseConnection().
        try {
            Object object = ois.readObject();
            LOG.debug(METHOD_NAME + "ois.readObject completed");
            return object;
        } finally {
            ois.close();
            LOG.debug(METHOD_NAME + "ois.close completed");
        }

The ois.close() is calling AutoCloseInputStream.close, calling
ChunkedInputStream.close which is trying to exhaust the stream by
calling exhaustInputStream which eventually calls SocketInputStream.read
and just sits there until the connection reset happens.

Any clue as to what would cause it to hang there?

The confusing factors are that the object seems to be fully read (so I
would assume there is nothing left in the stream) and we average a
couple thousand good transactions for every 1 occurrence of a bad one.

Let me know if I need the full headers and wire. We have only the
header, not the content turned on because the volume of logging just
kills the clients when turned up to trace.
Also, we will be looking to migrate eventually to httpclient 4, but that
requires jdk5 and right now we have clients that are still
(unfortunately) jdk1.4 so we can't quite make that leap yet.
I have network captures as well, if you think you know what I should
look for please advise and I will do so.

Thanks,
Ken

Here is the pertinent httpclient log.
Notice the 1 minute gap in log statements between 15:54:25:723 EST and
15:55:25:756 EST
Also be aware that the reason you see logs in between the issue and the
time you see the stacktrace is that the postMethod.releaseConnection()
is in a finally, so of course it will complete before the exception is
propogated.

System.err: 2009/11/17 15:54:25:707 EST [TRACE] PostMethod - enter
PostMethod.clearRequestBody()
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:707 EST [TRACE] EntityEnclosingMethod -
enter EntityEnclosingMethod.clearRequestBody()
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:707 EST [DEBUG]
RemoteServiceExecutorImpl - executeRequest calling
httpClient.ExecuteMethod
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:707 EST [TRACE] HttpClient - enter
HttpClient.executeMethod(HttpMethod)
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:707 EST [TRACE] HttpClient - enter
HttpClient.executeMethod(HostConfiguration,HttpMethod,HttpState)
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:707 EST [TRACE]
MultiThreadedHttpConnectionManager - enter
HttpConnectionManager.getConnectionWithTimeout(HostConfiguration, long)
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:707 EST [DEBUG]
MultiThreadedHttpConnectionManager -
HttpConnectionManager.getConnection:  config =
HostConfiguration[host=http://my-loadbalancer:9810], timeout = 0
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:707 EST [TRACE]
MultiThreadedHttpConnectionManager - enter
HttpConnectionManager.ConnectionPool.getHostPool(HostConfiguration)
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:707 EST [TRACE]
MultiThreadedHttpConnectionManager - enter
HttpConnectionManager.ConnectionPool.getHostPool(HostConfiguration)
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:707 EST [DEBUG]
MultiThreadedHttpConnectionManager - Allocating new connection,
hostConfig=HostConfiguration[host=http://my-loadbalancer:9810]
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:707 EST [TRACE] HttpMethodDirector -
Attempt number 1 to process request
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:707 EST [TRACE] HttpConnection - enter
HttpConnection.open()
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:707 EST [DEBUG] HttpConnection - Open
connection to my-loadbalancer:9810
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:707 EST [TRACE] HttpMethodBase - enter
HttpMethodBase.execute(HttpState, HttpConnection)
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:707 EST [TRACE] HttpMethodBase - enter
HttpMethodBase.writeRequest(HttpState, HttpConnection)
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:707 EST [TRACE] HttpMethodBase - enter
HttpMethodBase.writeRequestLine(HttpState, HttpConnection)
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:707 EST [TRACE] HttpMethodBase - enter
HttpMethodBase.generateRequestLine(HttpConnection, String, String,
String, String)
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:707 EST [DEBUG] header - >> "POST
/cms.rcts/2009.9.1.006/ccs/prf.lr/2008.8.1/CDC/essd-remote/
HTTP/1.1[\r][\n]"
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:707 EST [TRACE] HttpConnection - enter
HttpConnection.print(String)
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:707 EST [TRACE] HttpConnection - enter
HttpConnection.write(byte[])
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:707 EST [TRACE] HttpConnection - enter
HttpConnection.write(byte[], int, int)
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:707 EST [TRACE] HttpMethodBase - enter
HttpMethodBase.writeRequestHeaders(HttpState,HttpConnection)
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:707 EST [TRACE] EntityEnclosingMethod -
enter EntityEnclosingMethod.addRequestHeaders(HttpState, HttpConnection)
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:707 EST [TRACE] ExpectContinueMethod -
enter ExpectContinueMethod.addRequestHeaders(HttpState, HttpConnection)
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:707 EST [TRACE] HttpMethodBase - enter
HttpMethodBase.addRequestHeaders(HttpState, HttpConnection)
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:707 EST [TRACE] HttpMethodBase - enter
HttpMethodBase.addUserAgentRequestHeaders(HttpState, HttpConnection)
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:707 EST [TRACE] HttpMethodBase - enter
HttpMethodBase.addHostRequestHeader(HttpState, HttpConnection)
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:707 EST [DEBUG] HttpMethodBase - Adding
Host request header
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:707 EST [TRACE] HttpMethodBase - enter
HttpMethodBase.addCookieRequestHeader(HttpState, HttpConnection)
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:707 EST [TRACE] HttpState - enter
HttpState.getCookies()
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:707 EST [TRACE] HttpMethodBase - enter
HttpMethodBase.addProxyConnectionHeader(HttpState, HttpConnection)
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:707 EST [TRACE] EntityEnclosingMethod -
enter EntityEnclosingMethod.addContentLengthRequestHeader(HttpState,
HttpConnection)
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:707 EST [TRACE] EntityEnclosingMethod -
enter EntityEnclosingMethod.getRequestContentLength()
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:707 EST [TRACE] PostMethod - enter
PostMethod.hasRequestContent()
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:707 EST [TRACE] EntityEnclosingMethod -
enter EntityEnclosingMethod.hasRequestContent()
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:707 EST [TRACE] HttpMethodBase -
HttpMethodBase.addRequestHeader(Header)
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:707 EST [DEBUG] header - >>
"Content-Type: application/x-java-serialized-object[\r][\n]"
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:707 EST [TRACE] HttpConnection - enter
HttpConnection.print(String)
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:707 EST [TRACE] HttpConnection - enter
HttpConnection.write(byte[])
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:707 EST [TRACE] HttpConnection - enter
HttpConnection.write(byte[], int, int)
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:707 EST [DEBUG] header - >>
"Content-Encoding: none[\r][\n]"
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:707 EST [TRACE] HttpConnection - enter
HttpConnection.print(String)
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:707 EST [TRACE] HttpConnection - enter
HttpConnection.write(byte[])
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:707 EST [TRACE] HttpConnection - enter
HttpConnection.write(byte[], int, int)
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:707 EST [DEBUG] header - >>
"Serialization-Type: java[\r][\n]"
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:707 EST [TRACE] HttpConnection - enter
HttpConnection.print(String)
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:707 EST [TRACE] HttpConnection - enter
HttpConnection.write(byte[])
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:707 EST [TRACE] HttpConnection - enter
HttpConnection.write(byte[], int, int)
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:707 EST [DEBUG] header - >> "User-Agent:
Jakarta Commons-HttpClient/3.1[\r][\n]"
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:707 EST [TRACE] HttpConnection - enter
HttpConnection.print(String)
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:707 EST [TRACE] HttpConnection - enter
HttpConnection.write(byte[])
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:707 EST [TRACE] HttpConnection - enter
HttpConnection.write(byte[], int, int)
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:707 EST [DEBUG] header - >> "Host:
my-loadbalancer:9810[\r][\n]"
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:707 EST [TRACE] HttpConnection - enter
HttpConnection.print(String)
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:707 EST [TRACE] HttpConnection - enter
HttpConnection.write(byte[])
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:707 EST [TRACE] HttpConnection - enter
HttpConnection.write(byte[], int, int)
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:707 EST [DEBUG] header - >>
"Content-Length: 3051[\r][\n]"
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:707 EST [TRACE] HttpConnection - enter
HttpConnection.print(String)
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:707 EST [TRACE] HttpConnection - enter
HttpConnection.write(byte[])
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:707 EST [TRACE] HttpConnection - enter
HttpConnection.write(byte[], int, int)
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:707 EST [TRACE] HttpConnection - enter
HttpConnection.writeLine()
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:707 EST [TRACE] HttpConnection - enter
HttpConnection.write(byte[])
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:707 EST [TRACE] HttpConnection - enter
HttpConnection.write(byte[], int, int)
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:707 EST [DEBUG] header - >> "[\r][\n]"
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:707 EST [TRACE] EntityEnclosingMethod -
enter EntityEnclosingMethod.writeRequestBody(HttpState, HttpConnection)
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:707 EST [TRACE] PostMethod - enter
PostMethod.hasRequestContent()
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:707 EST [TRACE] EntityEnclosingMethod -
enter EntityEnclosingMethod.hasRequestContent()
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:707 EST [TRACE] EntityEnclosingMethod -
enter EntityEnclosingMethod.getRequestContentLength()
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:707 EST [TRACE] PostMethod - enter
PostMethod.hasRequestContent()
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:707 EST [TRACE] EntityEnclosingMethod -
enter EntityEnclosingMethod.hasRequestContent()
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:723 EST [TRACE] HttpConnection - enter
HttpConnection.getRequestOutputStream()
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:723 EST [DEBUG] EntityEnclosingMethod -
Request body sent
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:723 EST [TRACE] HttpConnection - enter
HttpConnection.flushRequestOutputStream()
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:723 EST [TRACE] HttpMethodBase - enter
HttpMethodBase.readResponse(HttpState, HttpConnection)
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:723 EST [TRACE] HttpMethodBase - enter
HttpMethodBase.readStatusLine(HttpState, HttpConnection)
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:723 EST [TRACE] HttpConnection - enter
HttpConnection.readLine()
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:723 EST [TRACE] HttpParser - enter
HttpParser.readLine(InputStream, String)
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:723 EST [TRACE] HttpParser - enter
HttpParser.readRawLine()
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:723 EST [DEBUG] header - << "HTTP/1.1
200 OK[\r][\n]"
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:723 EST [DEBUG] header - << "HTTP/1.1
200 OK[\r][\n]"
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:723 EST [TRACE] HttpMethodBase - enter
HttpMethodBase.readResponseHeaders(HttpState,HttpConnection)
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:723 EST [TRACE] HttpConnection - enter
HttpConnection.getResponseInputStream()
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:723 EST [TRACE] HttpParser - enter
HeaderParser.parseHeaders(InputStream, String)
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:723 EST [TRACE] HttpParser - enter
HttpParser.readLine(InputStream, String)
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:723 EST [TRACE] HttpParser - enter
HttpParser.readRawLine()
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:723 EST [DEBUG] header - << "Date: Tue,
17 Nov 2009 20:54:25 GMT[\r][\n]"
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:723 EST [TRACE] HttpParser - enter
HttpParser.readLine(InputStream, String)
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:723 EST [TRACE] HttpParser - enter
HttpParser.readRawLine()
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:723 EST [DEBUG] header - <<
"Transfer-Encoding: chunked[\r][\n]"
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:723 EST [TRACE] HttpParser - enter
HttpParser.readLine(InputStream, String)
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:723 EST [TRACE] HttpParser - enter
HttpParser.readRawLine()
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:723 EST [DEBUG] header - <<
"Content-Type: application/x-java-serialized-object[\r][\n]"
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:723 EST [TRACE] HttpParser - enter
HttpParser.readLine(InputStream, String)
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:723 EST [TRACE] HttpParser - enter
HttpParser.readRawLine()
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:723 EST [DEBUG] header - <<
"Content-Encoding: none[\r][\n]"
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:723 EST [TRACE] HttpParser - enter
HttpParser.readLine(InputStream, String)
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:723 EST [TRACE] HttpParser - enter
HttpParser.readRawLine()
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:723 EST [DEBUG] header - <<
"Serialization-Type: java[\r][\n]"
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:723 EST [TRACE] HttpParser - enter
HttpParser.readLine(InputStream, String)
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:723 EST [TRACE] HttpParser - enter
HttpParser.readRawLine()
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:723 EST [DEBUG] header - << "[\r][\n]"
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:723 EST [TRACE] HttpMethodBase - enter
HttpMethodBase.processResponseHeaders(HttpState, HttpConnection)
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:723 EST [TRACE] HttpMethodBase - enter
HttpMethodBase.processCookieHeaders(Header[], HttpState, HttpConnection)
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:723 EST [TRACE] HttpMethodBase - enter
HttpMethodBase.readResponseBody(HttpState, HttpConnection)
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:723 EST [TRACE] HttpMethodBase - enter
HttpMethodBase.readResponseBody(HttpConnection)
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:723 EST [TRACE] HttpConnection - enter
HttpConnection.getResponseInputStream()
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:723 EST [TRACE] HttpMethodBase - enter
HttpMethodBase.canResponseHaveBody(int)
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:723 EST [TRACE] HeaderElement - enter
HeaderElement.parseElements(String)
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:723 EST [TRACE] HeaderElement - enter
HeaderElement.parseElements(char[])
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:723 EST [TRACE] HttpConnection - enter
HttpConnection.isResponseAvailable(int)
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:723 EST [DEBUG]
RemoteServiceExecutorImpl - executeRequesthttpClient.ExecuteMethod
completed
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:54:25:723 EST [DEBUG]
RemoteServiceExecutorImpl -
executeRequestpostMethod.getResponseBodyAsStream completed
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:55:25:756 EST [TRACE] HttpConnection - enter
HttpConnection.releaseConnection()
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:55:25:756 EST [DEBUG] HttpConnection -
Releasing connection back to connection manager.
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:55:25:756 EST [TRACE]
MultiThreadedHttpConnectionManager - enter
HttpConnectionManager.releaseConnection(HttpConnection)
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:55:25:756 EST [DEBUG]
MultiThreadedHttpConnectionManager - Freeing connection,
hostConfig=HostConfiguration[host=http://my-loadbalancer:9810]
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:55:25:756 EST [TRACE]
MultiThreadedHttpConnectionManager - enter
HttpConnectionManager.ConnectionPool.getHostPool(HostConfiguration)
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:55:25:756 EST [DEBUG] IdleConnectionHandler -
Adding connection at: 1258491325756
Error	[MsgId: MMSG-17999]
System.err: 2009/11/17 15:55:25:756 EST [DEBUG]
MultiThreadedHttpConnectionManager - Notifying no-one, there are no
waiting threads
Error	[MsgId: MMSG-17999]
Error: Failed to end Transaction "callTrackingService_insertWorkFlow"
(by name). Please check that you made a call to start transaction
operation.	[MsgId: MERR-16985]
Error: System.err: com.fusa.ssg.SystemException: Error making call[Error
executing Remote Method=[getCallSeqId] for Service
Url=[http://my-loadbalancer:9810/cms.rcts/2009.9.1.006/ccs/prf.lr/2008.8
.1/CDC] Exception=[java.net.SocketException: Connection reset
	at java.net.SocketInputStream.read(SocketInputStream.java:168)
	at
java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
	at
java.io.BufferedInputStream.read(BufferedInputStream.java:235)
	at
org.apache.commons.httpclient.ChunkedInputStream.getChunkSizeFromInputSt
ream(ChunkedInputStream.java:250)
	at
org.apache.commons.httpclient.ChunkedInputStream.nextChunk(ChunkedInputS
tream.java:221)
	at
org.apache.commons.httpclient.ChunkedInputStream.read(ChunkedInputStream
.java:176)
	at
org.apache.commons.httpclient.ChunkedInputStream.read(ChunkedInputStream
.java:196)
	at
org.apache.commons.httpclient.ChunkedInputStream.exhaustInputStream(Chun
kedInputStream.java:369)
	at
org.apache.commons.httpclient.ChunkedInputStream.close(ChunkedInputStrea
m.java:346)
	at java.io.FilterInputStream.close(FilterInputStream.java:159)
	at
org.apache.commons.httpclient.AutoCloseInputStream.notifyWatcher(AutoClo
seInputStream.java:194)
	at
org.apache.commons.httpclient.AutoCloseInputStream.close(AutoCloseInputS
tream.java:158)
	at
java.io.ObjectInputStream$PeekInputStream.close(ObjectInputStream.java:2
252)
	at
java.io.ObjectInputStream$BlockDataInputStream.close(ObjectInputStream.j
ava:2587)
	at java.io.ObjectInputStream.close(ObjectInputStream.java:853)
	at
com.fusa.ssg.infra.remoting.common.io.StandardJavaSerializer.readObject(
StandardJavaSerializer.java:41)
	at
com.fusa.ssg.infra.remoting.client.service.RemoteServiceExecutorImpl.des
erializeServiceResponse(RemoteServiceExecutorImpl.java:153)
	at
com.fusa.ssg.infra.remoting.client.service.RemoteServiceExecutorImpl.exe
cuteRequest(RemoteServiceExecutorImpl.java:111)
	at
com.fusa.ssg.infra.remoting.client.service.RemoteServiceProxy.invoke(Rem
oteServiceProxy.java:54)
	at $Proxy5.getCallSeqId(Unknown Source)
	at
com.ccs.cco.cms.rcts.client.calltracking.CallTrackingRemotingDelegate.ge
tCallSeqId(CallTrackingRemotingDelegate.java:66)
	at Actions.action(Actions.java:2327)
]]
Error	[MsgId: MERR-17999]
System.err: 	at
com.ccs.cco.cms.rcts.client.calltracking.CallTrackingRemotingDelegate.ge
tCallSeqId(CallTrackingRemotingDelegate.java:81)
Error	[MsgId: MMSG-17999]
System.err: 	at Actions.action(Actions.java:2327)
Error	[MsgId: MMSG-17999]
Error: System.err: ------------Wrapped Exception-----------
Error	[MsgId: MERR-17999]
Error: System.err:
com.fusa.ssg.infra.remoting.common.RemoteServiceException: Error
executing Remote Method=[getCallSeqId] for Service
Url=[http://my-loadbalancer:9810/cms.rcts/2009.9.1.006/ccs/prf.lr/2008.8
.1/CDC] Exception=[java.net.SocketException: Connection reset
	at java.net.SocketInputStream.read(SocketInputStream.java:168)
	at
java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
	at
java.io.BufferedInputStream.read(BufferedInputStream.java:235)
	at
org.apache.commons.httpclient.ChunkedInputStream.getChunkSizeFromInputSt
ream(ChunkedInputStream.java:250)
	at
org.apache.commons.httpclient.ChunkedInputStream.nextChunk(ChunkedInputS
tream.java:221)
	at
org.apache.commons.httpclient.ChunkedInputStream.read(ChunkedInputStream
.java:176)
	at
org.apache.commons.httpclient.ChunkedInputStream.read(ChunkedInputStream
.java:196)
	at
org.apache.commons.httpclient.ChunkedInputStream.exhaustInputStream(Chun
kedInputStream.java:369)
	at
org.apache.commons.httpclient.ChunkedInputStream.close(ChunkedInputStrea
m.java:346)
	at java.io.FilterInputStream.close(FilterInputStream.java:159)
	at
org.apache.commons.httpclient.AutoCloseInputStream.notifyWatcher(AutoClo
seInputStream.java:194)
	at
org.apache.commons.httpclient.AutoCloseInputStream.close(AutoCloseInputS
tream.java:158)
	at
java.io.ObjectInputStream$PeekInputStream.close(ObjectInputStream.java:2
252)
	at
java.io.ObjectInputStream$BlockDataInputStream.close(ObjectInputStream.j
ava:2587)
	at java.io.ObjectInputStream.close(ObjectInputStream.java:853)
	at
com.fusa.ssg.infra.remoting.common.io.StandardJavaSerializer.readObject(
StandardJavaSerializer.java:41)
	at
com.fusa.ssg.infra.remoting.client.service.RemoteServiceExecutorImpl.des
erializeServiceResponse(RemoteServiceExecutorImpl.java:153)
	at
com.fusa.ssg.infra.remoting.client.service.RemoteServiceExecutorImpl.exe
cuteRequest(RemoteServiceExecutorImpl.java:111)
	at
com.fusa.ssg.infra.remoting.client.service.RemoteServiceProxy.invoke(Rem
oteServiceProxy.java:54)
	at $Proxy5.getCallSeqId(Unknown Source)
	at
com.ccs.cco.cms.rcts.client.calltracking.CallTrackingRemotingDelegate.ge
tCallSeqId(CallTrackingRemotingDelegate.java:66)
	at Actions.action(Actions.java:2327)
]
Error	[MsgId: MERR-17999]
System.err: 	at
com.fusa.ssg.infra.remoting.client.service.RemoteServiceProxy.invoke(Rem
oteServiceProxy.java:58)
Error	[MsgId: MMSG-17999]
System.err: 	at $Proxy5.getCallSeqId(Unknown Source)
Error	[MsgId: MMSG-17999]
System.err: 	at
com.ccs.cco.cms.rcts.client.calltracking.CallTrackingRemotingDelegate.ge
tCallSeqId(CallTrackingRemotingDelegate.java:66)
Error	[MsgId: MMSG-17999]
System.err: 	at Actions.action(Actions.java:2327)
Error	[MsgId: MMSG-17999]
Error: System.err: -----------Wrapped Exception-----------
Error	[MsgId: MERR-17999]
Error: System.err:
com.fusa.ssg.infra.remoting.common.RemoteServiceException: Error
executing Remote Method=[getCallSeqId] for Service
Url=[http://my-loadbalancer:9810/cms.rcts/2009.9.1.006/ccs/prf.lr/2008.8
.1/CDC] Exception=[java.net.SocketException: Connection reset
	at java.net.SocketInputStream.read(SocketInputStream.java:168)
	at
java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
	at
java.io.BufferedInputStream.read(BufferedInputStream.java:235)
	at
org.apache.commons.httpclient.ChunkedInputStream.getChunkSizeFromInputSt
ream(ChunkedInputStream.java:250)
	at
org.apache.commons.httpclient.ChunkedInputStream.nextChunk(ChunkedInputS
tream.java:221)
	at
org.apache.commons.httpclient.ChunkedInputStream.read(ChunkedInputStream
.java:176)
	at
org.apache.commons.httpclient.ChunkedInputStream.read(ChunkedInputStream
.java:196)
	at
org.apache.commons.httpclient.ChunkedInputStream.exhaustInputStream(Chun
kedInputStream.java:369)
	at
org.apache.commons.httpclient.ChunkedInputStream.close(ChunkedInputStrea
m.java:346)
	at java.io.FilterInputStream.close(FilterInputStream.java:159)
	at
org.apache.commons.httpclient.AutoCloseInputStream.notifyWatcher(AutoClo
seInputStream.java:194)
	at
org.apache.commons.httpclient.AutoCloseInputStream.close(AutoCloseInputS
tream.java:158)
	at
java.io.ObjectInputStream$PeekInputStream.close(ObjectInputStream.java:2
252)
	at
java.io.ObjectInputStream$BlockDataInputStream.close(ObjectInputStream.j
ava:2587)
	at java.io.ObjectInputStream.close(ObjectInputStream.java:853)
	at
com.fusa.ssg.infra.remoting.common.io.StandardJavaSerializer.readObject(
StandardJavaSerializer.java:41)
	at
com.fusa.ssg.infra.remoting.client.service.RemoteServiceExecutorImpl.des
erializeServiceResponse(RemoteServiceExecutorImpl.java:153)
	at
com.fusa.ssg.infra.remoting.client.service.RemoteServiceExecutorImpl.exe
cuteRequest(RemoteServiceExecutorImpl.java:111)
	at
com.fusa.ssg.infra.remoting.client.service.RemoteServiceProxy.invoke(Rem
oteServiceProxy.java:54)
	at $Proxy5.getCallSeqId(Unknown Source)
	at
com.ccs.cco.cms.rcts.client.calltracking.CallTrackingRemotingDelegate.ge
tCallSeqId(CallTrackingRemotingDelegate.java:66)
	at Actions.action(Actions.java:2327)
]
Error	[MsgId: MERR-17999]
System.err: 	at
com.fusa.ssg.infra.remoting.client.service.RemoteServiceProxy.invoke(Rem
oteServiceProxy.java:58)
Error	[MsgId: MMSG-17999]
System.err: 	at $Proxy5.getCallSeqId(Unknown Source)
Error	[MsgId: MMSG-17999]
System.err: 	at
com.ccs.cco.cms.rcts.client.calltracking.CallTrackingRemotingDelegate.ge
tCallSeqId(CallTrackingRemotingDelegate.java:66)
Error	[MsgId: MMSG-17999]
System.err: 	at Actions.action(Actions.java:2327)
Error	[MsgId: MMSG-17999]










This transmission may contain information that is privileged,
confidential, legally privileged, and/or exempt from disclosure
under applicable law.  If you are not the intended recipient, you
are hereby notified that any disclosure, copying, distribution, or
use of the information contained herein (including any reliance
thereon) is STRICTLY PROHIBITED.  Although this transmission and
any attachments are believed to be free of any virus or other
defect that might affect any computer system into which it is
received and opened, it is the responsibility of the recipient to
ensure that it is virus free and no responsibility is accepted by
JPMorgan Chase & Co., its subsidiaries and affiliates, as
applicable, for any loss or damage arising in any way from its use.
 If you received this transmission in error, please immediately
contact the sender and destroy the material in its entirety,
whether in electronic or hard copy format. Thank you.
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message