synapse-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hubert, Eric" <>
Subject RE: Advice on choosing http core NIO version for Synapse 1.2
Date Sun, 06 Dec 2009 09:54:13 GMT
> I cannot think of any good reason to continue using HttpCore older than
> 4.0.1. Upgrade would be quite recommended.
Oleg, thanks a lot for your quick response and suggestion. I think we will update our integration
system to 4.0.1 first, and see how it runs there.

> The process of an I/O reactor shutdown is quite complex and therefore
> lots of things can go sideways if a particular I/O worker terminates
> abnormally due to a runtime or a fatal I/O error. To help deal with
> postmortem analysis I/O reactors maintain so called exception log that
> contains all exceptions thrown in the process of reactor shutdown
> including the one that triggered it.
> Whenever an I/O reactor terminates it is advisable to examine the audit
> log and if it contains any entries print them out to the application log.

Maybe I have to have a closer look at the code level for both Synapse and http Core to get
a better understanding... So please excuse, if my following thoughts are incorrect and rather
correct them.

Don't we generally have to differentiate between 
1) situations the code using the http nio library initiates/requests a IOReactor shutdown
2) those, where the http nio library does this on its own based on an IOReactorException?

For the first case I now understand, that one should always check the audit log in order to
also find out if there have been exceptions during the shutdown process. AFAIK Synapse currently
does not follow this suggestion.

For the second case I understood that the client code has a chance to intercept the exception
handling by providing an IOReactorExceptionHandler which can attempt to decide which exceptions
shall be considered recoverable and which not. This is the place where the client can/should
also log the cause of both IOExceptions and RuntimeExceptions. For Synapse this is the case.
Anyway it looks like there are situations where IOReactorExceptions are thrown at other places
without a chance for the client to "intercept". In this cases the client code finds out that
the IOReactor has been shutdown or is in the process of being shutdown (IllegalStateException
while calling connect()) and can just attempt to restart it.

Is it possible, that the actual cause of an IOReactor shutdown will be logged a couple of
time after it actually occurred?

Or to be more concrete, first error output:
ERROR 2009-12-02 10:50:34,802 [HttpClientWorker-97][][] org.apache.synapse.core.axis2.Axis2Sender
'Unexpected error during sending message out'
java.lang.IllegalStateException: I/O reactor has been shut down
        at org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor.connect(
        at org.apache.synapse.transport.nhttp.HttpCoreNIOSender.sendAsyncRequest(

[...  a few 100 log lines later ...]

FATAL 2009-12-02 10:50:35,031 [HttpCoreNIOSender][][] org.apache.synapse.transport.nhttp.HttpCoreNIOSender
'Encountered an I/O error: Failure registering channel with the selector'
org.apache.http.nio.reactor.IOReactorException: Failure registering channel with the selector
        at org.apache.http.impl.nio.reactor.AbstractIOReactor.processNewChannels(
        at org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(
        at org.apache.http.impl.nio.reactor.BaseIOReactor.execute(
        at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$
Caused by: java.nio.channels.ClosedChannelException
        at java.nio.channels.spi.AbstractSelectableChannel.configureBlocking(
        at org.apache.http.impl.nio.reactor.AbstractIOReactor.processNewChannels(
        ... 4 more

Was in this case the java.nio.channels.ClosedChannelException the cause of the IOReactor shutdown?
Was it correct to terminate the Reactor? If some code within the core nio library decides
to throw an IOReactorException could it make sense to log the cause immediately or are there
cases where the code at higher levels may still consider it otherwise.

Anyway, if I'm not wrong and all of the above makes at least partially sense, than it looks
to me that the fix in
which went in beta-3 should have prevented the IOReactor restart.

Oleg, please correct me if my understanding is wrong. I'm not very familiar with the code.


View raw message