tomcat-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Laurent Petit <lpe...@yseop.com>
Subject Re: Re : Re: Issue with keep-alive connections, when using APR Connector on Windows and starting Processes from Servlets
Date Mon, 02 Jul 2012 08:59:53 GMT
On Fri, 2012-06-29 at 17:18 +0200, Jeff MAURY wrote:
> Hello,
> 
> I found the correct code to look at:
> http://svn.apache.org/repos/asf/tomcat/native/branches/1.1.x/native/src/network.c
> It does not set the inherit flag but it is using the apr library that maybe
> set the flag under Windows.
> If this is the case and according to your references, it is likely the
> client will not receive the ack and this is a problem with APR and
> Runtime.exec

Hello,

Seems like it's function apr_socket_create, defined in this APR file:

http://svn.apache.org/repos/asf/apr/apr/trunk/network_io/win32/sockets.c


which apparently already tries to create a non-inherited handle.

The code, for reasons beyond my current understanding, relies at compile
time on the flags APR_HAS_UNICODE_FS and _WIN32_WCE.

It also relies at runtime on IF_WIN_OS_IS_UNICODE, which may (?) be an
indirect indication that the underlying OS needs SetHandleInformation to
be explicitly called.


Jeff, do you have ideas/indications on how I could help push the issue
resolution forward ? We're reaching the limits of my current abilities
to help, not having a windows box with C compiler to do tests, etc.  :-(




You'll find below the relevant code:

#ifdef WIN32
    /* Socket handles are never truly inheritable, there are too many
     * bugs associated.  WSADuplicateSocket will copy them, but for our
     * purposes, always transform the socket() created as a
non-inherited
     * handle
     */
#if APR_HAS_UNICODE_FS && !defined(_WIN32_WCE)
    IF_WIN_OS_IS_UNICODE {
        /* A different approach.  Many users report errors such as 
         * (32538)An operation was attempted on something that is not 
         * a socket.  : Parent: WSADuplicateSocket failed...
         *
         * This appears that the duplicated handle is no longer
recognized
         * as a socket handle.  SetHandleInformation should overcome
that
         * problem by not altering the handle identifier.  But this
won't
         * work on 9x - it's unsupported.
         */
        SetHandleInformation((HANDLE) (*new)->socketdes, 
                             HANDLE_FLAG_INHERIT, 0);
    }
#if APR_HAS_ANSI_FS
    /* only if APR_HAS_ANSI_FS && APR_HAS_UNICODE_FS */
    ELSE_WIN_OS_IS_ANSI
#endif
#endif
#if APR_HAS_ANSI_FS || defined(_WIN32_WCE)
    {
        HANDLE hProcess = GetCurrentProcess();
        HANDLE dup;
        if (DuplicateHandle(hProcess, (HANDLE) (*new)->socketdes,
hProcess, 
                            &dup, 0, FALSE, DUPLICATE_SAME_ACCESS)) {
            closesocket((*new)->socketdes);
            (*new)->socketdes = (SOCKET) dup;
        }
    }
#endif

#endif /* def WIN32 */




> 
> Jeff
> 
> 
> On Fri, Jun 29, 2012 at 4:38 PM, Laurent Petit <lpetit@yseop.com> wrote:
> 
> > On Fri, 2012-06-29 at 15:27 +0200, Jeff MAURY wrote:
> > > No,
> > >
> > > it does not (yet) make sense because one piece of the puzzle is missing.
> > > The Local socket is not a TCP socket, it is created from a path rather
> > than
> > > from an IP address.
> > > So this socket is in no way linked to the browser, at least directly, and
> > > the missing piece is what is the relation from this socket to the
> > browser.
> >
> > You're right, so the question is still open.
> >
> > In the mean time, I'm come across interesting google search results,
> > among which the first one seems really interesting:
> >
> >
> > Oracle Bug 6428742
> > ==================
> >
> > "Currently, on Windows, Runtime.exec is implemented by calling
> > CreateProcess with the bInheritHandles argument set to TRUE,
> > which causes open handles to not be closed if they are not
> > explicitly set to be non-inherited.  This is the underlying cause
> > of many subtle bugs, such as
> >
> > 6347873: (so) Ports opened with ServerSocketChannel blocks when using
> > Runtime.exec
> > http://monaco.sfbay/detail.jsf?cr=6347873"
> >
> >
> >
> > ==> Note the "if they are not explicitly set to be non-inherited" =>
> > this seems like a game changer, since our implicit assumption was that
> > we would just look for code which would have accidentally set the handle
> > to be inherited, not the other way around.
> >
> > http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6428742
> >
> >
> >
> > QT Project
> > ==========
> >
> > "When using spawned processes, sockets opened by the client remain in
> > CLOSE_WAIT state after closing the connection. Sockets on the remote
> > side would be in FIN_WAIT_2 state and waiting for ACK.
> > When the spawned (child) process is closed, then ACK is sent and sockets
> > are finally moved to CLOSE state on both sides.
> > This seems to be caused by the default handle inheritance on Windows by
> > child processes."
> >
> > https://bugreports.qt-project.org/browse/QTBUG-4465
> >
> >
> >
> > Zeromq2 project
> > ===============
> >
> > "On Windows platform, in order to prevent child processes to inherit
> > parent sockets, SetHandleInformation
> > (
> > http://msdn.microsoft.com/en-us/library/windows/desktop/ms724935(v=vs.85).aspx)
> > should be called on the socket handle.
> >
> > This patch comes out from discussion at:
> > https://github.com/zeromq/clrzmq/issues/60"
> >
> > https://github.com/zeromq/zeromq2-x/pull/51
> > and
> > https://zeromq.jira.com/browse/LIBZMQ-366
> >
> >
> > I found way more interesting results, but I'll spare space and won't
> > paste them all in this thread
> > ( google search criteria : "windows inheriting socket handles" )
> >
> >
> > >
> > > Jeff
> > >
> > >
> > > On Fri, Jun 29, 2012 at 2:40 PM, Laurent Petit <lpetit@yseop.com> wrote:
> > >
> > > > Hello,
> > > >
> > > > On Fri, 2012-06-29 at 11:45 +0200, Jeff MAURY wrote:
> > > >
> > > > > That what I guessed but I don't understand everything.
> > > > > The code you are referencing is related to NTPipes and not sockets.
> > > >
> > > >
> > > > I'm not a C expert, but my bet is that the file name is misleading.
> > > >
> > > > Indeed, the
> > > >
> > > > con->sa.bInheritHandle = TRUE;
> > > >
> > > > statement appears in the definition driven by the following macro
> > > > expansion:
> > > >
> > > > TCN_IMPLEMENT_CALL(jlong, Local, create)(TCN_STDARGS, jstring name,
> > > >
> > > >                                         jlong pool)
> > > >
> > > > And when the macro call TCN_IMPLEMENT_CALL(jlong, Local, create) is
> > > > preprocessed, it gives:
> > > >
> > > >
> > > > JNIEXPORT RT JNICALL Java_org_apache_tomcat_jni_##Local##_##create
> > > >
> > > > , thus the bInheritHandle = TRUE statement is defined in the following
> > > > function signature:
> > > >
> > > > JNIEXPORT RT JNICALL
> > > > Java_org_apache_tomcat_jni_##Local##_##create(TCN_STDARGS, jstring
> > name,
> > > >                                         jlong pool)
> > > >
> > > > which is a function called by JNDI, and as I guess, by class
> > > > org.apache.tomcat.ini.Local, method create() :
> > > >
> > > >
> > > >
> > http://svn.apache.org/repos/asf/tomcat/native/branches/1.1.x/java/org/apache/tomcat/jni/Local.java
> > > >
> > > > And the javadoc for class Local is: "Local socket"
> > > > And for methode create :
> > > > /**
> > > >
> > > >  * Create a socket.
> > > >  * @param path The address of the new socket.
> > > >  * @param cont The parent pool to use
> > > >  * @return The new socket that has been set up.
> > > >  */
> > > >
> > > >
> > > > It seems like the native code uses a pool, and my bet is that the pool
> > > > is responsible for trying to reuse the socket.
> > > >
> > > > And something like this may happen: the socket handle, inherited by the
> > > > child process, puts the socket in such a state that the pool either
> > > > can't, either doesn't want to reuse it.
> > > >
> > > > And then what happens is that on the other side of the socket, the
> > > > client continues to write HTTP requests, and nobody is listening.
> > > >
> > > > And then when the Tomcat's child process is killed, the socket is
> > > > finally closed somehow, the client browser notices it and creates a new
> > > > connection to the server and retries the HTTP request.
> > > >
> > > >
> > > > Does that make sense ?
> > > >
> > > >
> > > >
> > > > > So I'm
> > > > > not familiar with Tomcat Native implementation, but do you know the
> > > > global
> > > > > architecture ?
> > > > > What I guessed is that the Tomcat native stuff created with the
> > inherit
> > > > > flag, but even if it does, I don't see why it should not work if
the
> > > > socket
> > > > > is closed after a timeout.
> > > > >
> > > > > Regards
> > > > > Jeff
> > > > >
> > > > >
> > > > >
> > > > > On Fri, Jun 29, 2012 at 11:22 AM, Laurent Petit <lpetit@yseop.com>
> > > > wrote:
> > > > >
> > > > > > Hello Jeff, Konstantin & all,
> > > > > >
> > > > > > On Mon, 2012-06-25 at 20:52 +0200, verlag.preisser@t-online.dewrote:
> > > > > > > Hello Jeff & all,
> > > > > > >
> > > > > > > > Von: Jeff MAURY <jeffmaury@jeffmaury.com>
> > > > > > > > Datum: Mon, 25 Jun 2012 18:46:02 +0200
> > > > > > >
> > > > > > > > Konstantin,
> > > > > > > >
> > > > > > > > your explanations are very interesting but unclear
to me: what
> > do
> > > > you
> > > > > > > > call the inactivity timer ? When it is started ? After
the
> > request
> > > > has
> > > > > > > > been processed by the servlet ? In that case, I see
no
> > difference
> > > > > > > > between a servlet that launch a process and another
one.
> > > > > > > > It seems to me that a process that is launched does
not
> > inhererits
> > > > > > > > handles from its parent process but it's possible
that under
> > > > Windows,
> > > > > > > > it's an option so it would be interesting to watch.
> > > > > > > >
> > > > > > > > Jeff
> > > > > > > >
> > > > > > >
> > > > > > > Sorry, I'm just a normal Tomcat user, and I don't know
how
> > exactly
> > > > the
> > > > > > APR connector and its Timeout works, so I am unable to answer
that.
> > > > > > >
> > > > > > >
> > > > > > > Howewer, I did some further observations:
> > > > > > >
> > > > > > > -When I perform a request to the servlet that opens wordpad.exe,
> > the
> > > > TCP
> > > > > > connection from Tomcat does not close after the timeout - even
> > when I
> > > > kill
> > > > > > the Tomcat process (java.exe), the TCP connection is still open.
> > If I
> > > > kill
> > > > > > wordpad.exe, then finally the connection is closed/aborted.
> > > > > > > -When I have 1 TCP connection open to Tomcat and the servlet
> > starts
> > > > the
> > > > > > little C program, Task manager shows that it has 11 handles.
> > > > > > > However, when I have 5 TCP connections open to Tomcat,
and do the
> > > > > > request on one of them, Task maanger shows that the C program
has
> > 15
> > > > > > handles - so four more handles when there are four more
> > connections to
> > > > > > Tomcat. All of that 5 TCP connections don't close until I kill
that
> > > > process.
> > > > > > >
> > > > > > > That seems to me to be an indication that socket handles
could be
> > > > > > inherited by the child processes that are startet by ProcessBuilder
> > > > from
> > > > > > tomcat.
> > > > > > >
> > > > > > > A msdn article mentions how to create a new process using
> > > > > > CreateProcess(); it also mentions that socket handles can be
> > inherited:
> > > > > >
> > http://msdn.microsoft.com/en-us/library/windows/desktop/ms724466.aspx
> > > > > > >
> > > > > > > However, as I don't have much knowledge about programming
with
> > > > WinAPIs,
> > > > > > I don't know why those handles are inherited (the MSDN article
> > mentions
> > > > > > that a handle must be specified as inheritable when created,
to
> > allow a
> > > > > > child process to inherit it). Maybe someone with more WinAPI/Tomcat
> > > > Native
> > > > > > knowledge can help here.
> > > > > >
> > > > > >
> > > > > > I also had the vague intuition that this related to handles
> > > > inheritence.
> > > > > > Your recent tests & research tend to make this hypothesis
even more
> > > > > > appealing.
> > > > > >
> > > > > > I know nothing about tomcat-native implementation, but I was
able
> > to
> > > > see
> > > > > > that the bInheritHandle member is set to true here in tomcat
> > native's C
> > > > > > code:
> > > > > >
> > > > > > lpetit:~/tmp/tomcat-native $ grep "bInheritHandle" -R *
> > > > > > native/os/win32/ntpipe.c:    con->sa.bInheritHandle = TRUE;
> > > > > >
> > > > > > (
> > > > > >
> > > > > >
> > > >
> > http://svn.apache.org/repos/asf/tomcat/native/branches/1.1.x/native/os/win32/ntpipe.c
> > > > )
> > > > > >
> > > > > >
> > > > > > Jeff,
> > > > > >
> > > > > > What else could we do to help investigate / fix this issue ?
> > > > > >
> > > > > >
> > > > > > Cheers,
> > > > > >
> > > > > > --
> > > > > > Laurent
> > > > > >
> > > > > >
> > > > > > >
> > > > > > > Regards,
> > > > > > > Konstantin Preißer
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > ---------------------------------------------------------------------
> > > > > > > To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> > > > > > > For additional commands, e-mail: users-help@tomcat.apache.org
> > > > > > >
> > > > > >
> > > > > > --
> > > > > > Laurent Petit
> > > > > >
> > > > > > Agence +33 (0)4 78 47 07 49
> > > > > >
> > > > > > Email     lpetit@yseop.com
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > Yseop apporte une réponse intelligente et individualisée à
chacun
> > de
> > > > vos
> > > > > > clients
> > > > > >
> > > > > >
> > > > > >
> > > > > > www.yseop.com
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > ---------------------------------------------------------------------
> > > > > > To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> > > > > > For additional commands, e-mail: users-help@tomcat.apache.org
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > > Laurent Petit
> > > >
> > > > Agence +33 (0)4 78 47 07 49
> > > >
> > > > Email     lpetit@yseop.com
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > Yseop apporte une réponse intelligente et individualisée à chacun de
> > vos
> > > > clients
> > > >
> > > >
> > > >
> > > > www.yseop.com
> > > >
> > > >
> > > >
> > >
> > >
> >
> > --
> > Laurent Petit
> >
> > Agence +33 (0)4 78 47 07 49
> >
> > Email     lpetit@yseop.com
> >
> >
> >
> >
> >
> >
> >
> > Yseop apporte une réponse intelligente et individualisée à chacun de vos
> > clients
> >
> >
> >
> > www.yseop.com
> >
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> > For additional commands, e-mail: users-help@tomcat.apache.org
> >
> >
> 
> 

-- 
Laurent Petit

Agence +33 (0)4 78 47 07 49

Email     lpetit@yseop.com

 



 

Yseop apporte une réponse intelligente et individualisée à chacun de vos
clients

 

www.yseop.com




---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Mime
View raw message