httpd-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Taisuke Yamada <tyamad...@list.rakugaki.org>
Subject Re: [RFC] Adding UDP support to server
Date Fri, 23 Feb 2007 09:06:35 GMT
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Last year, I posted a question on implementation strategy of
HTTP-over-UDP(multicast) module to modules-dev list. I was
able to implement fully functional "UDP echo" service, but
for HTTP/U module, it suffered severe limitation due to how
core_filter handles output.

- - http://www.mail-archive.com/modules-dev@httpd.apache.org/msg00129.html

Here were my findings at the time (with 2.2.4-dev):

- - For initial UDP socket setup, current httpd implementation
  (listen.c) is OK, as I was able to bypass it by inserting
  custom ap_listen_rec (with custom accept_func function) into
  ap_listeners at open_log stage.

- - Another issue was ap_lingering_close called after request
  processing, but that too, can be worked around by doing
  ap_set_module_config(c->conn_config, &core_module, NULL);
  at process_connection stage.

- - Probe error from dummy_connection was an issue, but was not
  a blocker as I can suppress that by listening on same TCP
  port (Type2 implementation in above URL).

- - There was a timeout issue which caused server to stop
  responding when handling UDP socket. I worked around that
  by setting custom timeout value at process_connection stage,
  but this was only a kludge and requires deeper look for
  complete solution.

- - core_output_filter (core_filters.c) was a major blocker.
  This (to be precise, underlying apr_socket_sendv called)
  expects output socket to be able to return data with simple
  writev(2). That can't be done for un-connected socket.

To workaround last issue, I did apr_socket_connect to UDP socket,
but that destroys listening UDP socket. In order to handle this,
I tried several methods below:

- - Do a connect(2) on current socket, pass it to bucket handler,
  and dup2(2) newly created listening socket into socket registered
  to ap_listeners.
- -- This almost worked, but apache never responded to further
   request even though it continued to listen on UDP port.

- - Disconnect "connected" UDP socket by connecting with AF_UNSPEC
- -- Same as above...forgot the detail, but it didn't work.

- - Pass newly created "connected" socket to bucket handler
- -- This was a stupid idea as there's no way to pass received
   data to this new socket (only if there's accept(2) for UDP...).

All failed, and as a result this HTTP/U module handled only
first HTTP/UDP request properly, but no more.

Although I have no specific idea on implementation, I think the
patch posted can be made simpler by focusing on core_filter issue.
But maybe this is an APR issue rather than of httpd?

> I'm working on a protocol module for Apache 2.2 and ran into the lack of
> UDP support in httpd.  I'd like to try and remedy the situation in a
> manner best suited for merging to trunk + backporting where possible.  I
> know that people have asked about it in the past, and if we really want
> to be "d", we need support beyond TCP/SOCK_STREAM sockets.
> 
> I tried to go through the socket-related routines in httpd (those of the
> server; not of modules, clients, etc (mod_ssl, mod_proxy, mod_cgid,
> etc)) and came up with a list of areas which needed attention:
> 
> listen.c - alloc_listener is the focal point of the socket creation in
> the server.  There are a few issues that I can see with patching this
> function:
> 1) Currently it calls apr_socket_create with SOCK_STREAM, 0 (for type,
> protocol).  For adding UDP support only, we could make do with adding a
> flag to the function to toggle between the existing values and
> SOCK_DGRAM, APR_PROTO_UDP (which I'm personally against).  To be more
> portable, we'd pass both type and protocol as parameters (which I
> personally prefer).  Neither of these options makes much of a mess, as
> AFAIK only ap_set_listener calls alloc_listener.
> 2) In the code to check if we can re-use an existing listener, compare
> the socket-descriptor's type and protocol against the ones passed.
> 3) Additionally, I'd suggest making alloc_listener a public API function
> (we've already added arguments, so no extra harm in renaming it to
> ap_alloc_listener and adding it to ap_listen.h is there?) so that module
> authors can create directives for protocols/types beyond SOCK_STREAM/TCP
> and SOCK_DGRAM/UDP (see below).  [This would be a minor bump, right?  It
> adds functionality without any breakage or backwards compatibility
> issues, since as mentioned above only ap_set_listener currently calls
> alloc_listener today]
> 
> listen.c - make_sock sets a bunch of socket options.  I don't profess to
> be expert enough in socket programming to know what's kosher or not for
> passing to non-"SOCK_STREAM TCP" sockets.  More importantly, make_sock
> calls apr_socket_listen, which isn't correct for datagram sockets.  To
> get around this, perhaps wrap the call in "if (s->type == SOCK_STREAM) "
> (or whatever should be there to know if this type of socket needs to be
> listen()ed to).  Finally, similar logic (to detect if the socket needs
> to be accept()ed) should be applied to the accept_func for MPMs which
> utilize it.  IMHO, rewriting standard MPM's that use an accept_func to
> check if it's NULL before running it seems the smartest way to do it
> (would that be a bigger bump than the ap_alloc_listeners described
> above?).  Barring that, we could define an "empty" default listener
> function which does nothing, and have non-listening sockets have that
> assigned as their alloc_func instead of MPM_ACCEPT_FUNC
> 
> listen.c - ap_apply_accept_filter sets the accept filter for the
> protocol (["http", "https", etc], not [TCP, UDP, etc])  As above, I'm
> not sure how "kosher" this is, but TCP_DEFER_ACCEPT on UDP sounds wrong
> :-)  That could be solved by enclosing the filter in an "if (s->protocol
> == APR_PROTO_TCP)" block
> 
> listen.c - ap_set_listener is the user's interface to adding a listener.
>  I can't think of a nice graceful way to add the stream-type and
> protocol to the existing listen directive while remaining backwards
> compatible and keeping some resemblance of simplicity to the directive,
> so I propose adding a ListenUDP directive which does the same logic
> except that it calls alloc_listener with SOCK_DGRAM, APR_PROTO_UDP
> (since UDP has been asked about several times).  For other combinations,
> module authors can call ap_alloc_listener as described above
> 
> core.c - core_pre_connection disables the nagle algorithm by setting
> APR_TCP_NODELAY.  Can this be done safely to a non-TCP socket?  If not,
> just check "if (csd->protocl == APRC_PROTO_TCP)" as above
> 
> mpm_common.c - ap_sock_disable_nagle disables the nagle algorithm as
> above.
> 
> mpm_common.c - dummy_connection currently makes assumptions about
> SOCK_STREAM and TCP.  However, it also only connects to the port in the
> first listener's (in global ap_listener) port.  I suppose changing it to
> use ap_listeners->sd->type, ap_listeners->sd->protocol shouldn't hurt...
> should it?
> 
> Connection I/O should also be fairly easy.  The core input filter just
> uses a socket bucket to read from; if apr_bucket_type_socket supports
> UDP sockets, we should be good to go on that end.  The core output
> filter is also fairly simple - just check that we're a SOCK_STREAM
> socket before attempting to sendfile.
> 
> I'm also assuming (possibly incorrectly) that polling and socket timeout
> related code should all work as-is.
> 
> I *think* that's all that should be needed...  I have a patch against
> trunk demonstrating most of this (attached).   I'd really appreciate it
> if people who have a firmer understanding of the relevant bits of code
> that I'm proposing changes to could chime in and let me know if and what
> I'm doing wrong, or not taking into consideration.

- --
Taisuke Yamada <tyamadajp@spam.rakugaki.org>, http://rakugaki.org/
2268 E9A2 D4F9 014E F11D  1DF7 DCA3 83BC 78E5 CD3A

Message to my public address may not be handled in a timely manner.
For a direct contact, please use my private address on my namecard.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFF3q6b3KODvHjlzToRAq6JAJ9HB3XWDBPb1t+d8HMOwNC8P3rlFwCghLUW
+UqYm2BkxGa9rK5aHoIHe7w=
=7TEe
-----END PGP SIGNATURE-----

Mime
View raw message