hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Colin Patrick McCabe (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HADOOP-10667) implement TCP connection reuse for native client
Date Fri, 06 Jun 2014 07:59:01 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-10667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Colin Patrick McCabe updated HADOOP-10667:
------------------------------------------

    Attachment: HADOOP-10667-pnative.001.patch

* Fix connection re-use in conn.c.  This involves properly setting exemplar.username and fixing
the sort order of connections in the map a bit.

* namenode-rpc-unit.c: do two RPCs, not one, to test our connection reuse.

* rather than logging the pointer values of reactors, connections, and messengers, give these
objects names and log those instead.  This is a lot nicer to read.  Add more debug log messages.
 The names of connections include the remote address being connected to as well as the reactor.

* the proxies need to store the remote address they're connecting to.  Otherwise, we can't
re-use proxies without re-initializing the address.  Call objects should just have a pointer
to the proxy address, rather than an address value, since proxies can't be used by more than
one call at once anyway.

* {{hrpc_conn_read_alloc}}: improve the comments a bit.

* reactor: block delivery of {{SIGPIPE}} to reactor threads.

* some fixes to how we shut down our libuv objects when shutting down a messenger.  We need
to remove the async watcher so that there are no remaining references to the loop, then close
the loop.  This frees all the resources (according to valgrind).

> implement TCP connection reuse for native client
> ------------------------------------------------
>
>                 Key: HADOOP-10667
>                 URL: https://issues.apache.org/jira/browse/HADOOP-10667
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: native
>    Affects Versions: HADOOP-10388
>            Reporter: Colin Patrick McCabe
>            Assignee: Colin Patrick McCabe
>         Attachments: HADOOP-10667-pnative.001.patch
>
>
> The HDFS / YARN native clients should re-use TCP connections to avoid the overhead of
the three-way handshake, similar to how the Java code does.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message