hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Doug Cutting (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-312) Connections should not be cached
Date Thu, 24 Aug 2006 06:49:23 GMT
    [ http://issues.apache.org/jira/browse/HADOOP-312?page=comments#action_12430122 ] 
            
Doug Cutting commented on HADOOP-312:
-------------------------------------

> I thought that I could save the overhead of destroying/creating threads [ ...]

Threads should be cheap to create, but keeping too many around can be expensive.  Again, I
think the life of the thread should be the same as the life of the connection.

> I wanted to avoid so-many-unnecessary method invocations when caching is disabled.

Method invocations that check a flag are plenty fast.  This is not an inner loop.  I'd rather
have this code be easier to maintain than a few nanoseconds faster.

> Connections should not be cached
> --------------------------------
>
>                 Key: HADOOP-312
>                 URL: http://issues.apache.org/jira/browse/HADOOP-312
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: ipc
>            Reporter: Devaraj Das
>         Assigned To: Devaraj Das
>         Attachments: no_conn_caching.patch, no_conn_caching.patch, no_conn_caching.patch,
no_connection_caching.patch, no_connection_caching.patch
>
>
> Servers and clients (client include datanodes, tasktrackers, DFSClients & tasks)
should not cache connections or maybe cache them for very short periods of time. Clients should
set up & tear down connections to the servers everytime they need to contact the servers
(including the heartbeats). If connection is cached, then reuse the existing connection for
a few subsequent transactions until the connection expires. The heartbeat interval should
be more so that many more clients (order of  tens of thousands) can be accomodated within
1 heartbeat interval.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message