hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Colin Patrick McCabe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-4356) BlockReaderLocal should use passed file descriptors rather than paths
Date Thu, 03 Jan 2013 22:24:13 GMT

    [ https://issues.apache.org/jira/browse/HDFS-4356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13543352#comment-13543352

Colin Patrick McCabe commented on HDFS-4356:

This is the final subtask of HDFS-347.  It modifies {{BlockReaderLocal}} and {{DataXceiver}}
to use the domain socket abstractions.  It adds {{OpRequestShortCircuitAccessProto}} to {{hadoop-hdfs-project/hadoop-hdfs/src/main/proto/datatransfer.proto}}.

BlockReader gets a new method, {{available}}, which is used to determine how much data is
probably available from the {{BlockReader}} without performing network I/O.  It's a heuristic.
 This replaces a hard-coded check against TCP_WINDOW_SIZE in {{DFSClient.java}}.

{{DomainSocketFactory}} is a new class which manages creating UNIX domain sockets.  This includes
managing the blacklists of non-working domain sockets and tracking whether UNIX domain sockets
are configured and available.

The tests which inherit from {{TestParallelReadUtil}} now rely on the {{\@Test}} functions
declared in the superclass, rather than duplicating that code.  There are also a few new derived
classes like {{TestParallelUnixDomainRead}} and {{TestParallelShortCircuitReadNoChecksum}}.
> BlockReaderLocal should use passed file descriptors rather than paths
> ---------------------------------------------------------------------
>                 Key: HDFS-4356
>                 URL: https://issues.apache.org/jira/browse/HDFS-4356
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: datanode, hdfs-client, performance
>    Affects Versions: 2.0.3-alpha
>            Reporter: Colin Patrick McCabe
>            Assignee: Colin Patrick McCabe
>         Attachments: 04-cumulative.patch
> {{BlockReaderLocal}} should use file descriptors passed over UNIX domain sockets rather
than paths.  We also need some configuration options for these UNIX domain sockets.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message