hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Haohui Mai (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-5182) BlockReaderLocal must allow zero-copy reads only when the DN believes it's valid
Date Tue, 04 Feb 2014 19:12:14 GMT

    [ https://issues.apache.org/jira/browse/HDFS-5182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13891042#comment-13891042
] 

Haohui Mai commented on HDFS-5182:
----------------------------------

It looks like to me that passing the file descriptor is a cleaner design:

# If the data is shared through mmap(), an I/O error will trigger a SIGBUS on the client side.
# It requires taking up the virtual memory space for the client, it could be problematic if
the client is running in a container.
# From a portability prospective, {{DuplicateHandle}} on Windows can easily pass file descriptors
around.

I might have missed something. [~cmccabe], can you please elaborate why you prefer choice
#1 instead of #2? Thanks.

> BlockReaderLocal must allow zero-copy  reads only when the DN believes it's valid
> ---------------------------------------------------------------------------------
>
>                 Key: HDFS-5182
>                 URL: https://issues.apache.org/jira/browse/HDFS-5182
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: hdfs-client
>    Affects Versions: 3.0.0
>            Reporter: Colin Patrick McCabe
>            Assignee: Colin Patrick McCabe
>
> BlockReaderLocal must allow zero-copy reads only when the DN believes it's valid.  This
implies adding a new field to the response to REQUEST_SHORT_CIRCUIT_FDS.  We also need some
kind of heartbeat from the client to the DN, so that the DN can inform the client when the
mapped region is no longer locked into memory.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

Mime
View raw message