Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hdfs-issues@hadoop.apache.org
Date: Tue, 4 Feb 2014 19:12:14 +0000 (UTC)
From: "Haohui Mai (JIRA)" <jira@apache.org>
To: hdfs-issues@hadoop.apache.org
Message-ID: <JIRA.12667954.1378857814548.35665.1391541134444@arcas>
In-Reply-To: <JIRA.12667954.1378857814548@arcas>
References: <JIRA.12667954.1378857814548@arcas>
Subject: [jira] [Commented] (HDFS-5182) BlockReaderLocal must allow
 zero-copy  reads only when the DN believes it's valid
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HDFS-5182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13891042#comment-13891042 ] 

Haohui Mai commented on HDFS-5182:
----------------------------------

It looks like to me that passing the file descriptor is a cleaner design:

# If the data is shared through mmap(), an I/O error will trigger a SIGBUS on the client side.
# It requires taking up the virtual memory space for the client, it could be problematic if the client is running in a container.
# From a portability prospective, {{DuplicateHandle}} on Windows can easily pass file descriptors around.

I might have missed something. [~cmccabe], can you please elaborate why you prefer choice #1 instead of #2? Thanks.

> BlockReaderLocal must allow zero-copy  reads only when the DN believes it's valid
> ---------------------------------------------------------------------------------
>
>                 Key: HDFS-5182
>                 URL: https://issues.apache.org/jira/browse/HDFS-5182
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: hdfs-client
>    Affects Versions: 3.0.0
>            Reporter: Colin Patrick McCabe
>            Assignee: Colin Patrick McCabe
>
> BlockReaderLocal must allow zero-copy reads only when the DN believes it's valid.  This implies adding a new field to the response to REQUEST_SHORT_CIRCUIT_FDS.  We also need some kind of heartbeat from the client to the DN, so that the DN can inform the client when the mapped region is no longer locked into memory.


--
This message was sent by Atlassian JIRA
(v6.1.5#6160)