hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "James Clampffer (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-8765) Implement local block reader in libhdfspp
Date Thu, 17 Dec 2015 20:37:46 GMT

     [ https://issues.apache.org/jira/browse/HDFS-8765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

James Clampffer updated HDFS-8765:
    Attachment: short-circuit-demo.patch2

Cleaned up the demo slightly.  Got rid of all the old accrights style fd passing, improved
unit tests for passing two file descriptors, added lots of comments in domain socket code.
 Also got rid of old test binaries and a vim .swp file that got dragged into previous patch.

Unit tests are built using a little bash script to reduce number of variables in play until
this is working.  Running libhdfspp/lib/localreader/test/test.sh will generate a client and
server.  The server will open and pass two FDs to the client, the client will read the contents
of both.

> Implement local block reader in libhdfspp
> -----------------------------------------
>                 Key: HDFS-8765
>                 URL: https://issues.apache.org/jira/browse/HDFS-8765
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: hdfs-client
>            Reporter: James Clampffer
>            Assignee: James Clampffer
>         Attachments: short-circuit-demo.patch2, short_circuit_demo.patch
> Implement a block reader that uses the hdfs short circuit protocol to read colocated
data as efficiently as possible.  Implementation will be based on BlockReaderLocal.java +
the associated JNI bindings.

This message was sent by Atlassian JIRA

View raw message