hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-7966) New Data Transfer Protocol via HTTP/2
Date Tue, 12 May 2015 17:48:02 GMT

    [ https://issues.apache.org/jira/browse/HDFS-7966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14540331#comment-14540331

stack commented on HDFS-7966:

For your consideration:

On 1., are you sure this the case? There is a hacked up grpc poc over in hbaseland that implements
hbase's metadata-in-pb-and-data-follows-behind. Here is client write: https://github.com/louiscryan/hbase/commit/e2dae9e8e3f648125f1e13e0ff88944935ad39a3#diff-f5daefd77cbc7bbca73187505c2a9c13R179

On 2., does the write have to be streaming? It can't be a sized write with metadata to say
h*() when all received?

Let me ask my migration question another way: We thinking this will be an incompatible change?


> New Data Transfer Protocol via HTTP/2
> -------------------------------------
>                 Key: HDFS-7966
>                 URL: https://issues.apache.org/jira/browse/HDFS-7966
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>            Reporter: Haohui Mai
>            Assignee: Qianqian Shi
>              Labels: gsoc, gsoc2015, mentor
>         Attachments: GSoC2015_Proposal.pdf
> The current Data Transfer Protocol (DTP) implements a rich set of features that span
across multiple layers, including:
> * Connection pooling and authentication (session layer)
> * Encryption (presentation layer)
> * Data writing pipeline (application layer)
> All these features are HDFS-specific and defined by implementation. As a result it requires
non-trivial amount of work to implement HDFS clients and servers.
> This jira explores to delegate the responsibilities of the session and presentation layers
to the HTTP/2 protocol. Particularly, HTTP/2 handles connection multiplexing, QoS, authentication
and encryption, reducing the scope of DTP to the application layer only. By leveraging the
existing HTTP/2 library, it should simplify the implementation of both HDFS clients and servers.

This message was sent by Atlassian JIRA

View raw message