hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Allen Wittenauer (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-7887) Asynchronous native RPC v9 client
Date Thu, 05 Mar 2015 02:18:38 GMT

    [ https://issues.apache.org/jira/browse/HDFS-7887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14347992#comment-14347992

Allen Wittenauer commented on HDFS-7887:

Is this really a good, long term strategy given our use of protobuf now that gRPC exists?

> Asynchronous native RPC v9 client
> ---------------------------------
>                 Key: HDFS-7887
>                 URL: https://issues.apache.org/jira/browse/HDFS-7887
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>            Reporter: Haohui Mai
>            Assignee: Haohui Mai
> There are more and more integration happening between Hadoop and applications that are
implemented using languages other than Java.
> To access Hadoop, applications either have to go through JNI (e.g. libhdfs), or to reverse
engineer the Hadoop RPC protocol. (e.g. snakebite). Unfortunately, neither of them are satisfactory:
> * Integrating with JNI requires running a JVM inside the application. Some applications
(e.g., real-time processing, MPP database) does not want the footprints and GC behavior of
the JVM.
> * The Hadoop RPC protocol has a rich feature set including wire encryption, SASL, Kerberos
authentication. Many 3rd-party implementations can fully cover the feature sets thus they
might work in limited environment.
> This jira is to propose implementing an Hadoop RPC library in C++ that provides a common
ground to implement higher-level native client for HDFS, YARN, and MapReduce.

This message was sent by Atlassian JIRA

View raw message