hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zhanwei Wang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-6994) libhdfs3 - A native C/C++ HDFS client
Date Sat, 06 Sep 2014 01:42:29 GMT

    [ https://issues.apache.org/jira/browse/HDFS-6994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14124240#comment-14124240

Zhanwei Wang commented on HDFS-6994:

Hi [~cmccabe]

Thanks very much for your comments.

Dynamically loading libjvm is a good idea, but it seems not solve all the problems you mentioned
in HADOOP-10388. To make fall back feature work, users have to deploy the HDFS jars on every
machine. This adds operational complexity for non-Java clients that just want to integrate
with HDFS. Otherwise, fall back feature will not work. And fall back feature will finally
be removed when the native client  implement the full HDFS client feature.

About the boost, your are right. Actually boost is not required if the C++ compiler is not
too old. And I also think using boost can make libhdfs3 be useful for as many people as possible
who use the old C++ compiler. But, yes, I should not require a very new boost version, it
can be improved as well as other dependency issues.

So, the most important thing I think, is to figure out a way to integrate libhdfs3 and benefit
other features in HADOOP-10388.

What is your opinion?

> libhdfs3 - A native C/C++ HDFS client
> -------------------------------------
>                 Key: HDFS-6994
>                 URL: https://issues.apache.org/jira/browse/HDFS-6994
>             Project: Hadoop HDFS
>          Issue Type: Task
>          Components: hdfs-client
>            Reporter: Zhanwei Wang
>         Attachments: HDFS-6994-rpc-8.patch, HDFS-6994.patch
> Hi All
> I just got the permission to open source libhdfs3, which is a native C/C++ HDFS client
based on Hadoop RPC protocol and HDFS Data Transfer Protocol.
> libhdfs3 provide the libhdfs style C interface and a C++ interface. Support both HADOOP
RPC version 8 and 9. Support Namenode HA and Kerberos authentication.
> libhdfs3 is currently used by HAWQ of Pivotal
> I'd like to integrate libhdfs3 into HDFS source code to benefit others.
> You can find libhdfs3 code from github
> https://github.com/PivotalRD/libhdfs3

This message was sent by Atlassian JIRA

View raw message