hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Haohui Mai (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-6994) libhdfs3 - A native C/C++ HDFS client
Date Mon, 08 Sep 2014 17:59:31 GMT

    [ https://issues.apache.org/jira/browse/HDFS-6994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14125825#comment-14125825

Haohui Mai commented on HDFS-6994:

bq. I'm very much against boost showing up in Hadoop in any way, shape, or form. It's always
long term pain. I'd much rather take the short term hit of requiring a modern compiler than
deal with boost.

I think this is a fair point. C++11 covers a lot of issues for the original boost use case.
The only remaining gap I see, however, is {{boost::asio}}.

I see a couple possible approaches: 

# Wait until C++14 for the networking library.
# Import an WIP implementation for asio in http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2012/n3388.pdf
# Use other networking libraries.

I'm a little bit skeptical about (3) because these libraries might not be as mature and optimized
compared to {{boost::asio}}. It might be worthwhile to take a look at the second approach
given the fact that the current code has been using {{boost::asio}} already.


> libhdfs3 - A native C/C++ HDFS client
> -------------------------------------
>                 Key: HDFS-6994
>                 URL: https://issues.apache.org/jira/browse/HDFS-6994
>             Project: Hadoop HDFS
>          Issue Type: Task
>          Components: hdfs-client
>            Reporter: Zhanwei Wang
>         Attachments: HDFS-6994-rpc-8.patch, HDFS-6994.patch
> Hi All
> I just got the permission to open source libhdfs3, which is a native C/C++ HDFS client
based on Hadoop RPC protocol and HDFS Data Transfer Protocol.
> libhdfs3 provide the libhdfs style C interface and a C++ interface. Support both HADOOP
RPC version 8 and 9. Support Namenode HA and Kerberos authentication.
> libhdfs3 is currently used by HAWQ of Pivotal
> I'd like to integrate libhdfs3 into HDFS source code to benefit others.
> You can find libhdfs3 code from github
> https://github.com/PivotalRD/libhdfs3
> http://pivotalrd.github.io/libhdfs3/

This message was sent by Atlassian JIRA

View raw message