hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Binglin Chang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-10389) Native RPCv9 client
Date Tue, 17 Jun 2014 06:38:03 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-10389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14033486#comment-14033486

Binglin Chang commented on HADOOP-10389:

I'd like to add more inputs on this: 
I wrote a c++ rpc/hdfs/yarn client(https://github.com/decster/libhadoopclient), it uses c++11,
so it does not need boost(although many people use boost, they just for header only libraries,
and public headers does not include boost, so there are no version issues).
c++ 's main concern is abi compatibility, this can be resolved by using c or simple c++ class
public headers, hiding real implementation.

I think some issue using c++/pro using c are:
1. centos does not have enough support for c++11, c++11 is not generally available yet
2. remain libhdfs compatibility, since libhdfs is written in c, we might just continue using
c as well

Also there are some concerns about using c:
1. the protobuf-c library is just not so reliable as official protobuf library which is maintained
and verified by google and many other companies/projects, I read some of the protobuf-c code,
it uses a reflection style implementation to do serializing/deserializing, so performance,
security, compatibility may all at risk. I see https://github.com/protobuf-c/protobuf-c only
have 92 stars.
2. malloc/free/memset can easily generate buggy code, need additional care and checks, I see
many those kinds of code recently(HDFS-6534,HADOOP-10640,HADOOP-10706). it is OK to use c,
but we may need more care and effort. 

About JNIFS,  why do we need jnifs if we already have nativefs? using dlopen/dlsym to replace
jni apis is not trivial if both compile/runtime dependency is needed to be removed. 

> Native RPCv9 client
> -------------------
>                 Key: HADOOP-10389
>                 URL: https://issues.apache.org/jira/browse/HADOOP-10389
>             Project: Hadoop Common
>          Issue Type: Sub-task
>    Affects Versions: HADOOP-10388
>            Reporter: Binglin Chang
>            Assignee: Colin Patrick McCabe
>         Attachments: HADOOP-10388.001.patch, HADOOP-10389-alternative.000.patch, HADOOP-10389.002.patch,
HADOOP-10389.004.patch, HADOOP-10389.005.patch

This message was sent by Atlassian JIRA

View raw message