hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Colin Patrick McCabe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-5541) LIBHDFS questions and performance suggestions
Date Sat, 07 Dec 2013 00:48:35 GMT

    [ https://issues.apache.org/jira/browse/HDFS-5541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13841956#comment-13841956
] 

Colin Patrick McCabe commented on HDFS-5541:
--------------------------------------------

I have not looked into it, but I had heard that CMake's integration with Visual Studio was
actually very good.  Certainly, it generates .vcproj files, which allow you to use Visual
Studio natively.  Hopefully, a Visual Studio user can check it out at some point.

I also wrote a maven cmake plugin that I would like to get working at some point.  It's been
on the back burner for a while, but it would be nicer than the ant-based way of executing
C builds we have now.

But this is all somewhat of a tangent.  Long story short, thanks for looking at this, guys,
looking forward to seeing the follow-up.

> LIBHDFS questions and performance suggestions
> ---------------------------------------------
>
>                 Key: HDFS-5541
>                 URL: https://issues.apache.org/jira/browse/HDFS-5541
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: hdfs-client
>            Reporter: Stephen Bovy
>            Priority: Minor
>         Attachments: pdclibhdfs.zip
>
>
> Since libhdfs is a "client" interface",  and esspecially because it is a "C" interface
, it should be assumed that the code will be used accross many different platforms, and many
different compilers.
> 1) The code should be cross platform ( no Linux extras )
> 2) The code should compile on standard c89 compilers, the
> >>>  {least common denominator rule applies here} !! <<  
> C  code with  "c"   extension should follow the rules of the c standard  
> All variables must be declared at the begining of scope , and no (//) comments allowed

> >> I just spent a week white-washing the code back to nornal C standards so that
it could compile and build accross a wide range of platforms << 
> Now on-to  performance questions 
> 1) If threads are not used why do a thread attach ( when threads are not used all the
thread attach nonesense is a waste of time and a performance killer ) 
> 2) The JVM  init  code should not be imbedded within the context of every function call
  .  The  JVM init code should be in a stand-alone  LIBINIT function that is only invoked
once.   The JVM * and the JNI * should be global variables for use when no threads are utilized.
 
> 3) When threads are utilized the attach fucntion can use the GLOBAL  jvm * created by
the LIBINIT  { WHICH IS INVOKED ONLY ONCE } and thus safely outside the scope of any LOOP
that is using the functions 
> 4) Hash Table and Locking  Why ?????
> When threads are used the hash table locking is going to hurt perfromance .  Why not
use thread local storage for the hash table,that way no locking is required either with or
without threads.   
>  
> 5) FINALLY Windows  Compatibility 
> Do not use posix features if they cannot easilly be replaced on other platforms   !!



--
This message was sent by Atlassian JIRA
(v6.1#6144)

Mime
View raw message