hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Stephen Bovy (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDFS-5541) LIBHDFS questions and performance suggestions
Date Thu, 21 Nov 2013 03:25:35 GMT
Stephen Bovy created HDFS-5541:

             Summary: LIBHDFS questions and performance suggestions
                 Key: HDFS-5541
                 URL: https://issues.apache.org/jira/browse/HDFS-5541
             Project: Hadoop HDFS
          Issue Type: Improvement
          Components: hdfs-client
            Reporter: Stephen Bovy
            Priority: Minor

Since libhdfs is a "client" interface",  and esspecially because it is a "C" interface , it
should be assumed that the code will be used accross many different platforms, and many different

1) The code should be cross platform ( no Linux extras )
2) The code should compile on standard c89 compilers, the
>>>  {least common denominator rule applies here} !! <<  

C  code with  "c"   extension should follow the rules of the c standard  

All variables must be declared at the begining of scope , and no (//) comments allowed 

>> I just spent a week white-washing the code back to nornal C standards so that it
could compile and build accross a wide range of platforms << 

Now on-to  performance questions 

1) If threads are not used why do a thread attach ( when threads are not used all the thread
attach nonesense is a waste of time and a performance killer ) 

2) The JVM  init  code should not be imbedded within the context of every function call  
.  The  JVM init code should be in a stand-alone  LIBINIT function that is only invoked once.
  The JVM * and the JNI * should be global variables for use when no threads are utilized.

3) When threads are utilized the attach fucntion can use the GLOBAL  jvm * created by the
LIBINIT  { WHICH IS INVOKED ONLY ONCE } and thus safely outside the scope of any LOOP that
is using the functions 

4) Hash Table and Locking  Why ?????
When threads are used the hash table locking is going to hurt perfromance .  Why not use thread
local storage for the hash table,that way no locking is required either with or without threads.
5) FINALLY Windows  Compatibility 

Do not use posix features if they cannot easilly be replaced on other platforms   !!

This message was sent by Atlassian JIRA

View raw message