hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Nauroth (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (HDFS-5541) LIBHDFS questions and performance suggestions
Date Sat, 07 Dec 2013 06:30:39 GMT

     [ https://issues.apache.org/jira/browse/HDFS-5541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Chris Nauroth resolved HDFS-5541.
---------------------------------

    Resolution: Invalid

I've filed issues HDFS-5642, HDFS-5643 and HDFS-5644.  I'm going to resolve this one.  Thanks,
[~stevebovy] and [~cmccabe].

> LIBHDFS questions and performance suggestions
> ---------------------------------------------
>
>                 Key: HDFS-5541
>                 URL: https://issues.apache.org/jira/browse/HDFS-5541
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: hdfs-client
>            Reporter: Stephen Bovy
>            Priority: Minor
>         Attachments: pdclibhdfs.zip
>
>
> Since libhdfs is a "client" interface",  and esspecially because it is a "C" interface
, it should be assumed that the code will be used accross many different platforms, and many
different compilers.
> 1) The code should be cross platform ( no Linux extras )
> 2) The code should compile on standard c89 compilers, the
> >>>  {least common denominator rule applies here} !! <<  
> C  code with  "c"   extension should follow the rules of the c standard  
> All variables must be declared at the begining of scope , and no (//) comments allowed

> >> I just spent a week white-washing the code back to nornal C standards so that
it could compile and build accross a wide range of platforms << 
> Now on-to  performance questions 
> 1) If threads are not used why do a thread attach ( when threads are not used all the
thread attach nonesense is a waste of time and a performance killer ) 
> 2) The JVM  init  code should not be imbedded within the context of every function call
  .  The  JVM init code should be in a stand-alone  LIBINIT function that is only invoked
once.   The JVM * and the JNI * should be global variables for use when no threads are utilized.
 
> 3) When threads are utilized the attach fucntion can use the GLOBAL  jvm * created by
the LIBINIT  { WHICH IS INVOKED ONLY ONCE } and thus safely outside the scope of any LOOP
that is using the functions 
> 4) Hash Table and Locking  Why ?????
> When threads are used the hash table locking is going to hurt perfromance .  Why not
use thread local storage for the hash table,that way no locking is required either with or
without threads.   
>  
> 5) FINALLY Windows  Compatibility 
> Do not use posix features if they cannot easilly be replaced on other platforms   !!



--
This message was sent by Atlassian JIRA
(v6.1#6144)

Mime
View raw message