hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Stephen Bovy (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDFS-10412) Add optional non-thread support for better perfromance
Date Mon, 16 May 2016 22:20:12 GMT
Stephen Bovy created HDFS-10412:

             Summary: Add optional non-thread support for better perfromance
                 Key: HDFS-10412
                 URL: https://issues.apache.org/jira/browse/HDFS-10412
             Project: Hadoop HDFS
          Issue Type: Improvement
          Components: libhdfs
            Reporter: Stephen Bovy
            Priority: Minor

I would like to propose some simple optional changes that would be activated by a compiler
flag.  These changes would enable a typical monolithic single threaded application to use
libhdfs without the need to incur additional overhead for threads that is not needed. 

Here is a proposed for these changes:

#1 add a new function to "jni_helper.c"

/** getSoloJNIEnv: A helper function to get the JNIEnv* for non-threaded use.
 * If no JVM exists, then one will be created. JVM command line arguments
 * are obtained from the LIBHDFS_OPTS environment variable.
 * @param: None.
 * @return The JNIEnv* for non-thread use.
 * */
JNIEnv* getSoloJNIEnv(void)
    JNIEnv *env;
    env = getGlobalJNIEnv();
    if (!env) {
      fprintf(stderr, "getJNIEnv: getGlobalJNIEnv failed\n");
      return NULL;
    return env;

Add the following the following to "hdfs.c"

#1 a static global variable : "JNIEnv* hdfsJNIEnv;"

static JNIEnv* hdfsJNIEnv = NULL;
#define GETJNIENV()               \
if (hdfsJNIEnv == NULL) {         \
    hdfsJNIEnv = getSoloJNIEnv(); \
}                                 \
env = hdfsJNIEnv; 
#define  GETJNIENV() env = getJNIEnv(); 

The above NEW MACRO  would be used as in the following example:

int hdfsFileGetReadStatistics(hdfsFile file,
                              struct hdfsReadStatistics **stats)

    jthrowable jthr;
    jobject readStats = NULL;
    jvalue jVal;
    struct hdfsReadStatistics *s = NULL;
    int ret;
    JNIEnv* env;
//  JNIEnv* env = getJNIEnv();

    ( ... )


This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org

View raw message