hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andy Isaacson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-3808) fuse_dfs: postpone libhdfs intialization until after fork
Date Thu, 16 Aug 2012 18:23:38 GMT

    [ https://issues.apache.org/jira/browse/HDFS-3808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13436189#comment-13436189

Andy Isaacson commented on HDFS-3808:

+  fprintf(stderr, "LD_LIBRARY_PATH=%s",ld == NULL ? "NULL" : ld);
+  fprintf(stderr, "CLASSPATH=%s",cp == NULL ? "NULL" : cp);
Tiny cleanup:  let's add \n to these format strings so the debugging output is slightly easier
to read.
@@ -76,6 +86,21 @@ void init_protectedpaths(dfs_context *dfs) {
   dfs->protectedpaths[j] = NULL;
+  ret = fuseConnectInit(options.nn_uri, options.nn_port);
This call doesn't belong inside of {{init_protectedpaths}}.  Instead, please move it out to
right after the invocation of {{init_protectedpaths}} in {{dfs_init}}.  And while you're here,
you can make {{init_protectedpaths}} static.
+  char attrTimeoutBuf[512], entryTimeoutBuf[512];
-  {
-    char buf[1024];
-    snprintf(buf, sizeof buf, "-oattr_timeout=%d",options.attribute_timeout);
-    fuse_opt_add_arg(&args, buf);
+  snprintf(attrTimeoutBuf, sizeof(attrTimeoutBuf),
+           "-oattr_timeout=%d",options.attribute_timeout);
+  fuse_opt_add_arg(&args, attrTimeoutBuf);
-    snprintf(buf, sizeof buf, "-oentry_timeout=%d",options.entry_timeout);
-    fuse_opt_add_arg(&args, buf);
-  }
+  snprintf(entryTimeoutBuf, sizeof(entryTimeoutBuf),
+           "-oentry_timeout=%d",options.entry_timeout);
+  fuse_opt_add_arg(&args, entryTimeoutBuf);
No need to add two separate bufs here; fuse_opt_add_arg must be making a copy of the string
anyways since we're passing in automatic arrays, so please preserve just one {{buf[]}}.  However,
1024 is absurdly overlarge.  %d has a max expansion of 1+ceil(ln(2**64)/ln(10)) = 21, so make
the buffer 40 bytes or 80 bytes and call it a day.
> fuse_dfs: postpone libhdfs intialization until after fork
> ---------------------------------------------------------
>                 Key: HDFS-3808
>                 URL: https://issues.apache.org/jira/browse/HDFS-3808
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: fuse-dfs
>    Affects Versions: 2.2.0-alpha
>            Reporter: Colin Patrick McCabe
>            Assignee: Colin Patrick McCabe
>            Priority: Critical
>         Attachments: HDFS-3808.001.patch
> {{libhdfs}} may create threads or initialize internal JNI data structures when it is
used.  So we should be careful to {{daemonize()}} before initializing {{libhdfs}}, not after.
> Unfortunately, fuse_dfs is doing just this -- initializing libhdfs in its {{main}} function,
prior to calling {{fuse_main}} which does a {{daemonize}}.
> {{daemonize()}} does not preserve threads, because it is implemented by {{fork()}}ing
and then continuing execution in the child and allowing the parent to {{exit()}}.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message