Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hdfs-issues@hadoop.apache.org
Date: Fri, 17 Aug 2012 05:23:38 +1100 (NCT)
From: "Andy Isaacson (JIRA)" <jira@apache.org>
To: hdfs-issues@hadoop.apache.org
Message-ID: <407757071.20201.1345141418068.JavaMail.jiratomcat@arcas>
In-Reply-To: <171312319.16600.1345071878065.JavaMail.jiratomcat@arcas>
Subject: [jira] [Commented] (HDFS-3808) fuse_dfs: postpone libhdfs
 intialization until after fork
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HDFS-3808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13436189#comment-13436189 ] 

Andy Isaacson commented on HDFS-3808:
-------------------------------------

{code}
+  fprintf(stderr, "LD_LIBRARY_PATH=%s",ld == NULL ? "NULL" : ld);
+  fprintf(stderr, "CLASSPATH=%s",cp == NULL ? "NULL" : cp);
{code}
Tiny cleanup:  let's add \n to these format strings so the debugging output is slightly easier to read.
{code}
@@ -76,6 +86,21 @@ void init_protectedpaths(dfs_context *dfs) {
     j++;
   }
   dfs->protectedpaths[j] = NULL;
+
+  ret = fuseConnectInit(options.nn_uri, options.nn_port);
{code}
This call doesn't belong inside of {{init_protectedpaths}}.  Instead, please move it out to right after the invocation of {{init_protectedpaths}} in {{dfs_init}}.  And while you're here, you can make {{init_protectedpaths}} static.
{code}
+  char attrTimeoutBuf[512], entryTimeoutBuf[512];
...
-  {
-    char buf[1024];
-
-    snprintf(buf, sizeof buf, "-oattr_timeout=%d",options.attribute_timeout);
-    fuse_opt_add_arg(&args, buf);
+  snprintf(attrTimeoutBuf, sizeof(attrTimeoutBuf),
+           "-oattr_timeout=%d",options.attribute_timeout);
+  fuse_opt_add_arg(&args, attrTimeoutBuf);
 
-    snprintf(buf, sizeof buf, "-oentry_timeout=%d",options.entry_timeout);
-    fuse_opt_add_arg(&args, buf);
-  }
+  snprintf(entryTimeoutBuf, sizeof(entryTimeoutBuf),
+           "-oentry_timeout=%d",options.entry_timeout);
+  fuse_opt_add_arg(&args, entryTimeoutBuf);
{code}
No need to add two separate bufs here; fuse_opt_add_arg must be making a copy of the string anyways since we're passing in automatic arrays, so please preserve just one {{buf[]}}.  However, 1024 is absurdly overlarge.  %d has a max expansion of 1+ceil(ln(2**64)/ln(10)) = 21, so make the buffer 40 bytes or 80 bytes and call it a day.
                
> fuse_dfs: postpone libhdfs intialization until after fork
> ---------------------------------------------------------
>
>                 Key: HDFS-3808
>                 URL: https://issues.apache.org/jira/browse/HDFS-3808
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: fuse-dfs
>    Affects Versions: 2.2.0-alpha
>            Reporter: Colin Patrick McCabe
>            Assignee: Colin Patrick McCabe
>            Priority: Critical
>         Attachments: HDFS-3808.001.patch
>
>
> {{libhdfs}} may create threads or initialize internal JNI data structures when it is used.  So we should be careful to {{daemonize()}} before initializing {{libhdfs}}, not after.
> Unfortunately, fuse_dfs is doing just this -- initializing libhdfs in its {{main}} function, prior to calling {{fuse_main}} which does a {{daemonize}}.
> {{daemonize()}} does not preserve threads, because it is implemented by {{fork()}}ing and then continuing execution in the child and allowing the parent to {{exit()}}.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira