hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Colin Patrick McCabe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-6046) add dfs.client.mmap.enabled
Date Tue, 04 Mar 2014 07:07:21 GMT

    [ https://issues.apache.org/jira/browse/HDFS-6046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13919068#comment-13919068
] 

Colin Patrick McCabe commented on HDFS-6046:
--------------------------------------------

I committed this to 2.4.  Thanks for the reviews.

The findbugs issues were fixed by HDFS-6040.

bq. Interestingly, it appears the libhdfs tests cannot load the native code from within the
embedded JVM. That's worth following up separately.

Yeah.  The specific error is this one:

{code}
     [exec] nmdCreate: Builder#build error:
     [exec] java.lang.UnsupportedOperationException: libhadoop cannot be loaded.
     [exec]     at org.apache.hadoop.net.unix.DomainSocketWatcher.<init>(DomainSocketWatcher.java:229)
     [exec]     at org.apache.hadoop.hdfs.client.DfsClientShmManager.<init>(DfsClientShmManager.java:404)
     [exec]     at org.apache.hadoop.hdfs.client.ShortCircuitCache.<init>(ShortCircuitCache.java:380)
     [exec]     at org.apache.hadoop.hdfs.ClientContext.<init>(ClientContext.java:96)

     [exec] TEST_ERROR: failed on /home/jenkins/jenkins-slave/workspace/PreCommit-HDFS-Build/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/native/libhdfs/
   test_libhdfs_threaded.c:328 (errno: 2): got NULL from tlhCluster
{code}

Unfortunately, there's not much to go on in the log.  libhadoop.so clearly was built prior
to this point.  I guess we might be setting {{java.library.path}} incorrectly, but that doesn't
explain why it worked previously (and is continuing to work in most test runs).

> add dfs.client.mmap.enabled
> ---------------------------
>
>                 Key: HDFS-6046
>                 URL: https://issues.apache.org/jira/browse/HDFS-6046
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: hdfs-client
>    Affects Versions: 2.4.0
>            Reporter: Colin Patrick McCabe
>            Assignee: Colin Patrick McCabe
>             Fix For: 2.4.0
>
>         Attachments: HDFS-6046.001.patch, HDFS-6046.002.patch, HDFS-6046.003.patch
>
>
> It would be nice to have a configuration variable, dfs.client.mmap.enabled. which would
disable mmap reads in the client.  This could come in handy for debugging or performance benchmarking.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message