hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Colin Patrick McCabe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-6046) add dfs.client.mmap.enabled
Date Tue, 04 Mar 2014 07:07:21 GMT

    [ https://issues.apache.org/jira/browse/HDFS-6046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13919068#comment-13919068

Colin Patrick McCabe commented on HDFS-6046:

I committed this to 2.4.  Thanks for the reviews.

The findbugs issues were fixed by HDFS-6040.

bq. Interestingly, it appears the libhdfs tests cannot load the native code from within the
embedded JVM. That's worth following up separately.

Yeah.  The specific error is this one:

     [exec] nmdCreate: Builder#build error:
     [exec] java.lang.UnsupportedOperationException: libhadoop cannot be loaded.
     [exec]     at org.apache.hadoop.net.unix.DomainSocketWatcher.<init>(DomainSocketWatcher.java:229)
     [exec]     at org.apache.hadoop.hdfs.client.DfsClientShmManager.<init>(DfsClientShmManager.java:404)
     [exec]     at org.apache.hadoop.hdfs.client.ShortCircuitCache.<init>(ShortCircuitCache.java:380)
     [exec]     at org.apache.hadoop.hdfs.ClientContext.<init>(ClientContext.java:96)

     [exec] TEST_ERROR: failed on /home/jenkins/jenkins-slave/workspace/PreCommit-HDFS-Build/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/native/libhdfs/
   test_libhdfs_threaded.c:328 (errno: 2): got NULL from tlhCluster

Unfortunately, there's not much to go on in the log.  libhadoop.so clearly was built prior
to this point.  I guess we might be setting {{java.library.path}} incorrectly, but that doesn't
explain why it worked previously (and is continuing to work in most test runs).

> add dfs.client.mmap.enabled
> ---------------------------
>                 Key: HDFS-6046
>                 URL: https://issues.apache.org/jira/browse/HDFS-6046
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: hdfs-client
>    Affects Versions: 2.4.0
>            Reporter: Colin Patrick McCabe
>            Assignee: Colin Patrick McCabe
>             Fix For: 2.4.0
>         Attachments: HDFS-6046.001.patch, HDFS-6046.002.patch, HDFS-6046.003.patch
> It would be nice to have a configuration variable, dfs.client.mmap.enabled. which would
disable mmap reads in the client.  This could come in handy for debugging or performance benchmarking.

This message was sent by Atlassian JIRA

View raw message