hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Nauroth (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-5950) The DFSClient and DataNode should use shared memory segments to communicate short-circuit information
Date Mon, 03 Mar 2014 19:24:26 GMT

    [ https://issues.apache.org/jira/browse/HDFS-5950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13918464#comment-13918464
] 

Chris Nauroth commented on HDFS-5950:
-------------------------------------

Hi, [~cmccabe] and [~andrew.wang].  Since this patch got committed, I'm having trouble running
HDFS tests without the native code loaded.  It appears to be caused by HDFS client initialization
always ultimately trying to create a {{DomainSocketWatcher}}, even on platforms where it is
unsupported.  I'm testing on Mac right now.  I imagine Windows has the same problem, because
{{DomainSocketWatcher}} also exits early with an error if it detects Windows.  Here is the
stack trace:

{code}
java.lang.UnsupportedOperationException: libhadoop cannot be loaded.
	at org.apache.hadoop.net.unix.DomainSocketWatcher.<init>(DomainSocketWatcher.java:229)
	at org.apache.hadoop.hdfs.client.DfsClientShmManager.<init>(DfsClientShmManager.java:404)
	at org.apache.hadoop.hdfs.client.ShortCircuitCache.<init>(ShortCircuitCache.java:380)
	at org.apache.hadoop.hdfs.ClientContext.<init>(ClientContext.java:96)
	at org.apache.hadoop.hdfs.ClientContext.get(ClientContext.java:145)
	at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:587)
	at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:507)
	at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:497)
	at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:488)
	at org.apache.hadoop.hdfs.MiniDFSCluster.waitActive(MiniDFSCluster.java:1967)
	at org.apache.hadoop.hdfs.MiniDFSCluster.waitActive(MiniDFSCluster.java:1989)
	at org.apache.hadoop.hdfs.MiniDFSCluster.startDataNodes(MiniDFSCluster.java:1303)
	at org.apache.hadoop.hdfs.MiniDFSCluster.initMiniDFSCluster(MiniDFSCluster.java:718)
	at org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:374)
	at org.apache.hadoop.hdfs.MiniDFSCluster$Builder.build(MiniDFSCluster.java:355)
	at org.apache.hadoop.hdfs.server.namenode.TestNameNodeAcl.init(TestNameNodeAcl.java:35)
{code}


> The DFSClient and DataNode should use shared memory segments to communicate short-circuit
information
> -----------------------------------------------------------------------------------------------------
>
>                 Key: HDFS-5950
>                 URL: https://issues.apache.org/jira/browse/HDFS-5950
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: datanode, hdfs-client
>    Affects Versions: 2.4.0
>            Reporter: Colin Patrick McCabe
>            Assignee: Colin Patrick McCabe
>             Fix For: 2.4.0
>
>         Attachments: HDFS-5950.001.patch, HDFS-5950.003.patch, HDFS-5950.004.patch, HDFS-5950.006.patch,
HDFS-5950.007.patch, HDFS-5950.008.patch
>
>
> The DFSClient and DataNode should use the shared memory segments and unified cache added
in the other HDFS-5182 subtasks to communicate short-circuit information.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message