hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harsh J <ha...@cloudera.com>
Subject Re: fuse-dfs on hadoop-2.2.0
Date Tue, 27 May 2014 09:51:58 GMT
I forgot to send this earlier, but here's an answer with added links
that may help: http://stackoverflow.com/a/21655102/1660002

On Sat, May 17, 2014 at 9:54 AM, Harsh J <harsh@cloudera.com> wrote:
> The issue here is that JNI doesn't like wildcards in the classpath
> string - it does not evaluate them the same way the regular runtime
> does. Try placing a full list of explicit jars on the classpath and it
> will not throw that Class-Not-Found error anymore.
>
> On Thu, May 15, 2014 at 10:54 AM, Cheng Cheng <chengcheng@purdue.edu> wrote:
>> Hi All,
>>
>> With hadoop-2.2.0, I tried to mount hdfs by using fuse-dfs. I have successfully compiled
hadoop-2.2.0.tar.gz and fuse-dfs with "mvn package -Pdist,native -DskipTests -Dtar -e -X”
from hadoop-2.2.0-src.
>>
>> After I deployed hadoop-2.2.0.tar.gz and run single mode HDFS, I tried to run following
command to mount hdfs:
>>
>> ------------------------------------------------
>> LD_LIBRARY_PATH=/opt/hadoop-2.2.0/lib/native:/usr/java/jdk1.7.0_55/jre/lib/amd64/server
CLASSPATH=/opt/hadoop-2.2.0/etc/hadoop:/opt/hadoop-2.2.0/share/hadoop/common/lib/*:/opt/hadoop-2.2.0/share/hadoop/common/*:/opt/hadoop-2.2.0/share/hadoop/hdfs:/opt/hadoop-2.2.0/share/hadoop/hdfs/lib/*:/opt/hadoop-2.2.0/share/hadoop/hdfs/*:/opt/hadoop-2.2.0/share/hadoop/yarn/lib/*:/opt/hadoop-2.2.0/share/hadoop/yarn/*:/opt/hadoop-2.2.0/share/hadoop/mapreduce/lib/*:/opt/hadoop-2.2.0/share/hadoop/mapreduce/*:/opt/hadoop-2.2.0/contrib/capacity-scheduler/*.jar
bin/fuse_dfs -d dfs://localhost:8020 /mnt/hdfs
>> ------------------------------------------------
>>
>> However, I got failed with error message:
>>
>> ------------------------------------------------
>> INFO /root/hadoop-2.2.0-src/hadoop-hdfs-project/hadoop-hdfs/src/main/native/fuse-dfs/fuse_options.c:115
Ignoring option -d
>> INFO /root/hadoop-2.2.0-src/hadoop-hdfs-project/hadoop-hdfs/src/main/native/fuse-dfs/fuse_options.c:164
Adding FUSE arg /mnt/hdfs
>> FUSE library version: 2.8.3
>> nullpath_ok: 0
>> unique: 1, opcode: INIT (26), nodeid: 0, insize: 56
>> INIT: 7.13
>> flags=0x0000b07b
>> max_readahead=0x00020000
>> INFO /root/hadoop-2.2.0-src/hadoop-hdfs-project/hadoop-hdfs/src/main/native/fuse-dfs/fuse_init.c:98
Mounting with options: [ protected=(NULL), nn_uri=hdfs://nebula-vm14.cs.purdue.edu:8020, nn_port=0,
debug=0, read_only=0, initchecks=0, no_permissions=0, usetrash=0, entry_timeout=60, attribute_timeout=60,
rdbuffer_size=10485760, direct_io=0 ]
>> loadFileSystems error:
>> (unable to get stack trace for java.lang.NoClassDefFoundError exception: ExceptionUtils::getStackTrace
error.)
>> hdfsConfGetInt(hadoop.fuse.timer.period): new Configuration error:
>> (unable to get stack trace for java.lang.NoClassDefFoundError exception: ExceptionUtils::getStackTrace
error.)
>> Unable to determine the configured value for hadoop.fuse.timer.period.ERROR /root/hadoop-2.2.0-src/hadoop-hdfs-project/hadoop-hdfs/src/main/native/fuse-dfs/fuse_init.c:134
FATAL: dfs_init: fuseConnectInit failed with error -22!
>> ERROR /root/hadoop-2.2.0-src/hadoop-hdfs-project/hadoop-hdfs/src/main/native/fuse-dfs/fuse_init.c:34
LD_LIBRARY_PATH=/opt/hadoop-2.2.0/lib/native:/usr/java/jdk1.7.0_55/jre/lib/amd64/server
>> ERROR /root/hadoop-2.2.0-src/hadoop-hdfs-project/hadoop-hdfs/src/main/native/fuse-dfs/fuse_init.c:35
CLASSPATH=/opt/hadoop-2.2.0/etc/hadoop:/opt/hadoop-2.2.0/share/hadoop/common/lib/*:/opt/hadoop-2.2.0/share/hadoop/common/*:/opt/hadoop-2.2.0/share/hadoop/hdfs:/opt/hadoop-2.2.0/share/hadoop/hdfs/lib/*:/opt/hadoop-2.2.0/share/hadoop/hdfs/*:/opt/hadoop-2.2.0/share/hadoop/yarn/lib/*:/opt/hadoop-2.2.0/share/hadoop/yarn/*:/opt/hadoop-2.2.0/share/hadoop/mapreduce/lib/*:/opt/hadoop-2.2.0/share/hadoop/mapreduce/*:/opt/hadoop-2.2.0/contrib/capacity-scheduler/*.jar
>> ------------------------------------------------
>>
>> It looks like the configuration is not correctly loaded. But I have already configured
 “hadoop.fuse.timer.period" and “hadoop.fuse.connection.timeout" in $HADOOP_HOME/etc/hadoop/hdfs-site.xml.
>>
>> Can any one share some hints on how to fix this? How can I let fuse-dfs correctly
load the configuration?
>>
>> Thanks in advance!
>> Cheng
>
>
>
> --
> Harsh J



-- 
Harsh J

Mime
View raw message