hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stuti Awasthi <stutiawas...@hcl.com>
Subject RE: Mounting HDFS
Date Wed, 11 Jan 2012 16:10:13 GMT
Hi Alex, Kartheek,

Thanks for response. I did a clean fresh setup on a new machine CentOS- 64bit currently with
open-jdk installed and able to resolve the previous issue of No ClassDefinition found error.
http://wiki.apache.org/hadoop/MountableHDFS:
Steps worked properly till "Installing Step" and I am able to mount HDFS on /mnt through fuse_dfs_wrapper.sh
script. When I try to follow the "Deploying Step", I get the following error :

[root@slave ~]# mount /mnt
port=54310,server=slave
fuse-dfs didn't recognize /mnt,-2
fuse-dfs ignoring option -oallow_other
fuse-dfs ignoring option -ousetrash
fuse-dfs ignoring option dev
fuse-dfs ignoring option suid
fuse: unknown option `-oallow_other'

I searched this and found out  "http://sourceforge.net/apps/mediawiki/fuse/index.php?title=FAQ#Why_does_fusermount_fail_with_an_Unknown_option_error.3F"
I tried removing fusermount file from /bin and also removed symlink from /usr/bin but still
getting this issue.

Please suggest.
Thanks

From: kartheek muthyala [mailto:kartheek0274@gmail.com]
Sent: Tuesday, January 10, 2012 9:31 AM
To: hdfs-user@hadoop.apache.org
Subject: Re: Mounting HDFS

Stuti,
Can you send us the syslog details? (/var/log/syslog)
Thanks,
Kartheek.
On Tue, Jan 10, 2012 at 1:36 AM, alo.alt <wget.null@googlemail.com<mailto:wget.null@googlemail.com>>
wrote:
Hmm, do you have fuse installed?
http://fuse.sourceforge.net/

- Alex

--
Alexander Lorenz
http://mapredit.blogspot.com
On Jan 9, 2012, at 3:55 AM, Stuti Awasthi wrote:

> Hi Alo,
> I tried to fresh build of fuse-dfs and this time I mount in /mnt but facing same issues.
I edited the "fuse_dfs_wrapper.sh" script and added $JAVA_HOME/jre/lib/*.jar in the CLASSPATH
, echoed the env variable and then tried again but no luck. Getting same error. Below is the
trace:
>
> [root@slave fuse-dfs]# ./fuse_dfs_wrapper.sh dfs://slave:54310 /mnt -d
>
> HADOOP_HOME= /root/MountHDFS1/hadoop-0.20.2
>
> CLASSPATH= :ls:/root/MountHDFS1/hadoop-0.20.2/lib/commons-cli-1.2.jar:/root/MountHDFS1/hadoop-0.20.2/lib/commons-codec-1.3.jar:/root/MountHDFS1/hadoop-0.20.2/lib/commons-el-1.0.jar:/root/MountHDFS1/hadoop-0.20.2/lib/commons-httpclient-3.0.1.jar:/root/MountHDFS1/hadoop-0.20.2/lib/commons-logging-1.0.4.jar:/root/MountHDFS1/hadoop-0.20.2/lib/commons-logging-api-1.0.4.jar:/root/MountHDFS1/hadoop-0.20.2/lib/commons-net-1.4.1.jar:/root/MountHDFS1/hadoop-0.20.2/lib/core-3.1.1.jar:/root/MountHDFS1/hadoop-0.20.2/lib/hsqldb-1.8.0.10.jar:/root/MountHDFS1/hadoop-0.20.2/lib/jasper-compiler-5.5.12.jar:/root/MountHDFS1/hadoop-0.20.2/lib/jasper-runtime-5.5.12.jar:/root/MountHDFS1/hadoop-0.20.2/lib/jets3t-0.6.1.jar:/root/MountHDFS1/hadoop-0.20.2/lib/jetty-6.1.14.jar:/root/MountHDFS1/hadoop-0.20.2/lib/jetty-util-6.1.14.jar:/root/MountHDFS1/hadoop-0.20.2/lib/junit-3.8.1.jar:/root/MountHDFS1/hadoop-0.20.2/lib/kfs-0.2.2.jar:/root/MountHDFS1/hadoop-0.20.2/lib/log4j-1.2.15.jar:/root/MountHDFS1/hadoop-0.20.2/lib/mockito-all-1.8.0.jar:/root/MountHDFS1/hadoop-0.20.2/lib/oro-2.0.8.jar:/root/MountHDFS1/hadoop-0.20.2/lib/servlet-api-2.5-6.1.14.jar:/root/MountHDFS1/hadoop-0.20.2/lib/slf4j-api-1.4.3.jar:/root/MountHDFS1/hadoop-0.20.2/lib/slf4j-log4j12-1.4.3.jar:/root/MountHDFS1/hadoop-0.20.2/lib/xmlenc-0.52.jar:/root/MountHDFS1/hadoop-0.20.2/hadoop-0.20.2-ant.jar:/root/MountHDFS1/hadoop-0.20.2/hadoop-0.20.2-core.jar:/root/MountHDFS1/hadoop-0.20.2/hadoop-0.20.2-examples.jar:/root/MountHDFS1/hadoop-0.20.2/hadoop-0.20.2-test.jar:/root/MountHDFS1/hadoop-0.20.2/hadoop-0.20.2-tools.jar:ls:/usr/java/jdk1.6.0_30/jre/lib/alt-rt.jar:/usr/java/jdk1.6.0_30/jre/lib/alt-string.jar:/usr/java/jdk1.6.0_30/jre/lib/charsets.jar:/usr/java/jdk1.6.0_30/jre/lib/deploy.jar:/usr/java/jdk1.6.0_30/jre/lib/javaws.jar:/usr/java/jdk1.6.0_30/jre/lib/jce.jar:/usr/java/jdk1.6.0_30/jre/lib/jsse.jar:/usr/java/jdk1.6.0_30/jre/lib/management-agent.jar:/usr/java/jdk1.6.0_30/jre/lib/plugin.jar:/usr/java/jdk1.6.0_30/jre/lib/resources.jar:/usr/java/jdk1.6.0_30/jre/lib/rt.jar:/usr/java/jdk1.6.0_30/bin
>
> JAVA_HOME= /usr/java/jdk1.6.0_30
>
> LD_LIBRARY_PATH= /usr/lib:/usr/local/lib:/root/MountHDFS1/hadoop-0.20.2/build/libhdfs:/usr/java/jdk1.6.0_30/jre/lib/i386/server/:/lib/libfuse.so
>
> port=54310,server=slave
> fuse-dfs didn't recognize /mnt,-2
> fuse-dfs ignoring option -d
> unique: 1, opcode: INIT (26), nodeid: 0, insize: 56
> INIT: 7.10
> flags=0x0000000b
> max_readahead=0x00020000
>   INIT: 7.8
>   flags=0x00000001
>   max_readahead=0x00020000
>   max_write=0x00020000
>   unique: 1, error: 0 (Success), outsize: 40
> unique: 2, opcode: GETATTR (3), nodeid: 1, insize: 56
> Error occurred during initialization of VM
> java/lang/NoClassDefFoundError: java/lang/Object
>
> Im stuck with this. This is the output with fresh build of fuse-dfs. CLASSPATH variable
contains Hadoop*.jar, and $JAVA_HOME/jre/lib/*jar files. JAVA_HOME is set differently also
> What I am doing wrong. Any Idea ??
>
> Thanks
>
>
> -----Original Message-----
> From: alo.alt [mailto:wget.null@googlemail.com<mailto:wget.null@googlemail.com>]
> Sent: Saturday, January 07, 2012 11:20 PM
> To: hdfs-user@hadoop.apache.org<mailto:hdfs-user@hadoop.apache.org>
> Subject: Re: Mounting HDFS
>
> try:
> ./fuse_dfs_wrapper.sh dfs://namenode.local:<PORT> /MOUNT_POINT -d
>
> MOUNT_POINT has to be writable, try to use a mount point under / like /hdfs or similar.
Also remove the tailing /
>
> - Alex
>
>
> --
> Alexander Lorenz
> http://mapredit.blogspot.com
>
> On Jan 7, 2012, at 12:35 AM, Stuti Awasthi wrote:
>
>> Hi Alo,Srivas,
>>
>> Thanks for pointing this out. I am still getting the same error. This time with fuse_dfs_wrapper.sh
I echoed the environment variables value also.
>>
>> [root@slave fuse-dfs]# ./fuse_dfs_wrapper.sh dfs://slave:54310
>> /root/FreshMount/mnt1/ -d
>>
>> CLASSPATH=/usr/java/jdk1.6.0_30/jre/lib/rt.jar:/usr/java/jdk1.6.0_30/j
>> re/lib/jce.jar:/usr/java/jdk1.6.0_30/jre/lib/javaws.jar:/usr/java/jdk1
>> .6.0_30/jre/lib/deploy.jar:/usr/java/jdk1.6.0_30/jre/lib/jsse.jar:/usr
>> /java/jdk1.6.0_30/jre/lib/plugin.jar:ls:/root/FreshMount/hadoop-0.20.2
>> /lib/commons-cli-1.2.jar:/root/FreshMount/hadoop-0.20.2/lib/commons-co
>> dec-1.3.jar:/root/FreshMount/hadoop-0.20.2/lib/commons-el-1.0.jar:/roo
>> t/FreshMount/hadoop-0.20.2/lib/commons-httpclient-3.0.1.jar:/root/Fres
>> hMount/hadoop-0.20.2/lib/commons-logging-1.0.4.jar:/root/FreshMount/ha
>> doop-0.20.2/lib/commons-logging-api-1.0.4.jar:/root/FreshMount/hadoop-
>> 0.20.2/lib/commons-net-1.4.1.jar:/root/FreshMount/hadoop-0.20.2/lib/co
>> re-3.1.1.jar:/root/FreshMount/hadoop-0.20.2/lib/hsqldb-1.8.0.10.jar:/r
>> oot/FreshMount/hadoop-0.20.2/lib/jasper-compiler-5.5.12.jar:/root/Fres
>> hMount/hadoop-0.20.2/lib/jasper-runtime-5.5.12.jar:/root/FreshMount/ha
>> doop-0.20.2/lib/jets3t-0.6.1.jar:/root/FreshMount/hadoop-0.20.2/lib/je
>> tty-6.1.14.jar:/root/FreshMount/hadoop-0.20.2/lib/jetty-util-6.1.14.ja
>> r:/root/FreshMount/hadoop-0.20.2/lib/junit-3.8.1.jar:/root/FreshMount/
>> hadoop-0.20.2/lib/kfs-0.2.2.jar:/root/FreshMount/hadoop-0.20.2/lib/log
>> 4j-1.2.15.jar:/root/FreshMount/hadoop-0.20.2/lib/mockito-all-1.8.0.jar
>> :/root/FreshMount/hadoop-0.20.2/lib/oro-2.0.8.jar:/root/FreshMount/had
>> oop-0.20.2/lib/servlet-api-2.5-6.1.14.jar:/root/FreshMount/hadoop-0.20
>> .2/lib/slf4j-api-1.4.3.jar:/root/FreshMount/hadoop-0.20.2/lib/slf4j-lo
>> g4j12-1.4.3.jar:/root/FreshMount/hadoop-0.20.2/lib/xmlenc-0.52.jar:/ro
>> ot/FreshMount/hadoop-0.20.2/hadoop-0.20.2-ant.jar:/root/FreshMount/had
>> oop-0.20.2/hadoop-0.20.2-core.jar:/root/FreshMount/hadoop-0.20.2/hadoo
>> p-0.20.2-examples.jar:/root/FreshMount/hadoop-0.20.2/hadoop-0.20.2-tes
>> t.jar:/root/FreshMount/hadoop-0.20.2/hadoop-0.20.2-tools.jar
>>
>> LD_LIBRARY_PATH=/usr/lib:/usr/local/lib:/root/FreshMount/hadoop-0.20.2
>> /build/libhdfs:/usr/java/jdk1.6.0_30/jre/lib/i386/server/:/lib/libfuse
>> .so
>>
>> JAVA_HOME=/usr/java/jdk1.6.0_30
>>
>> Error:
>> port=54310,server=slave
>> fuse-dfs didn't recognize /root/FreshMount/mnt1/,-2 fuse-dfs ignoring
>> option -d
>> unique: 1, opcode: INIT (26), nodeid: 0, insize: 56
>> INIT: 7.10
>> flags=0x0000000b
>> max_readahead=0x00020000
>>  INIT: 7.8
>>  flags=0x00000001
>>  max_readahead=0x00020000
>>  max_write=0x00020000
>>  unique: 1, error: 0 (Success), outsize: 40
>> unique: 2, opcode: GETATTR (3), nodeid: 1, insize: 56 Error occurred
>> during initialization of VM
>> java/lang/NoClassDefFoundError: java/lang/Object
>>
>> -----Original Message-----
>> From: alo.alt [mailto:wget.null@googlemail.com<mailto:wget.null@googlemail.com>]
>> Sent: Friday, January 06, 2012 10:41 PM
>> To: hdfs-user@hadoop.apache.org<mailto:hdfs-user@hadoop.apache.org>
>> Subject: Re: Mounting HDFS
>>
>> Stuti, define in CLASSPATH="...." only the jars you really need for. An export of
all jars in a given directory is a red flag (done with *.jar).
>>
>> - Alex
>>
>>
>> On Jan 6, 2012, at 7:23 AM, M. C. Srivas wrote:
>>
>>>
>>> unique: 1, error: 0 (Success), outsize: 40
>>> unique: 2, opcode: GETATTR (3), nodeid: 1, insize: 56 Error occurred
>>> during initialization of VM
>>> java/lang/NoClassDefFoundError: java/lang/Object
>>>
>>> Exported Environment Variable:
>>>
>>> CLASSPATH="/root/FreshMount/hadoop-0.20.2/lib/*.jar:/root/FreshMount/hadoop-0.20.2/*.jar:/usr/bin/java:/usr/local/lib:/usr/lib:/usr/:/usr/java/jdk1.6.0_26/jre/lib/rt.jar:/usr/java/jdk1.6.0_26/jre/lib/"
>>>
>>>
>>> CLASSPATH is a list of jars, not a list of directories
>>>
>>>
>>> I know that this is simple Java Classpath Error but I have set JAVA_HOME correctly.
>>>
>>> [root@slave ~]# which java
>>> /usr/bin/java
>>>
>>
>>
>> ::DISCLAIMER::
>> ----------------------------------------------------------------------
>> -------------------------------------------------
>>
>> The contents of this e-mail and any attachment(s) are confidential and intended for
the named recipient(s) only.
>> It shall not attach any liability on the originator or HCL or its
>> affiliates. Any views or opinions presented in this email are solely those of the
author and may not necessarily reflect the opinions of HCL or its affiliates.
>> Any form of reproduction, dissemination, copying, disclosure,
>> modification, distribution and / or publication of this message
>> without the prior written consent of the author of this e-mail is
>> strictly prohibited. If you have received this email in error please delete it and
notify the sender immediately. Before opening any mail and attachments please check them for
viruses and defect.
>>
>> ----------------------------------------------------------------------
>> -------------------------------------------------
>


Mime
View raw message