hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stuti Awasthi <stutiawas...@hcl.com>
Subject RE: Facing issue in building Fuse-DFS
Date Fri, 09 Dec 2011 16:44:56 GMT
Hi all,

I finally resolved the issue and able to mount HDFS on Ubuntu. Thanks everyone


From: Stuti Awasthi
Sent: Friday, December 09, 2011 3:10 PM
To: hdfs-user@hadoop.apache.org
Subject: RE: Facing issue in building Fuse-DFS

Hi Kartheek, Alo

Good news is I am finally able to build fuse-dfs successfully. After the build now build/contrib/fuse-dfs/fuse_dfs
is created.
Issue was with libhdfs which was not compiling properly.

Command which worked for me :
ant compile-c++-libhdfs -Dlibhdfs=1 -Dislibhdfs=1

When now I try to mount, I am getting ClientProtocol version mismatch error. This is common
error in HDFS related to correct jar files in class path. I will try to resolve it and will
get back if faces any issues.

ThankU Guys so much for help.

Cheers



From: kartheek muthyala [mailto:kartheek0274@gmail.com]
Sent: Thursday, December 08, 2011 4:47 PM
To: hdfs-user@hadoop.apache.org
Subject: Re: Facing issue in building Fuse-DFS

Hi stuti,
I have installed fuse_dfs in ubuntu lot of times. I have written a script for this. See if
this helps

ant compile -Dcompile.c++=true -Dlibhdfs=true
ant -Dcompile.native=true package -Djava5.home=/usr/lib/jvm/java-1.5.0-sun -Dforrest.home=/p/apache-forrest-0.8
ant compile-contrib -Dlibhdfs=1 -Dfusedfs=1 -Djava5.home=/usr/lib/jvm/java-1.5.0-sun

~Kartheek
On Thu, Dec 8, 2011 at 3:01 PM, Stuti Awasthi <stutiawasthi@hcl.com<mailto:stutiawasthi@hcl.com>>
wrote:
The environment variable which I set :

 export ANT_HOME=/home/stuti/Installer/apache-ant-1.8.2/
 export PATH=$PATH:/home/stuti/Installer/apache-ant-1.8.2/bin/
 export HADOOP_HOME=/home/stuti/Installer/hadoop-0.20.2/
 export JAVA_HOME=/usr/lib/jvm/java-6-openjdk/
 export PATH=$PATH:/usr/lib/jvm/java-6-openjdk/bin/
 ln -s /home/stuti/Installer/hadoop-0.20.2/build/c++/Linux-i386-32/lib $HADOOP_HOME/build/libhdfs
 export LD_LIBRARY_PATH=/usr/lib:/usr/local/lib:$HADOOP_HOME/build/libhdfs:$JAVA_HOME/jre/lib/i386/server/
 export FUSE_HOME=/usr/include/fuse
 export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/lib/libfuse.so

-----Original Message-----
From: Stuti Awasthi
Sent: Thursday, December 08, 2011 2:53 PM
To: hdfs-user@hadoop.apache.org<mailto:hdfs-user@hadoop.apache.org>
Subject: RE: Facing issue in building Fuse-DFS
Here are the build logs. I also found out that after building no folder of fuse-dfs is created
inside $HADOOP_HOME/build/

Logs are :
1. ant compile-c++-libhdfs -Dlibhdfs=1 > compile.txt 2. ant -Djava5.home=/home/stuti/Installer/JDK1.5/jdk1.5.0_22/
-Dforrest.home=/home/stuti/Installer/apache-forrest-0.8/ package > package.txt 3. ant compile-contrib
-Dlibhdfs=1 -Dfusedfs=1 > compile-contrib.txt

I have also fired command : "ant compile -Dcompile.c++=true -Dlibhdfs=true " which I have
not captured in logs. Let me know if that is also required.
Thanks

From: alo alt [mailto:wget.null@googlemail.com<mailto:wget.null@googlemail.com>]
Sent: Thursday, December 08, 2011 1:42 PM
To: hdfs-user@hadoop.apache.org<mailto:hdfs-user@hadoop.apache.org>
Subject: Re: Facing issue in building Fuse-DFS

Hi Stuti,

I've build them on CentOS, with Ubuntu I never tried. Could you please send the buildlog?
On my servers fuse_dfs is found under /usr/lib/hadoop-0.20/bin/

- alex
On Thu, Dec 8, 2011 at 9:00 AM, Stuti Awasthi <stutiawasthi@hcl.com<mailto:stutiawasthi@hcl.com>>
wrote:
Hi Alo,
My architecture is i386 . I am using Ubuntu 11.4 and added required fuse packages in it. I
am not getting any build errors now. Initially I got many build errors but now I have overcome
by including correct packages and  exporting correct environment  variables.

Thanks



From: alo alt [mailto:wget.null@googlemail.com<mailto:wget.null@googlemail.com>]
Sent: Thursday, December 08, 2011 1:19 PM
To: hdfs-user@hadoop.apache.org<mailto:hdfs-user@hadoop.apache.org>
Subject: Re: Facing issue in building Fuse-DFS

Hi Stuti,

what architecture do you have? If AMD64 lib_fuse wont be build unless you edit the makefile
to macht the architecture. Also your kernel has fuse-support included?

- alex
On Thu, Dec 8, 2011 at 5:34 AM, Stuti Awasthi <stutiawasthi@hcl.com<mailto:stutiawasthi@hcl.com>>
wrote:
Hi all,
I am successful in building Fuse-Dfs but facing issue that executable fuse_dfs is not getting
created but build is getting successful.
I followed : http://wiki.apache.org/hadoop/MountableHDFS

Got Build Successful by executing. Exported required paths in environment

in HADOOP_HOME: ant compile-c++-libhdfs -Dlibhdfs=1 in HADOOP_HOME: ant package to deploy
libhdfs in HADOOP_HOME: ant compile-contrib -Dlibhdfs=1 -Dfusedfs=1

Configured fuse_dfs_wrapper.sh present in $HADOOP_HOME/src/contrib/fuse-dfs/src with JAVA_HOME
and HADOOP_HOME And executed fuse_dfs_wrapper.sh but got the error :
fuse_dfs: not found

I searched but cant find fuse_dfs executable after the build. What I have done wrong. Did
I missed anything.
Please Help. I am so close to get this done.


-----Original Message-----
From: Stuti Awasthi
Sent: Wednesday, November 30, 2011 4:11 PM
To: 'hdfs-user@hadoop.apache.org<mailto:hdfs-user@hadoop.apache.org>'
Subject: RE: Facing issue in building Fuse-DFS

Hi,
I also tried "hdfs-1757-1.patch" but unsuccessful to apply it correctly.

cd $HADOOP_HOME
patch -p1 < ~/Downloads/ hdfs-1757-1.patch

Got errors :(

-----Original Message-----
From: Stuti Awasthi
Sent: Wednesday, November 30, 2011 3:48 PM
To: hdfs-user@hadoop.apache.org<mailto:hdfs-user@hadoop.apache.org>
Subject: RE: Facing issue in building Fuse-DFS

Hi,
After some googling I found this link :
http://search-hadoop.com/m/Ee9Vj1ZNSGR1&subj=Re+Hadoop+Hdfs+trunk+Commit+Build+560+Failure

I tried to apply patch but getting errors in applying the patch.
Here is what I did :

cd $HADOOP_HOME
patch -p1 < ~/Downloads/hdfs-780-4.patch

But got the errors. Please help.

-----Original Message-----
From: Stuti Awasthi
Sent: Wednesday, November 30, 2011 2:19 PM
To: hdfs-user@hadoop.apache.org<mailto:hdfs-user@hadoop.apache.org>
Subject: Facing issue in building Fuse-DFS Hi All,

I am using Hadoop 0.20.2 and tried to build Fuse DFS. After facing lots of issues and resolving
them I am stuck at the following error.
Can please somebody help me in resolving this .

Command : ant compile-contrib -Dlibhdfs=1 -Dfusedfs=1

Error:
[exec] make[1]: Entering directory `/home/stuti/Installer/hadoop-0.20.2/src/contrib/fuse-dfs'
    [exec] make[1]: Nothing to be done for `all-am'.
    [exec] make[1]: Leaving directory `/home/stuti/Installer/hadoop-0.20.2/src/contrib/fuse-dfs'
    [exec] Making all in src
    [exec] make[1]: Entering directory `/home/stuti/Installer/hadoop-0.20.2/src/contrib/fuse-dfs/src'
    [exec] gcc -DPACKAGE_NAME=\"fuse_dfs\" -DPACKAGE_TARNAME=\"fuse_dfs\" -DPACKAGE_VERSION=\"0.1.0\"
-DPACKAGE_STRING=\"fuse_dfs\ 0.1.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DSTDC_HEADERS=1
-DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1
-DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DGETGROUPS_T=gid_t
-DHAVE_GETGROUPS=1 -DGETGROUPS_T=gid_t -I.  -DPERMS=1 -D_FILE_OFFSET_BITS=64 -I/usr/lib/jvm/java-6-openjdk//include
-I/home/stuti/Installer/hadoop-0.20.2/src/c++/libhdfs/ -I/usr/lib/jvm/java-6-openjdk//include/linux/
-D_FUSE_DFS_VERSION=\"0.1.0\" -DPROTECTED_PATHS=\"\" -I/usr/include/fuse/include   -Wall -O3
-MT fuse_dfs.o -MD -MP -MF .deps/fuse_dfs.Tpo -c -o fuse_dfs.o fuse_dfs.c
    [exec] make[1]: Leaving directory `/home/stuti/Installer/hadoop-0.20.2/src/contrib/fuse-dfs/src'
    [exec] In file included from fuse_dfs.c:21:0:
    [exec] fuse_impls.h:37:46: error: expected declaration specifiers or '...' before 'fuse_fill_dir_t'
    [exec] fuse_impls.h:38:38: warning: 'struct fuse_file_info' declared inside parameter
list
    [exec] fuse_impls.h:38:38: warning: its scope is only this definition or declaration,
which is probably not what you want
    [exec] fuse_impls.h:40:28: warning: 'struct fuse_file_info' declared inside parameter
list
    [exec] fuse_impls.h:41:41: warning: 'struct statvfs' declared inside parameter list
    [exec] fuse_impls.h:49:39: warning: 'struct fuse_file_info' declared inside parameter
list
    [exec] fuse_impls.h:51:36: warning: 'struct fuse_file_info' declared inside parameter
list
    [exec] fuse_impls.h:52:43: warning: 'struct fuse_file_info' declared inside parameter
list
    [exec] fuse_impls.h:54:54: warning: 'struct fuse_file_info' declared inside parameter
list
    [exec] fuse_impls.h:55:40: warning: 'struct fuse_file_info' declared inside parameter
list
    [exec] fuse_dfs.c: In function 'is_protected':
    [exec] fuse_dfs.c:27:3: warning: implicit declaration of function 'fuse_get_context'
    [exec] fuse_dfs.c:27:54: error: invalid type argument of '->' (have 'int')
    [exec] fuse_dfs.c: At top level:
    [exec] fuse_dfs.c:40:15: error: variable 'dfs_oper' has initializer but incomplete type
    [exec] fuse_dfs.c:41:3: error: unknown field 'getattr' specified in initializer
    [exec] fuse_dfs.c:41:3: warning: excess elements in struct initializer
    [exec] fuse_dfs.c:41:3: warning: (near initialization for 'dfs_oper')
    [exec] fuse_dfs.c:42:3: error: unknown field 'access' specified in initializer
    [exec] fuse_dfs.c:42:3: warning: excess elements in struct initializer
    [exec] fuse_dfs.c:42:3: warning: (near initialization for 'dfs_oper')
    [exec] fuse_dfs.c:43:3: error: unknown field 'readdir' specified in initializer
    [exec] fuse_dfs.c:43:3: warning: excess elements in struct initializer
    [exec] fuse_dfs.c:43:3: warning: (near initialization for 'dfs_oper')
    [exec] fuse_dfs.c:44:3: error: unknown field 'destroy' specified in initializer
    [exec] fuse_dfs.c:44:3: warning: excess elements in struct initializer
    [exec] fuse_dfs.c:44:3: warning: (near initialization for 'dfs_oper')
    [exec] fuse_dfs.c:45:3: error: unknown field 'init' specified in initializer
    [exec] fuse_dfs.c:45:3: warning: excess elements in struct initializer
    [exec] fuse_dfs.c:45:3: warning: (near initialization for 'dfs_oper')
    [exec] fuse_dfs.c:46:3: error: unknown field 'open' specified in initializer
    [exec] fuse_dfs.c:46:3: warning: excess elements in struct initializer
    [exec] fuse_dfs.c:46:3: warning: (near initialization for 'dfs_oper')
    [exec] fuse_dfs.c:47:3: error: unknown field 'read' specified in initializer
    [exec] fuse_dfs.c:47:3: warning: excess elements in struct initializer
    [exec] fuse_dfs.c:47:3: warning: (near initialization for 'dfs_oper')
    [exec] fuse_dfs.c:48:3: error: unknown field 'symlink' specified in initializer
    [exec] fuse_dfs.c:48:3: warning: excess elements in struct initializer
    [exec] fuse_dfs.c:48:3: warning: (near initialization for 'dfs_oper')
    [exec] fuse_dfs.c:49:3: error: unknown field 'statfs' specified in initializer
    [exec] fuse_dfs.c:49:3: warning: excess elements in struct initializer
    [exec] fuse_dfs.c:49:3: warning: (near initialization for 'dfs_oper')
    [exec] fuse_dfs.c:50:3: error: unknown field 'mkdir' specified in initializer
    [exec] fuse_dfs.c:50:3: warning: excess elements in struct initializer
    [exec] fuse_dfs.c:50:3: warning: (near initialization for 'dfs_oper')
    [exec] fuse_dfs.c:51:3: error: unknown field 'rmdir' specified in initializer
    [exec] fuse_dfs.c:51:3: warning: excess elements in struct initializer
    [exec] fuse_dfs.c:51:3: warning: (near initialization for 'dfs_oper')
    [exec] fuse_dfs.c:52:3: error: unknown field 'rename' specified in initializer
    [exec] fuse_dfs.c:52:3: warning: excess elements in struct initializer
    [exec] fuse_dfs.c:52:3: warning: (near initialization for 'dfs_oper')
    [exec] fuse_dfs.c:53:3: error: unknown field 'unlink' specified in initializer
    [exec] fuse_dfs.c:53:3: warning: excess elements in struct initializer
    [exec] fuse_dfs.c:53:3: warning: (near initialization for 'dfs_oper')
    [exec] fuse_dfs.c:54:3: error: unknown field 'release' specified in initializer
    [exec] fuse_dfs.c:54:3: warning: excess elements in struct initializer
    [exec] fuse_dfs.c:54:3: warning: (near initialization for 'dfs_oper')
    [exec] fuse_dfs.c:55:3: error: unknown field 'create' specified in initializer
    [exec] fuse_dfs.c:55:3: warning: excess elements in struct initializer
    [exec] fuse_dfs.c:55:3: warning: (near initialization for 'dfs_oper')
    [exec] fuse_dfs.c:56:3: error: unknown field 'write' specified in initializer
    [exec] fuse_dfs.c:56:3: warning: excess elements in struct initializer
    [exec] fuse_dfs.c:56:3: warning: (near initialization for 'dfs_oper')
    [exec] fuse_dfs.c:57:3: error: unknown field 'flush' specified in initializer
    [exec] fuse_dfs.c:57:3: warning: excess elements in struct initializer
    [exec] fuse_dfs.c:57:3: warning: (near initialization for 'dfs_oper')
    [exec] fuse_dfs.c:58:3: error: unknown field 'mknod' specified in initializer
    [exec] fuse_dfs.c:58:3: warning: excess elements in struct initializer
    [exec] fuse_dfs.c:58:3: warning: (near initialization for 'dfs_oper')
    [exec] fuse_dfs.c:59:2: error: unknown field 'utimens' specified in initializer
    [exec] fuse_dfs.c:59:2: warning: excess elements in struct initializer
    [exec] fuse_dfs.c:59:2: warning: (near initialization for 'dfs_oper')
    [exec] fuse_dfs.c:60:3: error: unknown field 'chmod' specified in initializer
    [exec] fuse_dfs.c:60:3: warning: excess elements in struct initializer
    [exec] fuse_dfs.c:60:3: warning: (near initialization for 'dfs_oper')
    [exec] fuse_dfs.c:61:3: error: unknown field 'chown' specified in initializer
    [exec] fuse_dfs.c:61:3: warning: excess elements in struct initializer
    [exec] fuse_dfs.c:61:3: warning: (near initialization for 'dfs_oper')
    [exec] fuse_dfs.c:62:3: error: unknown field 'truncate' specified in initializer
    [exec] fuse_dfs.c:62:3: warning: excess elements in struct initializer
    [exec] fuse_dfs.c:62:3: warning: (near initialization for 'dfs_oper')
    [exec] fuse_dfs.c: In function 'main':
    [exec] fuse_dfs.c:132:3: warning: implicit declaration of function 'fuse_main'
    [exec] make[1]: *** [fuse_dfs.o] Error 1
    [exec] make: *** [all-recursive] Error 1

BUILD FAILED
/home/stuti/Installer/hadoop-0.20.2/build.xml:497: The following error occurred while executing
this line:
/home/stuti/Installer/hadoop-0.20.2/src/contrib/build.xml:30: The following error occurred
while executing this line:
/home/stuti/Installer/hadoop-0.20.2/src/contrib/fuse-dfs/build.xml:57: exec returned: 2

-Stuti

::DISCLAIMER::
-----------------------------------------------------------------------------------------------------------------------

The contents of this e-mail and any attachment(s) are confidential and intended for the named
recipient(s) only.
It shall not attach any liability on the originator or HCL or its affiliates. Any views or
opinions presented in this email are solely those of the author and may not necessarily reflect
the opinions of HCL or its affiliates.
Any form of reproduction, dissemination, copying, disclosure, modification, distribution and
/ or publication of this message without the prior written consent of the author of this e-mail
is strictly prohibited. If you have received this email in error please delete it and notify
the sender immediately. Before opening any mail and attachments please check them for viruses
and defect.

-----------------------------------------------------------------------------------------------------------------------




--
Alexander Lorenz
http://mapredit.blogspot.com

P Think of the environment: please don't print this email unless you really need to.






--
Alexander Lorenz
http://mapredit.blogspot.com

P Think of the environment: please don't print this email unless you really need to.


Mime
View raw message