hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From <tate...@nttdata.co.jp>
Subject RE: fuse-dfs
Date Wed, 13 Jan 2010 02:19:12 GMT
Hi Eli,

Thank you for your reply and advise.

I uncommented "// #define DOTRACE" in fuse_dfs.h.

$ cd $HADOOP_HOME/src/contrib/fuse-dfs/src
$ vi fuse_dfs.h
     		・
     		・         
     53 //#define DOTRACE ⇒ #define DOTRACE
     54 #ifdef DOTRACE
     		・
     		・

Then, I re-compiled that.

$ cd $HADOOP_HOME
$ ant compile-c++-libhdfs -Dlibhdfs=1 -Dcompile.c++=1
$ ln -s c++/Linux-amd64-64/lib/ build/libhdfs
$ ant compile-contrib -Dlibhdfs=1 -Dfusedfs=1

However, it became a same result.
I'm using HADOOP-0.20.1.

■Term A
$ ./fuse_dfs_wrapper.sh dfs://drbd-test-vm03:8020 /mnt/hdfs/ -d
port=8020,server=drbd-test-vm03
fuse-dfs didn't recognize /mnt/hdfs/,-2
fuse-dfs ignoring option -d
unique: 1, opcode: INIT (26), nodeid: 0, insize: 56
INIT: 7.8
flags=0x00000003
max_readahead=0x00020000
   INIT: 7.8
   flags=0x00000001
   max_readahead=0x00020000
   max_write=0x00020000
   unique: 1, error: 0 (Success), outsize: 40
		・
		・
		・
■Term B
$ df
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/mapper/VolGroup00-LogVol00
                       9047928   6148328   2436000  72% /
/dev/xvda1              101086     13230     82637  14% /boot
tmpfs                  1048576         0   1048576   0% /dev/shm
/dev/drbd0             4925336    141244   4533892   4% /drbd
$ ls -ltr /mnt/hdfs
ls: reading directory /mnt/hdfs: Input/output error
total 0

I executed "ls -ltr /mnt/hdfs" several times until a different result came out.
$ ls -ltr /mnt/hdfs
total 0
?--------- ? ? ? ?            ? l

■Results
When I executed "df" in Term B,
It became the following result in Term A:

unique: 2, opcode: STATFS (17), nodeid: 1, insize: 40
fuse_dfs TRACE - statfs /
   unique: 2, error: 0 (Success), outsize: 96

When I executed "ls -ltr /mnt/hdfs" in Term B,
It became the following result in Term A:

unique: 3, opcode: GETATTR (3), nodeid: 1, insize: 40
fuse_dfs TRACE - getattr /
   unique: 3, error: 0 (Success), outsize: 112
unique: 4, opcode: GETXATTR (22), nodeid: 1, insize: 72
   unique: 4, error: -38 (Function not implemented), outsize: 16
unique: 5, opcode: GETATTR (3), nodeid: 1, insize: 40
fuse_dfs TRACE - getattr /
   unique: 5, error: 0 (Success), outsize: 112
unique: 6, opcode: OPENDIR (27), nodeid: 1, insize: 48
   unique: 6, error: 0 (Success), outsize: 32
unique: 7, opcode: GETATTR (3), nodeid: 1, insize: 40
fuse_dfs TRACE - getattr /
   unique: 7, error: 0 (Success), outsize: 112
unique: 8, opcode: READDIR (28), nodeid: 1, insize: 64
fuse_dfs TRACE - readdir /
   unique: 8, error: 0 (Success), outsize: 104
unique: 9, opcode: RELEASEDIR (29), nodeid: 1, insize: 64
   unique: 9, error: 0 (Success), outsize: 16

When I executed "ls -ltr /mnt/hdfs" several times in Term B,
It became the following different result in Term A:

unique: 44, opcode: GETATTR (3), nodeid: 1, insize: 40
fuse_dfs TRACE - getattr /
   unique: 44, error: 0 (Success), outsize: 112
unique: 45, opcode: GETATTR (3), nodeid: 1, insize: 40
fuse_dfs TRACE - getattr /
   unique: 45, error: 0 (Success), outsize: 112
unique: 46, opcode: OPENDIR (27), nodeid: 1, insize: 48
   unique: 46, error: 0 (Success), outsize: 32
unique: 47, opcode: GETATTR (3), nodeid: 1, insize: 40
fuse_dfs TRACE - getattr /
   unique: 47, error: 0 (Success), outsize: 112
unique: 48, opcode: READDIR (28), nodeid: 1, insize: 64
fuse_dfs TRACE - readdir /
   unique: 48, error: 0 (Success), outsize: 112
unique: 49, opcode: LOOKUP (1), nodeid: 1, insize: 42
LOOKUP /l
fuse_dfs TRACE - getattr /l
   unique: 49, error: -2 (No such file or directory), outsize: 16
unique: 50, opcode: READDIR (28), nodeid: 1, insize: 64
   unique: 50, error: 0 (Success), outsize: 16
unique: 51, opcode: RELEASEDIR (29), nodeid: 1, insize: 64
   unique: 51, error: 0 (Success), outsize: 16

Are there anything else I can do?

Best Regards,
Tadashi.

> -----Original Message-----
> From: Eli Collins [mailto:eli@cloudera.com]
> Sent: Wednesday, January 13, 2010 5:20 AM
> To: hdfs-user@hadoop.apache.org
> Subject: Re: fuse-dfs
> 
> Hey Tadashi,
> 
> Think the next step is to uncomment "// #define DOTRACE" in
> fuse_dfs.h, re-compile and see what output that yields. I'd also be
> curious if you see the issue using 20.1.
> 
> Thanks,
> Eli
> 
> 2010/1/11  <tatebet@nttdata.co.jp>:
> > Hi Eli,
> >
> > Thank you for your reply,
> >
> >> Is this the same exact hostname (drbd-test-vm03:8020) you use for
> >> fs.default.name in hadoop-site.xml/core-site.xml? They need to match
> >> up.
> >
> > I use the same exact hostname(drbd-test-vm03).
> >
> > Core-site.xml
> > <property>
> >  <name>fs.default.name</name>
> >  <value>hdfs://drbd-test-vm03/</value>
> > </property>
> >
> > Are there anything else I should check?
> >
> > Best regards,
> > Tadashi
> >> -----Original Message-----
> >> From: Eli Collins [mailto:eli@cloudera.com]
> >> Sent: Sunday, January 10, 2010 6:03 PM
> >> To: hdfs-user@hadoop.apache.org
> >> Subject: Re: fuse-dfs
> >>
> >> > I executed the following command with term A:
> >> > ./fuse_dfs_wrapper.sh dfs://drbd-test-vm03:8020 /mnt/hdfs -d
> >>
> >> Is this the same exact hostname (drbd-test-vm03:8020) you use for
> >> fs.default.name in hadoop-site.xml/core-site.xml? They need to match
> >> up.
> >>
> >> Thanks,
> >> Eli
> >
Mime
View raw message