hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bastian Lorenz <bstn.l...@googlemail.com>
Subject Re: HDFS-Fuse/virtual machine images
Date Tue, 11 May 2010 14:42:28 GMT
> Hi everybody,
>
>
> i have set up an hdfs cluster with three nodes(one master, two
> datanodes). Everything works fine…so far! Now i want to mount my hdfs
> with Fuse. This works also. The mountpoint is /export/hdfs. I want to
> store qcow2 files in hdfs an boot from an iso which is also stored in
> hdfs.
>
> The ‘Big Picture‘: I want storing (and booting) virtual machines (with
> kvm) with hdfs
>
> Fuse is mounted with following command:
>
> $$ fuse_dfs_wrapper.sh dfs://NameNode:9000 -obig_writes /export/hdfs/ -d
>
>
> I create the qcow2 file and start the vm with the following command:
>
> $$virt-install --connect qemu:///system -n vm10 -r 512 --vcpus=2 -f
> /export/hdfs/vm1/vm10.qcow2 -s 12 -c
> /export/hdfs/debian-504-amd64-netinst.iso --vnc --noautoconsole
> --os-type linux --os-variant debianLenny --accelerate
> --network=bridge:br0 –hvm
>
> But when i run this command, i got the following error message from Fuse
> output:
>
> port=9000,server=NameNode
>
> fuse-dfs didn't recognize /export/hdfs/,-2
>
> fuse-dfs ignoring option -d
>
> FUSE library version: 2.8.4
>
> nullpath_ok: 0
>
> unique: 1, opcode: INIT (26), nodeid: 0, insize: 56
>
> INIT: 7.13
>
> flags=0x0000007b
>
> max_readahead=0x00020000
>
>   INIT: 7.12
>
>   flags=0x00000031
>
>   max_readahead=0x00020000
>
>   max_write=0x00020000
>
>   unique: 1, success, outsize: 40
>
> unique: 2, opcode: GETATTR (3), nodeid: 1, insize: 56
>
> getattr /
>
>   unique: 2, success, outsize: 120
>
> unique: 3, opcode: LOOKUP (1), nodeid: 1, insize: 44
>
> LOOKUP /vm1
>
> getattr /vm1
>
>   NODEID: 2
>
>   unique: 3, success, outsize: 144
>
> unique: 4, opcode: LOOKUP (1), nodeid: 2, insize: 51
>
> LOOKUP /vm1/vm10.qcow2
>
> getattr /vm1/vm10.qcow2
>
>   unique: 4, error: -2 (No such file or directory), outsize: 16
>
> unique: 5, opcode: LOOKUP (1), nodeid: 2, insize: 51
>
> LOOKUP /vm1/vm10.qcow2
>
> getattr /vm1/vm10.qcow2
>
>   unique: 5, error: -2 (No such file or directory), outsize: 16
>
> unique: 6, opcode: LOOKUP (1), nodeid: 2, insize: 51
>
> LOOKUP /vm1/vm10.qcow2
>
> getattr /vm1/vm10.qcow2
>
>   unique: 6, error: -2 (No such file or directory), outsize: 16
>
> unique: 7, opcode: LOOKUP (1), nodeid: 2, insize: 51
>
> LOOKUP /vm1/vm10.qcow2
>
> getattr /vm1/vm10.qcow2
>
>   unique: 7, error: -2 (No such file or directory), outsize: 16
>
> unique: 8, opcode: LOOKUP (1), nodeid: 2, insize: 51
>
> LOOKUP /vm1/vm10.qcow2
>
> getattr /vm1/vm10.qcow2
>
>   unique: 8, error: -2 (No such file or directory), outsize: 16
>
> unique: 9, opcode: STATFS (17), nodeid: 2, insize: 40
>
> statfs /vm1
>
>   unique: 9, success, outsize: 96
>
> unique: 10, opcode: LOOKUP (1), nodeid: 2, insize: 51
>
> LOOKUP /vm1/vm10.qcow2
>
> getattr /vm1/vm10.qcow2
>
>   unique: 10, error: -2 (No such file or directory), outsize: 16
>
> unique: 11, opcode: STATFS (17), nodeid: 2, insize: 40
>
> statfs /vm1
>
>   unique: 11, success, outsize: 96
>
> unique: 12, opcode: LOOKUP (1), nodeid: 1, insize: 69
>
> LOOKUP /debian-504-amd64-netinst.iso
>
> getattr /debian-504-amd64-netinst.iso
>
>   NODEID: 3
>
>   unique: 12, success, outsize: 144
>
> unique: 13, opcode: LOOKUP (1), nodeid: 2, insize: 51
>
> LOOKUP /vm1/vm10.qcow2
>
> getattr /vm1/vm10.qcow2
>
>   unique: 13, error: -2 (No such file or directory), outsize: 16
>
> unique: 14, opcode: LOOKUP (1), nodeid: 2, insize: 51
>
> LOOKUP /vm1/vm10.qcow2
>
> getattr /vm1/vm10.qcow2
>
>   unique: 14, error: -2 (No such file or directory), outsize: 16
>
> unique: 15, opcode: LOOKUP (1), nodeid: 2, insize: 51
>
> LOOKUP /vm1/vm10.qcow2
>
> getattr /vm1/vm10.qcow2
>
>   unique: 15, error: -2 (No such file or directory), outsize: 16
>
> unique: 16, opcode: LOOKUP (1), nodeid: 2, insize: 51
>
> LOOKUP /vm1/vm10.qcow2
>
> getattr /vm1/vm10.qcow2
>
>   unique: 16, error: -2 (No such file or directory), outsize: 16
>
> unique: 17, opcode: LOOKUP (1), nodeid: 2, insize: 51
>
> LOOKUP /vm1/vm10.qcow2
>
> getattr /vm1/vm10.qcow2
>
>   unique: 17, error: -2 (No such file or directory), outsize: 16
>
> unique: 18, opcode: STATFS (17), nodeid: 2, insize: 40
>
> statfs /vm1
>
>   unique: 18, success, outsize: 96
>
> unique: 19, opcode: LOOKUP (1), nodeid: 2, insize: 51
>
> LOOKUP /vm1/vm10.qcow2
>
> getattr /vm1/vm10.qcow2
>
>   unique: 19, error: -2 (No such file or directory), outsize: 16
>
> unique: 20, opcode: LOOKUP (1), nodeid: 2, insize: 51
>
> LOOKUP /vm1/vm10.qcow2
>
> getattr /vm1/vm10.qcow2
>
>   unique: 20, error: -2 (No such file or directory), outsize: 16
>
> unique: 21, opcode: CREATE (35), nodeid: 2, insize: 67
>
> create flags: 0x8041 /vm1/vm10.qcow2 0100755 umask=0022
>
> WARN: hdfs does not truly support O_CREATE && O_EXCL
>
>   create[140030497597744] flags: 0x81ed /vm1/vm10.qcow2
>
> getattr /vm1/vm10.qcow2
>
>   NODEID: 4
>
>   unique: 21, success, outsize: 160
>
> unique: 22, opcode: SETATTR (4), nodeid: 4, insize: 128
>
> truncate /vm1/vm10.qcow2 12884901888
>
>   unique: 22, error: -95 (Operation not supported), outsize: 16
>
> unique: 23, opcode: FLUSH (25), nodeid: 4, insize: 64
>
> flush[140030497597744]
>
>   unique: 23, success, outsize: 16
>
> unique: 24, opcode: RELEASE (18), nodeid: 4, insize: 64
>
> release[140030497597744] flags: 0x8001
>
>   unique: 24, success, outsize: 16
>
>
> Why i can´t store qcow2 files in hdfs? Got somebody an idea?
>
> Can anyone help?
>
>
>
> Thanks a lot!
>
>
>
> Greetings from germany!
>
>
>
> basti
>

Mime
View raw message