hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Christian Baun <c...@unix-ag.uni-kl.de>
Subject Re: iozone is not working with HDFS over libfuse
Date Tue, 06 Jul 2010 18:14:02 GMT
Hi,

After some time, bonnie++ exits too with this error message:

Can't write block.: Invalid argument
Bonnie: drastic I/O error (re write(2)): Invalid argument
[1]+  Exit 1                  bonnie++ -s 3584 -d /hdfs/ -x5 -u hadoop -q -b -m HDFS_test
> /tmp/ausgabe.txt 2>&1

This looks not so good. :-(

Best Regards,
   Christian


Am Dienstag, 6. Juli 2010 schrieb Christian Baun:
> Hi,
> 
> I installed a small Hadoop-Cluster with Cloudera CDH3.
> 
> 2 Server are running. One is running the NameNode and DataNode. The other server is just
a DataNode.
> A dedicated Client exists too.
> 
> I want to run some performance tests with bonnie++ and iozone.
> bonnie++ works but when I start iozone, it interrups.
> 
> Is this a known issue? 
> 
> $ mount | grep hdfs
> fuse_dfs on /hdfs type fuse.fuse_dfs (rw,nosuid,nodev,allow_other,default_permissions)
> 
> $ iozone -Rac -g 3584M -f /hdfs/testfile | tee -a /tmp/iozone_results.txt &
> [1] 6283
> hadoop@ip-10-243-13-191:/home/ubuntu$ 	Iozone: Performance Test of File I/O
> 	        Version $Revision: 3.308 $
> 		Compiled for 32 bit mode.
> 		Build: linux 
> 
> 	Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins
> 	             Al Slater, Scott Rhine, Mike Wisner, Ken Goss
> 	             Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR,
> 	             Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner,
> 	             Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy,
> 	             Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root.
> 
> 	Run began: Tue Jul  6 16:40:35 2010
> 
> 	Excel chart generation enabled
> 	Auto Mode
> 	Include close in write timing
> 	Using maximum file size of 3670016 kilobytes.
> 	Command line used: iozone -Rac -g 3584M -f /hdfs/testfile
> 	Output is in Kbytes/sec
> 	Time Resolution = 0.000001 seconds.
> 	Processor cache size set to 1024 Kbytes.
> 	Processor cache line size set to 32 bytes.
> 	File stride size set to 17 * record size.
>                                                             random  random    bkwd  
record   stride                                   
>               KB  reclen   write rewrite    read    reread    read   write    read  rewrite
    read   fwrite frewrite   fread  freread
>               64       4write: Invalid argument
> 
> Error writing block 0, fd= 3
> 
> iozone: interrupted
> 
> 
> Best Regards, 
>     Christian 
> 



Mime
View raw message