hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Brian Bockelman <bbock...@cse.unl.edu>
Subject Re: iozone is not working with HDFS over libfuse
Date Wed, 07 Jul 2010 21:31:43 GMT

On Jul 7, 2010, at 2:56 AM, Christian Baun wrote:

> Hi Brian,
> 
> I wanted to test HDFS against several distributed filesystems.
> 
> Do you know any popular performance benchmarks that run with HDFS? 
> 

I can't think of anything off the top of my head.  Any ideas out there on the list?

The issue is that HDFS is not very much like a distributed file system.  Maybe a good way
is to do one of the MapReduce-based Test IO benchmarks on top of HDFS versus Lustre versus
GPFS, etc?

Brian

> Best Regards,
>   Christian 
> 
> 
> Am Dienstag, 6. Juli 2010 schrieb Allen Wittenauer:
>> 
>> On Jul 6, 2010, at 11:56 AM, Brian Bockelman wrote:
>>> HDFS does not support the full set of POSIX semantics.  I suspect bonnie++ is
exercising things which HDFS doesn't support (such as writes to a file which is already closed
or seeks during writes).
>> 
>> 
>> 	In fact, I think the last time this came up a year or two ago, the stack trace showed
that iozone was blowing up on a call to truncate(), which is definitely not supported.
> 


Mime
View raw message