hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aaron Kimball ...@cs.washington.edu>
Subject Re: Amazon S3 questions
Date Sun, 02 Mar 2008 01:09:09 GMT
Tests of S3--EC2 transfer speed using curl have shown that gigabit 
ethernet speeds can be reached between nodes but only when multiple 
transfers are overlapped. See: 
http://info.rightscale.com/2007/11/29/network-performance-in-ec2-and-s3

- Aaron

Toby DiPasquale wrote:
> On Sat, Mar 1, 2008 at 4:19 PM, Bradford Stephens
> <bradfordstephens@gmail.com> wrote:
>> What sort of performance hit is there for using S3 vs.  a local cluster?
> 
> In my experience on EC2 using Hadoop, S3-backed HDFS is 25-33% slower
> than disk-backed HDFS. That was using 0.14.x, however, so it might be
> better now...
> 

Mime
View raw message