hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From tim robertson <timrobertson...@gmail.com>
Subject Re: Generating many small PNGs to Amazon S3 with MapReduce
Date Thu, 16 Apr 2009 12:38:18 GMT
> However, do the math on the costs for S3. We were doing something similar,
> and found that we were spending a fortune on our put requests at $0.01 per
> 1000, and next to nothing on storage. I've since moved to a more complicated
> model where I pack many small items in each object and store an index in
> simpledb. You'll need to partition your SimpleDBs if you do this.

Thanks a lot for Kevin for this - I stupidly overlooked the S3 put
cost thinking EC2->S3 transfer was free, without realising there is
still a PUT cost...

I will reconsider and look at copying your approach and compare it
with a few rendering EC2 instances running off mysql or so.

Thanks again.


View raw message