hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael Stoppelman" <stop...@gmail.com>
Subject Re: Hadoop & EC2
Date Tue, 02 Sep 2008 17:31:36 GMT
Tom White's blog has a nice piece on the different setups you can have for a
hadoop cluster on EC2:
http://www.lexemetech.com/2008/08/elastic-hadoop-clusters-with-amazons.html

With the EBS volumes you can bring up and take down your cluster at will so
you don't need to have 20 machines running all the time. We're still
collecting performance numbers, but it's definitely faster to use EBS or
local storage on EC2 than it is to use S3 (we were seeing 2Mb/s - 10Mb/s).

M

On Tue, Sep 2, 2008 at 8:59 AM, Andrzej Bialecki <ab@getopt.org> wrote:

> tim robertson wrote:
>
>  Incidentally, I have most of the basics of a "MapReduce-Lite" which I
>> aim to port to use the exact Hadoop API since I am *only* working on
>> 10's-100's GB of data and find that it is running really fine on my
>> laptop and I don't need the distributed failover.  My goal for that
>>
>
> If it's going to be API-compatible with regular Hadoop, then I'm sure many
> people will find it useful. E.g. many Nutch users bemoan the complexity of
> distributed Hadoop setup, and they are not satisfied with the "local"
> single-threaded physical-copy execution mode.
>
>
> --
> Best regards,
> Andrzej Bialecki     <><
>  ___. ___ ___ ___ _ _   __________________________________
> [__ || __|__/|__||\/|  Information Retrieval, Semantic Web
> ___|||__||  \|  ||  |  Embedded Unix, System Integration
> http://www.sigram.com  Contact: info at sigram dot com
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message