hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jonathan Aquilina <jaquil...@eagleeyet.net>
Subject Re: AWS Setting for setting up Hadoop cluster
Date Thu, 05 Mar 2015 23:11:52 GMT

Advantage of EMR is that you dont have to stay screwing around with
installing hadoop it does all that for you so you are ready to go 

Jonathan Aquilina
Founder Eagle Eye T

On 2015-03-05 23:51, Krish Donald wrote: 

> Because I am new to AWS, I would like to explore the free service first and then later
I can use EMR. 
> Which one is fast in EC2 and free too? 
> Thanks 
> On Thu, Mar 5, 2015 at 2:47 PM, Jonathan Aquilina <jaquilina@eagleeyet.net> wrote:
> I have experience with my full time job using EMR damn thing is quick and cheap. The
interesting part is wrapping your head around the concepts. If you need things quickly and
fast EMR is the way to go. It spawns up a number of ec2 instances 
> by default you have 1 master and 2 core nodes. The three of them are m3.large nodes which
run you 7 cents per hour. to run one years with of data which is about 1.1 billion records
from the database it took 50 min from cluster spawn up to completion and shutting down of
the cluster. 
> ---
> Regards,
> Jonathan Aquilina
> Founder Eagle Eye T
> On 2015-03-05 23:41, Dieter De Witte wrote: 
> You can install Hadoop on Amazon EC2 instances and use the free tier for new members
but you can also use Amazon EMR which is not free but is up and running in a couple of seconds...

> 2015-03-05 23:28 GMT+01:00 Krish Donald <gotomypc27@gmail.com>:
> Hi, 
> I am tired of setting Hadoop cluster using my laptop which has 8GB RAM. 
> I tried 2gb for namenode and 1-1 gb for 3 datanoded so total 5gb I was using . 
> And I was using very basic Hadoop services only. 
> But it is so slow that I am not able to do anything on that. 
> Hence I would like to try the AWS service now. 
> Can anybody please help me, which configuration I should use it without paying at all?

> What are the tips you have for AWS ? 
> Thanks 
> Krish
View raw message