hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jonathan Aquilina <jaquil...@eagleeyet.net>
Subject Re: AWS Setting for setting up Hadoop cluster
Date Thu, 05 Mar 2015 22:47:26 GMT

I have experience with my full time job using EMR damn thing is quick
and cheap. The interesting part is wrapping your head around the
concepts. If you need things quickly and fast EMR is the way to go. It
spawns up a number of ec2 instances 

by default you have 1 master and 2 core nodes. The three of them are
m3.large nodes which run you 7 cents per hour. to run one years with of
data which is about 1.1 billion records from the database it took 50 min
from cluster spawn up to completion and shutting down of the cluster. 

Jonathan Aquilina
Founder Eagle Eye T

On 2015-03-05 23:41, Dieter De Witte wrote: 

> You can install Hadoop on Amazon EC2 instances and use the free tier for new members
but you can also use Amazon EMR which is not free but is up and running in a couple of seconds...

> 2015-03-05 23:28 GMT+01:00 Krish Donald <gotomypc27@gmail.com>:
>> Hi, 
>> I am tired of setting Hadoop cluster using my laptop which has 8GB RAM. 
>> I tried 2gb for namenode and 1-1 gb for 3 datanoded so total 5gb I was using . 
>> And I was using very basic Hadoop services only. 
>> But it is so slow that I am not able to do anything on that. 
>> Hence I would like to try the AWS service now. 
>> Can anybody please help me, which configuration I should use it without paying at
>> What are the tips you have for AWS ? 
>> Thanks 
>> Krish
View raw message