hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From WangRamon <ramon_w...@hotmail.com>
Subject RE: How does AWS know how many map/reduce slot should be configured to each EC2 instance?
Date Fri, 19 Jul 2013 06:55:59 GMT
Hi Tianyi
Thanks for the reply, that's really help. So i have two further questions:
1.  You said i can customize the number of the slots on AWS, how to do it? i know i can do
it in the mapred-site.xml if i created the cluster without AWS.2.  You mentioned about the
EMR node, will the hadoop-ec2 launch-cluster command start EMR node or common EC2 instance?
Thanks a lot.
CheersRamon

Date: Fri, 19 Jul 2013 16:37:21 +1000
Subject: Re: How does AWS know how many map/reduce slot should be configured to each EC2 instance?
From: tianyi.zhu@facilitatedigital.com
To: user@hadoop.apache.org

1. Yes, it's depends on instance type. Generally, number of map slots + number of reduce slots
= number of ECU, number of map slots / number of reduce slots >= 3. You can customize these
numbers.
2. Yes, Number of nodes * Running hours * Price per EMR node per hour (EMR node is a little
bit more expensive than EC2 node)3. Yes, you can find them in admin console.


On 19 July 2013 16:23, WangRamon <ramon_wang@hotmail.com> wrote:




Hi All
We have a plan to move to Amazon AWS cloud, by doing some research i find that i can start
the map/reduce cluster in AWS with the following command:% bin/hadoop-ec2 launch-cluster test-cluster
2

The command allows me to start a cluster with required nodes(no more than 20, correct me if
i were wrong), so here comes to my questions:
1. How does AWS know how many map/reduce slot should be configured to each EC2 instance? Is
it depends on the EC2 instance type (m1.large, m1.xlarge...)?
2. How it is charged? Nodes number * price per node per hour ?3. Is each node like a single
EC2 instance in my admin console? 

Thanks in advance!
CheersRamon
 		 	   		  

 		 	   		  
Mime
View raw message