hadoop-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tim Hawkins <tim.hawk...@bejant.com>
Subject Security Groups in Hadoop-ec2
Date Fri, 16 Jan 2009 15:56:39 GMT
I have been playing with hadoop-ec2 (src/contrib/ec2) and have found a  
minor problem

When a cluster is launched, it creates two security groups and cross  
links them, ie for cluster-name xxxx it creates EC2 security groups  
xxxx and xxxx-master , the cross-linking is i believe to allow all the  
masters and slaves to be able to communicate with each other.

Unfortunately this means that the security groups are now mutually  
dependant on each other, and the amazon API will no longer allow them  
to be deleted, none of the GUI tools (rightscale, elasticFox, and  
consol.aws) or the command-line tools seem to be able to remove the  
security groups either, presumably because they too are dependant on  
the API.

I believe the solution would be to create 3 security groups, not two,  
xxxx, xxxx-slave and xxxx-master , and only inherit permissions from  
xxxx into the other two, which would achieve the same result but be  
more "friendly". It would also potentially offer the ability for  
multiple clusters to share base security descriptors with other  
subsystems without having to open publicly accessible holes, by  
allowing the name of xxxx to be set independent of the cluster-name,  
allowing it to be shared as a base.

I can make this change to the scripts and test it on our set-up, but  
am not sure how to contribute the changes back to ensure that this  
problem does not effect others.

View raw message