hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tom White (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-6464) Write a Rackspace cloud provider
Date Wed, 06 Jan 2010 18:05:54 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-6464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Tom White updated HADOOP-6464:

    Attachment: HADOOP-6464.patch

> Can you please describe the test plan you used to actually test end-to-end coverage of

I tested this by manually spinning up a cluster, running a job, checking the results and terminating
the cluster. The commands to do this are in the README.

I've now modified the integration test script to run on Rackspace, which I successfully ran
with the following (note that the image ID is private, since Rackspace does not support shared

HADOOP_CLOUD_PROVIDER=rackspace IMAGE_ID=200181 INSTANCE_TYPE=4 HADOOP_HOME=~/dev/hadoop-0.20.1/

> regarding the patch:
> see comments re. TERREMARK_KEY in HADOOP-6465 for RACKSPACE_KEY, etc.

Agree that this would be a useful way to specify keys. I've created HADOOP-6480 for this.

> re. boot script: I noticed a lot of very similar code in the terremark boot script. Can
some of this be factored out into a common utility script that gets sourced in? This seems
to be a looming maintenence problem.

This could become a problem, but I'd like to address it in a separate issue.

> RackspaceCluster.wait_for_instances() looks like a clone of TerremarkCluster.wait_for_instances().
Can this be pushed up into cluster?
> Ditto with _all_started(), _ip_list_to_string(), terminate()?

These are the same methods. I'd like to commit this one first, then I'll re-work HADOOP-6465
so the methods are pulled up into Cluster.

> Why is rackspace remote-setup.sh strictly-ubuntu-based? The boot script looks like it
also handles centos. 

I've only tested on Ubuntu so far, but it would be straightforward to run on Centos too. I'll
create another JIRA for this.

Thanks for the review!

> Write a Rackspace cloud provider
> --------------------------------
>                 Key: HADOOP-6464
>                 URL: https://issues.apache.org/jira/browse/HADOOP-6464
>             Project: Hadoop Common
>          Issue Type: New Feature
>          Components: contrib/cloud
>            Reporter: Tom White
>            Assignee: Tom White
>         Attachments: HADOOP-6464.patch, HADOOP-6464.patch
> The scripts in contrib/cloud currently only support running on EC2. This issue is to
add support for running Hadoop clusters on Rackspace Cloud Servers.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message