hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tom White <...@cloudera.com>
Subject Re: a patch for src/contrib/cloud scripts
Date Mon, 05 Apr 2010 14:04:22 GMT
On Mon, Apr 5, 2010 at 4:56 AM, Andrew Klochkov <diggerk@gmail.com> wrote:
> Hi,
>
> I'm using new src/contrib/cloud scripts created by Tom White recently. Great
> work, now we can easily deploy 0.20.x onto EC2. Thanks, Tom!
>
> I've just fixed several things in the scripts:
>
> 1. A typo in src/py/hadoop/cloud/cli.py: on "push" command the script
> actually invokes proxy creation
> 2. Modified src/py/hadoop/cloud/service.py, to copy AWS_ACCESS_KEY_ID
> and AWS_SECRET_ACCESS_KEY variables into a user data script environment, so
> later it can fill them in hadoop configs.
> 3. Fixed user data script to fill /home/hadoop/.bashrc file additionally to
> root's .bashrc. Seems like with new scripts, it's better to issue jobs from
> "hadoop" user. Some HDFS permissions don't let to run jobs from "root" user.
>
> Should I open a JIRA issue on it? Or should I post the patch here, in the
> mailing list?

Please open JIRAs and attach patches there.

>
> And also I'm thinking if it's a good idea to create a "hardware" provider to
> use cloud scripts for hardware (non-cloud) cluster deployment. What do you
> think? Is there a better way to automate hardware  cluster deployment?

That would be a great addition.

Cheers,
Tom

>
> --
> Andrew Klochkov
>

Mime
View raw message