hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alejandro Abdelnur" <tuc...@gmail.com>
Subject Re: Do we need for more flexibility in the way tasks are run ?
Date Sat, 21 Jun 2008 20:43:51 GMT
Not exactly the same but ... regarding the security side of things, we
have a similar situation where the cluster is shared by different

As our jobs are Java M/R (no streaming) we force the task JVM to run
with the security manager using an admin provided java policy file to
control what resources jobs can access, in the slaves and in the


On Sat, Jun 21, 2008 at 2:31 AM, Brice Arnould <brice.arnould@vleu.net> wrote:
> Hi !
> I would like your advice about an proposing I was going to create on
> Jira when I realised that I might be completely wrong ^.^
> So, the context is that, with [HADOOP-3421] speaking about sharing a
> cluster among more than one organization (so potentially with
> non-cooperative users), and posts here speaking about virtualization and
> the ability of re-using the TaskTracker's VM to run new tasks, I told
> myself that it could be useful to choose the way TaskRunners run their
> children.
> More specifically, it could be useful to at least provide a way to
> imprison a Task in its working directory, or in a virtual machine.
> The usefulness of sharing the TaskTracker VM seems more arguable, but it
> might still be interesting.
> So I started writing a prototype, by extracting and generalizing parts
> of the TaskRuner, like I tried before with the JobTracker. But then I
> realized that, if we don't need more than imprisoning, it could be much
> easier to just provide a class inheriting of ShellCommandExecutor that
> would run its commands in a sandbox.
> So I am asking for your opinions if my first idea was a good one, or if
> I should re-work the second one... or drop both :-P
> Thanks !
> Brice
> PS: I already posted that, but I think that it get lost in a lot of mail
> sent by Jira :-P Sorry for the noise.

View raw message