hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Filipe David Manana <fdman...@apache.org>
Subject Re: Using an umbilical interface like Hadoop MapReduce
Date Fri, 03 Dec 2010 11:04:51 GMT
On Fri, Dec 3, 2010 at 12:19 AM, Edward J. Yoon <edwardyoon@apache.org> wrote:
> Oh, Is HAMA-331 patch OK for you? If there's some wrong with your
> plan, Pls let me know.

It's fine.



>
> On Fri, Dec 3, 2010 at 7:42 AM, Edward J. Yoon <edwardyoon@apache.org> wrote:
>> +1 !!
>>
>> On Fri, Dec 3, 2010 at 6:40 AM, Filipe David Manana <fdmanana@apache.org> wrote:
>>> Hi all,
>>>
>>> While thinking about how to implement HAMA-298 (killing a job):
>>>
>>> - Each task of a user's submitted job is run in a thread
>>> (BSPTaskRunner) spawned by a GroomServer. Stopping a thread is
>>> something not trivial, and it requires that the thread cooperates.
>>> Typically one sets a volatile boolean to true or false, the thread
>>> periodically checks it and then *should* stop if it's value changed to
>>> true or false. This requires implementers of the class BSP to check
>>> for this boolean and possibly adding the same sort of behaviour to
>>> BSPPeer.
>>>
>>> - If a task is simply blocked, for whatever reason, it will never stops.
>>>
>>> - If we assume users' job code will do this check, we're being too optimistic.
>>>
>>> Hadoop spawns a new JVM for each task, and the child JVM communicates
>>> with the parent (TaskTracker) though the TaskUmbilicalProtocol. This
>>> makes it easier to stop a task (simply kill the child process),
>>> doesn't require users' job code to periodically check some special
>>> boolean, and equally important, allows isolation of errors.
>>>
>>> I think we should implement the same idea, not only helps implementing
>>> a better job kill solution, but it will also be a big +1 for when we
>>> support multiple running tasks per GroomServer.
>>>
>>> If everyone agrees with this, I'll start working on it (it will
>>> possibly imply adding much new code).
>>>
>>> regards,
>>>
>>> --
>>> Filipe David Manana,
>>> fdmanana@gmail.com, fdmanana@apache.org
>>>
>>> "Reasonable men adapt themselves to the world.
>>>  Unreasonable men adapt the world to themselves.
>>>  That's why all progress depends on unreasonable men."
>>>
>>
>>
>>
>> --
>> Best Regards, Edward J. Yoon
>> edwardyoon@apache.org
>> http://blog.udanax.org
>>
>
>
>
> --
> Best Regards, Edward J. Yoon
> edwardyoon@apache.org
> http://blog.udanax.org
>



-- 
Filipe David Manana,
fdmanana@gmail.com, fdmanana@apache.org

"Reasonable men adapt themselves to the world.
 Unreasonable men adapt the world to themselves.
 That's why all progress depends on unreasonable men."

Mime
View raw message