mesos-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ben Mahler" <benjamin.mah...@gmail.com>
Subject Re: Review Request: Kill tasks that never properly launch.
Date Mon, 06 May 2013 20:57:28 GMT


> On May 6, 2013, 8:40 p.m., Ben Mahler wrote:
> > Awesome, thanks Brenden! There are currently cases in mesos where tasks are lost
and the updates don't make it to the scheduler, is that what you were seeing?
> > 
> > Are you able to do this with a java.util.Timer instead? We can schedule the kill
operation on each launched task (you'll want to pass in the Driver as well).
> 
> Brenden Matthews wrote:
>     I don't remember exactly what was happening with this one, but it sounds like you
summed it up correctly.  There are a lot of edge cases, many of which are beyond the control
of Mesos, and this helps to catch some of those.

This definitely makes the Hadoop Scheduler more robust! Let me know when you've updated to
use java.util.Timer to asynchronously kill the task. I can provide more pointers if needed
:)


- Ben


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/10931/#review20227
-----------------------------------------------------------


On May 3, 2013, 6:41 p.m., Brenden Matthews wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/10931/
> -----------------------------------------------------------
> 
> (Updated May 3, 2013, 6:41 p.m.)
> 
> 
> Review request for mesos.
> 
> 
> Description
> -------
> 
> From 9c0d29ec0d317bca1df0236722cb671309a59b2d Mon Sep 17 00:00:00 2001
> From: Brenden Matthews <brenden.matthews@airbnb.com>
> Date: Thu, 2 May 2013 16:50:53 -0700
> Subject: [PATCH 7/9] Kill tasks that never properly launch.
> 
> After trying to launch a task tracker, we'll wait up to 5 minutes before
> giving up and killing the task.
> ---
>  .../java/org/apache/hadoop/mapred/MesosScheduler.java  |   16 ++++++++++++++++
>  1 file changed, 16 insertions(+)
> 
> 
> Diffs
> -----
> 
>   hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosScheduler.java afe401f 
> 
> Diff: https://reviews.apache.org/r/10931/diff/
> 
> 
> Testing
> -------
> 
> Used in production at airbnb.
> 
> 
> Thanks,
> 
> Brenden Matthews
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message