mesos-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ben Mahler" <benjamin.mah...@gmail.com>
Subject Re: Review Request: Fix TaskTracker pending tasks calculation.
Date Tue, 04 Jun 2013 01:21:40 GMT


> On June 3, 2013, 5:36 p.m., Ben Mahler wrote:
> > What differences were you seeing between job.pendingMaps() and this new technique?
> > 
> > Looking at JobInProgress.pendingMaps():
> > 
> >   public synchronized int pendingMaps() {
> >     return numMapTasks - runningMapTasks - failedMapTIPs - 
> >     finishedMapTasks + speculativeMapTasks;
> >   }
> > 
> > vs. jobdetails_jsp.printTaskSummary() {
> >    int totalTasks = tasks.length;
> >     int runningTasks = 0;
> >     int finishedTasks = 0;
> >     int killedTasks = 0;
> >     int failedTaskAttempts = 0;
> >     int killedTaskAttempts = 0;
> >     for(int i=0; i < totalTasks; ++i) {
> >       TaskInProgress task = tasks[i];
> >       if (task.isComplete()) {
> >         finishedTasks += 1;
> >       } else if (task.isRunning()) {
> >         runningTasks += 1;
> >       } else if (task.wasKilled()) {
> >         killedTasks += 1;
> >       }
> >       failedTaskAttempts += task.numTaskFailures();
> >       killedTaskAttempts += task.numKilledTasks();
> >     }
> >     int pendingTasks = totalTasks - runningTasks - killedTasks - finishedTasks;

> >     ...
> > }
> > 
> > It seems like the difference here might be between failed_ vs. killed_ and/or the
fact that the latter case uses speculativeMapTasks?
> 
> Brenden Matthews wrote:
>     Yes, possibly.  To be honest I don't remember, it's been months since I fixed this.
 All I remember specifically is that it was wrong.

Ah, yes that's what I'm getting at, wrong in what way?


- Ben


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/11116/#review21326
-----------------------------------------------------------


On June 3, 2013, 6:02 p.m., Brenden Matthews wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/11116/
> -----------------------------------------------------------
> 
> (Updated June 3, 2013, 6:02 p.m.)
> 
> 
> Review request for mesos.
> 
> 
> Description
> -------
> 
> Fix TaskTracker pending tasks calculation.
> 
> Review: https://reviews.apache.org/r/11116
> 
> 
> Diffs
> -----
> 
>   hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosScheduler.java afe401f5265e3d9494af7eace42eec45943184a3

> 
> Diff: https://reviews.apache.org/r/11116/diff/
> 
> 
> Testing
> -------
> 
> Used in production at airbnb.
> 
> 
> Thanks,
> 
> Brenden Matthews
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message