incubator-mesos-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From " (Commented) (JIRA)" <>
Subject [jira] [Commented] (MESOS-45) Document semantics and state transitions in mesos user-pluggable interfaces
Date Tue, 10 Jan 2012 19:29:40 GMT

] commented on MESOS-45:

bq.  On 2012-01-10 14:26:24, John Sirois wrote:
bq.  > src/java/src/org/apache/mesos/, line 57
bq.  > <>
bq.  >
bq.  >     There appears to be no need fo indexed access - Collection if you want to provide
size, but it seems Iterable is fine here.
bq.  Matei Zaharia wrote:
bq.      Going over the offers in the same order multiple times is useful in complex schedulers
-- for example, we do it in Spark. I suggest leaving it this way, unless you want everyone
to copy their Collection to a list. I can't anticipate a scenario when it would be more efficient
to use an unordered collection here, and in other languages, such as Python, the collection
will be ordered anyway.
bq.  John Sirois wrote:
bq.      The comment wasn't derived from efficiency concerns - a List could still be passed
since its Iterable.  I was suggesting this change as an api maintainer - Iterable gives you
more freedom going forward to change implementation without breaking clients and your clients
still get a sequence of offers they can repeatedly iterate over.  You might add extra javadoc
that says the iterators are guaranteed to be stable wrt each other, but this is implicit for
all Iterables I've encountered in java land.
bq.  Matei Zaharia wrote:
bq.      I realize that the comment isn't from efficiency concerns, but I'm just saying that
I can't imagine a case where the implementation would change in a way that prevents meeting
the List interface. Just to be clear, the code I'm talking about doesn't simply want to iterate
over the list multiple times -- it wants to know the size of the list and keep some data about
each machine offered, and it does use indexes into it. For example, here's a common thing
to do: you have a list of tasks you want to launch and a list of machines offered, and you
want to spread your tasks across the machines in a round-robin manner as long as they have
enough resources. You can do something like this:
bq.      memoryLeft = // int array initialized to the memory on each machine
bq.      cpusLeft = // int array initialized to the CPUs on each machine
bq.      while (tasksToLaunch > 0) {
bq.        for (i = 0 to offers.size) {
bq.          if (memoryLeft[i] >= taskMemory && cpusLeft[i] >= taskCpus) {
bq.            launch a task on machine i
bq.            decrease memoryLeft[i] and cpusLeft[i]
bq.          }
bq.        }
bq.      }
bq.      Basically, by not providing a List, you're losing some usability without gaining
much maintainability. I agree you shouldn't be providing an array or something, but giving
a less convenient collection type for something that is already an ordered collection in other
languages seems overly picky.

SGTM -  I still don't get the need for an index since the slot an offer is in in the List
has no mesos meaning afaict, but I can buy size.  And I agree with your point that List is
likely not too restrictive - I just tend to be _very_ defensive with exposed api types.

- John

This is an automatically generated e-mail. To reply, visit:

On 2012-01-10 06:48:03, Benjamin Hindman wrote:
bq.  -----------------------------------------------------------
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  -----------------------------------------------------------
bq.  (Updated 2012-01-10 06:48:03)
bq.  Review request for mesos, Andy Konwinski, Charles Reiss, Matei Zaharia, John Sirois,
and Vinod Kone.
bq.  Summary
bq.  -------
bq.  Provides initial documentation to the public interfaces (thanks to Vinod Kone for pairing
with me on this).
bq.  This addresses bug MESOS-45.
bq.  Diffs
bq.  -----
bq.    include/mesos/executor.hpp d883f1a 
bq.    include/mesos/mesos.proto 77a9067 
bq.    include/mesos/scheduler.hpp 93dda98 
bq.    src/java/src/org/apache/mesos/ 5ad8cee 
bq.    src/java/src/org/apache/mesos/ be01d21 
bq.    src/java/src/org/apache/mesos/ 399c8b8 
bq.    src/java/src/org/apache/mesos/ 04809a6 
bq.    src/java/src/org/apache/mesos/ 63a06fe 
bq.    src/java/src/org/apache/mesos/ 23a246c 
bq.    src/python/src/ 5fc60e2 
bq.  Diff:
bq.  Testing
bq.  -------
bq.  Thanks,
bq.  Benjamin

> Document semantics and state transitions in mesos user-pluggable interfaces
> ---------------------------------------------------------------------------
>                 Key: MESOS-45
>                 URL:
>             Project: Mesos
>          Issue Type: Improvement
>          Components: c++-api, java-api, python-api
>            Reporter: John Sirois
> In particular, scheduler and executor lifecycle and re-entrancy requirements should be
documented well either centrally, in a HOW-TO/spec or better, additionally or solely in binding
interfaces in the supported languages (cpp, java, python).

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:!default.jspa
For more information on JIRA, see:


View raw message