aurora-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zhitao Li" <zhitaoli...@gmail.com>
Subject Re: Review Request 42126: New class to allocate resources of multiple roles from offer.
Date Wed, 13 Jan 2016 17:09:39 GMT


> On Jan. 13, 2016, 2:57 a.m., Bill Farner wrote:
> > src/main/java/org/apache/aurora/scheduler/AcceptedOffer.java, line 129
> > <https://reviews.apache.org/r/42126/diff/3/?file=1195147#file1195147line129>
> >
> >     This is a good general-purpose implementation of this behavior, but in practice
it seems like overkill.  Can you instead match the behavior currently on master and synthesize
the `Resource` with `makeMesosRangeResource(PORTS, selectedPorts)`?

I think we still need this function, because ports can be reserved to a particular role, similar
to cpus/mem/disk.

In practice, it seems possible to me that cluster operator decided to partition available
port range among differnet frameworks. Certain Mesos framework (Cassandra) even requires fixed
ports in resource offers to launch their workload, because their workload only works well
with these fixed ports.

Given such possibility, it seems we still need to have correct role for the ports used when
sending them back to Mesos, mostly for handling all possible configuration purpose.

Please let me know if you think otherwise.


- Zhitao


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/42126/#review114128
-----------------------------------------------------------


On Jan. 13, 2016, 2:08 a.m., Zhitao Li wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/42126/
> -----------------------------------------------------------
> 
> (Updated Jan. 13, 2016, 2:08 a.m.)
> 
> 
> Review request for Aurora, Maxim Khutornenko, Dmitriy Shirchenko, and Bill Farner.
> 
> 
> Bugs: AURORA-1109
>     https://issues.apache.org/jira/browse/AURORA-1109
> 
> 
> Repository: aurora
> 
> 
> Description
> -------
> 
> This review is a prototype for introducing multiple role support in Aurora.
> This creates a new class OfferAllocation, which allcoates resources to resources field
in TaskInfo and ExecutorInfo from an offer.
> 
> Current implementation prefers reserved resources over shared resources ('*' role) if
both are present
> 
> Several caveats:
> 1. This performs the allocate after scheduling decision in TaskAssigner.maybeAssign is
done, which leaves possibility of inconsistency and late failure.
> 
> 
> Diffs
> -----
> 
>   src/main/java/org/apache/aurora/scheduler/AcceptedOffer.java PRE-CREATION 
>   src/main/java/org/apache/aurora/scheduler/ResourceSlot.java 7c3d681c216b78eeecebbe950186e5a79c6fe982

>   src/main/java/org/apache/aurora/scheduler/Resources.java db422a959ee7b982c2a44323de41ad75d1a40754

>   src/main/java/org/apache/aurora/scheduler/mesos/CommandLineDriverSettingsModule.java
2255dd407cd1810c7df5baf17cfa85f79bfffeb8 
>   src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 8fdadda67478bb3110aa442b7d78493cf9c3edb4

>   src/main/java/org/apache/aurora/scheduler/state/TaskAssigner.java 7e8e456e288986eb0ce92a123b294e1e25d8ed18

>   src/test/java/org/apache/aurora/scheduler/AcceptedOfferTest.java PRE-CREATION 
>   src/test/java/org/apache/aurora/scheduler/ResourceSlotTest.java e4ae943303823ac4bfbe999ed22f5999484462d8

>   src/test/java/org/apache/aurora/scheduler/mesos/CommandLineDriverSettingsModuleTest.java
33149ab415292eff04f38b61f2b1d1eac79f347a 
>   src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java a5793bffabf4e5d6195b1b99f2363d241c0cecf9

>   src/test/java/org/apache/aurora/scheduler/state/TaskAssignerImplTest.java 3cbe9acd75def14ae2e0986914ba621fb164b3e4

> 
> Diff: https://reviews.apache.org/r/42126/diff/
> 
> 
> Testing
> -------
> 
> 1. Unit tested with old and new tests;
> 2. vagrant integration tests: I manually separate out the vagrant box's cpu and memory
between 'aurora-test' role and '*' and verified that jobs can still be launched (I can post
the vagrant change in another follow upon request).
> 
> 
> Thanks,
> 
> Zhitao Li
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message