aurora-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bill Farner" <wfar...@apache.org>
Subject Re: Review Request 42126: New interface to allocate resources of multiple roles from offer.
Date Tue, 12 Jan 2016 16:00:03 GMT


> On Jan. 11, 2016, 3:42 p.m., Zhitao Li wrote:
> > src/main/java/org/apache/aurora/scheduler/OfferAllocation.java, line 188
> > <https://reviews.apache.org/r/42126/diff/1/?file=1191434#file1191434line188>
> >
> >     Hmm, I think I'd like to keep this for two reasons:
> >     
> >     1. Even though only one callsite uses value other than `false`, the `false`
value still dictates a `Predicate` to use, so we'll have more code duplicate if we move this
out;
> >     2. Right now aurora only supports `cpus` as revocable resource. I actually wanted
to discuss this separately as current prediction in our company indicates that we might be
more memory bounded. In such a case, we could be using memory as revocable resources, too.

> Even though only one callsite uses value other than false, the false value still dictates
a Predicate to use, so we'll have more code duplicate if we move this out

Not sure i agree with the code duplication, but i will follow up on the next patch draft.

> ...we might be more memory bounded. In such a case, we could be using memory as revocable
resources, too.

I'm not sure if that is possible, but it definitely is not recommended.

http://mesos.apache.org/documentation/latest/oversubscription/
> It is recommended only to oversubscribe compressible resources such as cpu shares, bandwidth,
etc.


> On Jan. 11, 2016, 3:42 p.m., Zhitao Li wrote:
> > src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java, line 120
> > <https://reviews.apache.org/r/42126/diff/1/?file=1191437#file1191437line120>
> >
> >     I don't plan on any further usage of the interface other than here in this series
of patches.
> >     
> >     The only possible customization that could happen is the reverse ordering of
resource preference (`'*'` vs `reserved role`) but I'd like to withhold it until someone request
such feature.
> >     
> >     I think I'll keep the interface definition, change the `AcceptedOfferImpl` to
package private, and use a factory method anyway. We can revisit this later. How does this
sound?

I will follow up on the latest draft.


- Bill


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/42126/#review113862
-----------------------------------------------------------


On Jan. 11, 2016, 6:29 p.m., Zhitao Li wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/42126/
> -----------------------------------------------------------
> 
> (Updated Jan. 11, 2016, 6:29 p.m.)
> 
> 
> Review request for Aurora, Maxim Khutornenko, Dmitriy Shirchenko, and Bill Farner.
> 
> 
> Bugs: AURORA-1109
>     https://issues.apache.org/jira/browse/AURORA-1109
> 
> 
> Repository: aurora
> 
> 
> Description
> -------
> 
> This review is a prototype for introducing multiple role support in Aurora.
> This creates a new interface OfferAllocation (with implementation), which allcoates resources
to resources field in TaskInfo and ExecutorInfo from an offer.
> 
> Current implementation prefers reserved resources over shared resources ('*' role) if
both are present. This can be  customized with a differnet comparator from PREFER_RESERVED
in the future when needed.
> 
> Several caveats:
> 1. This performs the allocate after scheduling decision in TaskAssigner.maybeAssign is
done, which leaves possibility of inconsistency and late failure;
> 2. Old code used to construct resources has not been cleaned up yet;
> 3. OfferAllocationImpl's constructor allows to throw, which is a bit awkward.
> 
> 
> Diffs
> -----
> 
>   src/main/java/org/apache/aurora/scheduler/AcceptedOffer.java PRE-CREATION 
>   src/main/java/org/apache/aurora/scheduler/ResourceSlot.java 7c3d681c216b78eeecebbe950186e5a79c6fe982

>   src/main/java/org/apache/aurora/scheduler/Resources.java db422a959ee7b982c2a44323de41ad75d1a40754

>   src/main/java/org/apache/aurora/scheduler/mesos/CommandLineDriverSettingsModule.java
2255dd407cd1810c7df5baf17cfa85f79bfffeb8 
>   src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 8fdadda67478bb3110aa442b7d78493cf9c3edb4

>   src/main/java/org/apache/aurora/scheduler/state/TaskAssigner.java 7e8e456e288986eb0ce92a123b294e1e25d8ed18

>   src/test/java/org/apache/aurora/scheduler/AcceptedOfferImplTest.java PRE-CREATION 
>   src/test/java/org/apache/aurora/scheduler/ResourceSlotTest.java e4ae943303823ac4bfbe999ed22f5999484462d8

>   src/test/java/org/apache/aurora/scheduler/mesos/CommandLineDriverSettingsModuleTest.java
33149ab415292eff04f38b61f2b1d1eac79f347a 
>   src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java a5793bffabf4e5d6195b1b99f2363d241c0cecf9

>   src/test/java/org/apache/aurora/scheduler/state/TaskAssignerImplTest.java 3cbe9acd75def14ae2e0986914ba621fb164b3e4

> 
> Diff: https://reviews.apache.org/r/42126/diff/
> 
> 
> Testing
> -------
> 
> 1. Unit tested with old and new tests;
> 2. vagrant integration tests: I manually separate out the vagrant box's cpu and memory
between 'aurora-test' role and '*' and verified that jobs can still be launched (I can post
the vagrant change in another follow upon request).
> 
> 
> Thanks,
> 
> Zhitao Li
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message