aurora-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bill Farner (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (AURORA-116) Improve efficiency of saving host attributes (or avoid saving host attributes)
Date Sat, 25 Jan 2014 06:04:37 GMT

    [ https://issues.apache.org/jira/browse/AURORA-116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13881698#comment-13881698
] 

Bill Farner commented on AURORA-116:
------------------------------------

Incremental improvement: https://reviews.apache.org/r/17351/

Will observe that in the wild and re-assess whether to move on to other approaches.

> Improve efficiency of saving host attributes (or avoid saving host attributes)
> ------------------------------------------------------------------------------
>
>                 Key: AURORA-116
>                 URL: https://issues.apache.org/jira/browse/AURORA-116
>             Project: Aurora
>          Issue Type: Task
>          Components: Scheduler
>            Reporter: Bill Farner
>            Assignee: Bill Farner
>            Priority: Critical
>
> The scheduler performs multiple write operations for every resource offer, to save slave
attributes:
> {noformat}
>   public void resourceOffers(SchedulerDriver driver, List<Offer> offers) {
>     Preconditions.checkState(registered, "Must be registered before receiving offers.");
>     for (final Offer offer : offers) {
>       log(Level.FINE, "Received offer: %s", offer);
>       resourceOffers.incrementAndGet();
>       storage.write(new MutateWork.NoResult.Quiet() {
>         @Override protected void execute(MutableStoreProvider storeProvider) {
>           storeProvider.getAttributeStore().saveHostAttributes(Conversions.getAttributes(offer));
>         }
>       });
> {noformat}
> This can unnecessarily block the singly-threaded message dispatch in the scheduler driver.
 An incremental improvement would be to aggregate all slave info and save it in one write
operation.  Better yet would be to perform writes asynchronously (taking care to not break
task scheduling, since attributes are expected to be present).  Even better yet, it would
be great to determine if we can avoid storing host attributes.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

Mime
View raw message