aurora-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bill Farner (JIRA)" <>
Subject [jira] [Commented] (AURORA-137) Save host attributes only when a task is being scheduled
Date Sat, 19 Nov 2016 17:07:59 GMT


Bill Farner commented on AURORA-137:

The actual number of write activity for attributes would be useful data to decide whether
{{BatchWorker}} is useful here.  Echoing one of Stephan's points - {{DbAttributeStore.saveHostAttributes()}}
_should_ only be triggering a write when attributes have actually changed, so i'd expect attribute
log entries to rapidly quiesce.

Relevant snippet from {{DbAttributeStore. saveHostAttributes()}}:
Optional<IHostAttributes> existing = getHostAttributes(hostAttributes.getHost());
if (existing.equals(Optional.of(hostAttributes))) {
  return false;

I think my original goal with this ticket was to eliminate the call to {{storage.write()}}
when handling {{resourceOffers()}}, and instead store host attributes when a scheduling match
is found (and we're already in a write transaction).  This would satisfy the current purpose
of host attributes (satisfying diversity constraints) while avoiding holding a highly-contended
lock unnecessarily.

 have noticed contention on storage write lock
[~mnurolahzade] is there any data you can share on this?  Presumably the existing {{scheduler_resource_offers_*}}
no longer give insight, as the method body is asynchronous.  Perhaps another low-effort action
item is to extract the synchronous stage of that method so that the {{@Timed}} stats are meaningful.

> Save host attributes only when a task is being scheduled
> --------------------------------------------------------
>                 Key: AURORA-137
>                 URL:
>             Project: Aurora
>          Issue Type: Story
>          Components: Scheduler
>            Reporter: Bill Farner
>            Priority: Minor
> The scheduler currently aggressively saves host attributes when handling {{resourceOffers}},
however it seems tractable for this to only happen when a task is actually scheduled.  Context:
the scheduler stores host attributes to satisfy scheduling constraints (like host/rack diversity).
 Doing this would allow us to avoid waiting for the storage write lock, and handle {{resourceOffers}}
in a more deterministic time frame.
> One caveat with this approach is that the Offer would need to be plumbed into {{SchedulingFilterImpl}}
in a way so as to ensure that the attributes are available for the offer being inspected.
 In other words, we need to avoid the chicken and egg of trying to read the attributes for
a host when this is the first offer ever received for it.

This message was sent by Atlassian JIRA

View raw message