aurora-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bill Farner (JIRA)" <j...@apache.org>
Subject [jira] [Assigned] (AURORA-1090) Optimize or remove shard uniqueness check from StorageBackfill
Date Tue, 03 Feb 2015 00:41:34 GMT

     [ https://issues.apache.org/jira/browse/AURORA-1090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Bill Farner reassigned AURORA-1090:
-----------------------------------

    Assignee: Bill Farner  (was: David McLaughlin)

> Optimize or remove shard uniqueness check from StorageBackfill
> --------------------------------------------------------------
>
>                 Key: AURORA-1090
>                 URL: https://issues.apache.org/jira/browse/AURORA-1090
>             Project: Aurora
>          Issue Type: Task
>          Components: Scheduler, Technical Debt
>            Reporter: Bill Farner
>            Assignee: Bill Farner
>            Priority: Critical
>
> We have noticed that during scheduler startup, the operation, there can be a significant
amount of time spent between the following log lines:
> {noformat}
> Performing shard uniqueness sanity check.
> storage state machine transition PREPARED -> READY
> {noformat}
> Looking at what happens in the scheduler between those points, the expensive operation
seems to be {{guaranteeShardUniqueness}}.
> This operation aims to validate the integrity of the storage, but its value is dubious.
 There are many other things that could be done to validate integrity, but they should probably
not be done every time the scheduler loads its database.
> If the operation is kept, it can be dramatically optimized.  It currently performs an
O(n^2) scan of tasks, and this could trivially be reduced to O\(n\).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message