spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Patrick Wendell (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (SPARK-4525) MesosSchedulerBackend.resourceOffers cannot decline unused offers from acceptedOffers
Date Tue, 25 Nov 2014 03:26:12 GMT

     [ https://issues.apache.org/jira/browse/SPARK-4525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Patrick Wendell updated SPARK-4525:
-----------------------------------
    Fix Version/s: 1.2.0

> MesosSchedulerBackend.resourceOffers cannot decline unused offers from acceptedOffers
> -------------------------------------------------------------------------------------
>
>                 Key: SPARK-4525
>                 URL: https://issues.apache.org/jira/browse/SPARK-4525
>             Project: Spark
>          Issue Type: Bug
>          Components: Mesos
>    Affects Versions: 1.2.0, 1.3.0
>            Reporter: Jongyoul Lee
>            Assignee: Jongyoul Lee
>            Priority: Blocker
>             Fix For: 1.2.0
>
>
> After resourceOffers function is refactored - SPARK-2269 -, that function doesn't decline
unused offers from accepted offers. That's because when driver.launchTasks is called, if that's
tasks is empty, driver.launchTask calls the declineOffer(offer.id). 
> {quote}
> Invoking this function with an empty collection of tasks declines offers in their entirety
(see SchedulerDriver.declineOffer(OfferID, Filters)).
> - http://mesos.apache.org/api/latest/java/org/apache/mesos/MesosSchedulerDriver.html#launchTasks(OfferID,%20java.util.Collection,%20Filters)
> {quote}
> In branch-1.1, resourcesOffers calls a launchTask function for all offered offers, so
driver declines unused resources, however, in current master, at first offers are divided
accepted and declined offers by their resources, and delinedOffers are declined explicitly,
and offers with task from acceptedOffers are launched by driver.launchTasks, but, offers without
from acceptedOfers are not launched with empty task or declined explicitly. Thus, mesos master
judges thats offers used by TaskScheduler and there are no resources remaing.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message