reef-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Brian Cho (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (REEF-771) Locality-aware Task Scheduler
Date Tue, 05 Jan 2016 02:21:39 GMT

    [ https://issues.apache.org/jira/browse/REEF-771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15082235#comment-15082235
] 

Brian Cho commented on REEF-771:
--------------------------------

[~shahankhatch]

The Task scheduler works as follows:
1. If there are no queued tasks (each task runs on an evaluator), the number of evaluators
remains at min (*)
2. If there are queued tasks, additional evaluators will be launched

I think for situation 1, we should just do a best-effort locality among the open evaluators.
For situation 2, we could indeed prefer launching an additional evaluator on a local node
to a queued task. Does this make sense?

(*) I just realized that I haven't made the changes to REEF-701. Thus, the current min is
always one. I'll take care of the PR soon.



> Locality-aware Task Scheduler
> -----------------------------
>
>                 Key: REEF-771
>                 URL: https://issues.apache.org/jira/browse/REEF-771
>             Project: REEF
>          Issue Type: Improvement
>          Components: REEF Examples
>            Reporter: Byung-Gon Chun
>
> The current TaskScheduler example does not consider data locality when tasks are launched.
A nice extension of the example is to consider data locality to schedule tasks when data's
stored in HDFS. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message