spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Apache Spark (JIRA)" <j...@apache.org>
Subject [jira] [Assigned] (SPARK-7988) Mechanism to control receiver scheduling
Date Wed, 03 Jun 2015 07:46:50 GMT

     [ https://issues.apache.org/jira/browse/SPARK-7988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Apache Spark reassigned SPARK-7988:
-----------------------------------

    Assignee:     (was: Apache Spark)

> Mechanism to control receiver scheduling 
> -----------------------------------------
>
>                 Key: SPARK-7988
>                 URL: https://issues.apache.org/jira/browse/SPARK-7988
>             Project: Spark
>          Issue Type: New Feature
>          Components: Streaming
>            Reporter: Nishkam Ravi
>            Priority: Critical
>
> Streaming receivers are currently scheduled by the underlying Spark scheduler in the
same way as other tasks. Will help performance if we added a knob to control receiver scheduling.
In most cases, even distribution of receivers among workers should yield best results. In
some cases, dense scheduling (e.g, all receivers on one node) may perform better. Maybe support
for a new configuration parameter spark.receiver.scheduling.mode (or some such) would help.
With 'default' as the current scheduling scheme and 'round-robin' for even distribution among
workers (and potentially a third mode for 'dense' scheduling).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message