spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Apache Spark (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SPARK-11627) Spark Streaming backpressure mechanism has no initial input rate limit,receivers receive data at the maximum speed , it might cause OOM exception
Date Tue, 10 Nov 2015 09:57:11 GMT

    [ https://issues.apache.org/jira/browse/SPARK-11627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14998329#comment-14998329
] 

Apache Spark commented on SPARK-11627:
--------------------------------------

User 'junhaoMg' has created a pull request for this issue:
https://github.com/apache/spark/pull/9593

> Spark Streaming backpressure mechanism  has no initial input rate limit,receivers receive
data at the maximum speed , it might cause OOM exception
> --------------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: SPARK-11627
>                 URL: https://issues.apache.org/jira/browse/SPARK-11627
>             Project: Spark
>          Issue Type: Bug
>          Components: Streaming
>    Affects Versions: 1.5.1
>            Reporter: junhaoMg
>   Original Estimate: 72h
>  Remaining Estimate: 72h
>
> Spark Streaming backpressure mechanism  has no initial input rate limit, receivers receive
data at the maximum speed they can reach in the  first batch, the data received might exhaust
executors memory resources and  cause out of memory  exception.  Eventually the streaming
job would failed, the  backpressure mechanism become invalid.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message