spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shixiong Zhu (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (SPARK-11627) Spark Streaming backpressure mechanism has no initial input rate limit,receivers receive data at the maximum speed , it might cause OOM exception
Date Wed, 17 Feb 2016 03:45:18 GMT

     [ https://issues.apache.org/jira/browse/SPARK-11627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Shixiong Zhu resolved SPARK-11627.
----------------------------------
       Resolution: Fixed
         Assignee: junhaoMg
    Fix Version/s: 2.0.0

> Spark Streaming backpressure mechanism  has no initial input rate limit,receivers receive
data at the maximum speed , it might cause OOM exception
> --------------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: SPARK-11627
>                 URL: https://issues.apache.org/jira/browse/SPARK-11627
>             Project: Spark
>          Issue Type: Bug
>          Components: Streaming
>    Affects Versions: 1.5.1, 1.6.0
>            Reporter: junhaoMg
>            Assignee: junhaoMg
>             Fix For: 2.0.0
>
>   Original Estimate: 72h
>  Remaining Estimate: 72h
>
> Spark Streaming backpressure mechanism  has no initial input rate limit, receivers receive
data at the maximum speed they can reach in the  first batch, the data received might exhaust
executors memory resources and  cause out of memory  exception.  Eventually the streaming
job would failed, the  backpressure mechanism become invalid.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message