spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shay Elbaz (JIRA)" <j...@apache.org>
Subject [jira] [Created] (SPARK-26438) Driver waits to spark.sql.broadcastTimeout before throwing OutOfMemoryError - is this by design?
Date Tue, 25 Dec 2018 11:35:00 GMT
Shay Elbaz created SPARK-26438:
----------------------------------

             Summary: Driver waits to spark.sql.broadcastTimeout before throwing OutOfMemoryError
- is this by design?
                 Key: SPARK-26438
                 URL: https://issues.apache.org/jira/browse/SPARK-26438
             Project: Spark
          Issue Type: Question
          Components: Spark Core, SQL
    Affects Versions: 2.3.0
            Reporter: Shay Elbaz


When broadcasting too large DataFrame, the driver does not fail immediately when the broadcast
thread throws OutOfMemoryError. Instead it waits for `spark.sql.broadcastTimeout` to meet.
Is that by design or a bug?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message