spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Juliusz Sompolski (JIRA)" <j...@apache.org>
Subject [jira] [Created] (SPARK-23366) Improve hot reading path in ReadAheadInputStream
Date Fri, 09 Feb 2018 03:48:00 GMT
Juliusz Sompolski created SPARK-23366:
-----------------------------------------

             Summary: Improve hot reading path in ReadAheadInputStream
                 Key: SPARK-23366
                 URL: https://issues.apache.org/jira/browse/SPARK-23366
             Project: Spark
          Issue Type: Improvement
          Components: Spark Core
    Affects Versions: 2.3.0
            Reporter: Juliusz Sompolski


ReadAheadInputStream was introduced in [apache/spark#18317|https://github.com/apache/spark/pull/18317] to
optimize reading spill files from disk.
However, investigating flamegraphs of profiles from investigating some regressed workloads
after switch to Spark 2.3, it seems that the hot path of reading small amounts of data (like
readInt) is inefficient - it involves taking locks, and multiple checks.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message