spark-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From viirya <...@git.apache.org>
Subject [GitHub] spark pull request #22754: [SPARK-25776][CORE][MINOR]The disk write buffer s...
Date Tue, 23 Oct 2018 02:04:57 GMT
Github user viirya commented on a diff in the pull request:

    https://github.com/apache/spark/pull/22754#discussion_r227198037
  
    --- Diff: core/src/main/java/org/apache/spark/util/collection/unsafe/sort/UnsafeSorterSpillWriter.java
---
    @@ -62,6 +62,8 @@ public UnsafeSorterSpillWriter(
           int fileBufferSize,
           ShuffleWriteMetrics writeMetrics,
           int numRecordsToWrite) throws IOException {
    +    // Space used by prefix + len + recordLength is more than 4 + 8 bytes
    +    assert (diskWriteBufferSize > 12);
    --- End diff --
    
    Doesn't the updated `checkValue` of `spark.shuffle.spill.diskWriteBufferSize` already
guarantee this?


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Mime
View raw message