spark-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From tdas <...@git.apache.org>
Subject [GitHub] spark pull request #20710: [SPARK-23559][SS] Add epoch ID to DataWriterFacto...
Date Sat, 03 Mar 2018 00:11:15 GMT
Github user tdas commented on a diff in the pull request:

    https://github.com/apache/spark/pull/20710#discussion_r171993983
  
    --- Diff: sql/core/src/main/java/org/apache/spark/sql/sources/v2/writer/streaming/StreamWriter.java
---
    @@ -39,21 +36,21 @@
        * If this method fails (by throwing an exception), this writing job is considered
to have been
        * failed, and the execution engine will attempt to call {@link #abort(WriterCommitMessage[])}.
        *
    -   * To support exactly-once processing, writer implementations should ensure that this
method is
    -   * idempotent. The execution engine may call commit() multiple times for the same epoch
    -   * in some circumstances.
    +   * The execution engine may call commit() multiple times for the same epoch in some
circumstances.
    --- End diff --
    
    Somewhere in this file, add docs about what epochId means for MB and C execution. 


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Mime
View raw message