spark-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From jose-torres <...@git.apache.org>
Subject [GitHub] spark issue #21490: [SPARK-24462][SS] Initialize the offsets correctly when ...
Date Mon, 04 Jun 2018 22:54:58 GMT
Github user jose-torres commented on the issue:

    https://github.com/apache/spark/pull/21490
  
    The proposed strategy still won't work correctly, though. If a batch fails before commit,
the restart won't replay the same records that were in that batch, but the new ones incoming
from the socket.
    
    I agree that the current state is non-ideal, but IMO we should either make batches actually
replayable or add an exception explicitly stating that it's not allowed to restart a text
socket source.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Mime
View raw message