flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Elias Levy (JIRA)" <j...@apache.org>
Subject [jira] [Created] (FLINK-4501) Cassandra sink can lose messages
Date Thu, 25 Aug 2016 22:53:20 GMT
Elias Levy created FLINK-4501:
---------------------------------

             Summary: Cassandra sink can lose messages
                 Key: FLINK-4501
                 URL: https://issues.apache.org/jira/browse/FLINK-4501
             Project: Flink
          Issue Type: Bug
          Components: Cassandra Connector
    Affects Versions: 1.1.0
            Reporter: Elias Levy


The problem is the same as I pointed out with the Kafka producer sink (FLINK-4027).  The CassandraTupleSink's
send() and CassandraPojoSink's send() both send data asynchronously to Cassandra and record
whether an error occurs via a future callback.  But CassandraSinkBase does not implement Checkpointed,
so it can't stop checkpoint from happening even though the are Cassandra queries in flight
from the checkpoint that may fail.  If they do fail, they would subsequently not be replayed
when the job recovered, and would thus be lost.

In addition, 
CassandraSinkBase's close should check whether there is a pending exception and throw it,
rather than silently close.  It should also wait for any pending async queries to complete
and check their status before closing.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message