flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tzu-Li (Gordon) Tai (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (FLINK-5353) Elasticsearch Sink loses well-formed documents when there are malformed documents
Date Wed, 01 Feb 2017 08:18:51 GMT

    [ https://issues.apache.org/jira/browse/FLINK-5353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15848109#comment-15848109
] 

Tzu-Li (Gordon) Tai edited comment on FLINK-5353 at 2/1/17 8:18 AM:
--------------------------------------------------------------------

Ok, then I think the actual cause for the well-formed documents to be missing also isn't because
the {{BulkProcessor}} is dropping them due to the malformed documents, but simply because
at-least-once isn't properly supported in the ES connector yet (see FLINK-5487). Any documents
that were buffered in the {{BulkProcessor}} at the time of failure simply could not be recovered.

Thank you for the feedback. With the suggested {{FailedActionRequestHandler}}, you can simply
just drop the malformed document or re-process it if you want to.


was (Author: tzulitai):
Ok, then I think the actual cause for the well-formed documents to be missing also isn't because
the {{BulkProcessor}} is dropping them due to the malformed documents, but simply because
at-least-once isn't properly supported in the ES connector yet (see FLINK-5487). Any documents
that were buffered in the {{BulkProcessor}} at the time of failure simply could not be recovered.

With the suggested {{FailedActionRequestHandler}}, you can simply just drop the malformed
document or re-process it if you want to.

> Elasticsearch Sink loses well-formed documents when there are malformed documents
> ---------------------------------------------------------------------------------
>
>                 Key: FLINK-5353
>                 URL: https://issues.apache.org/jira/browse/FLINK-5353
>             Project: Flink
>          Issue Type: Bug
>          Components: Streaming Connectors
>    Affects Versions: 1.1.3
>            Reporter: Flavio Pompermaier
>




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message