hudi-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "liujinhui (Jira)" <j...@apache.org>
Subject [jira] [Commented] (HUDI-648) Implement error log/table for Datasource/DeltaStreamer/WriteClient/Compaction writes
Date Thu, 19 Mar 2020 02:29:00 GMT

    [ https://issues.apache.org/jira/browse/HUDI-648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17062242#comment-17062242
] 

liujinhui commented on HUDI-648:
--------------------------------

Hello, I also encountered this problem recently. Occasionally the kafka data received by hudi
is wrong, the consumption is reported wrong, and it needs to be skipped. It seems that it
cannot be skipped at the moment. Do you have a solution for this?

[~vinoth]

> Implement error log/table for Datasource/DeltaStreamer/WriteClient/Compaction writes
> ------------------------------------------------------------------------------------
>
>                 Key: HUDI-648
>                 URL: https://issues.apache.org/jira/browse/HUDI-648
>             Project: Apache Hudi (incubating)
>          Issue Type: New Feature
>          Components: DeltaStreamer, Spark Integration, Writer Core
>            Reporter: Vinoth Chandar
>            Priority: Major
>             Fix For: 0.6.0
>
>
> We would like a way to hand the erroring records from writing or compaction back to the
users, in a separate table or log. This needs to work generically across all the different
writer paths.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message