hudi-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [incubator-hudi] vinothchandar commented on issue #1377: [HUDI-663] Fix HoodieDeltaStreamer offset not handled correctly
Date Mon, 09 Mar 2020 06:52:23 GMT
vinothchandar commented on issue #1377: [HUDI-663] Fix HoodieDeltaStreamer offset not handled
correctly
URL: https://github.com/apache/incubator-hudi/pull/1377#issuecomment-596358157
 
 
   >>Create a kafka topic(don't produce any data to kafka) which means a topic with
no messages, then start the delta streamer, hudi will store the empty checkpoint.
   
   I assume there will be partitions created already.. So instead of special casing the empty
checkpoint for a specific source (which seems higher maintenance), can we make the code store
an actual checkpoint.. Our contract can be that if a checkpoint is stored, then it is indeed
non empty.. 
   
   I am with @garyli1019  that erroring out on empty checkpoints is better.. Not sure if we
expect an empty checkpoint in other sources as well 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

Mime
View raw message