hudi-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <>
Subject [GitHub] [hudi] liujinhui1994 commented on pull request #2438: [HUDI-1447] DeltaStreamer kafka source supports consuming from specified timestamp
Date Wed, 02 Jun 2021 12:04:59 GMT

liujinhui1994 commented on pull request #2438:

   I am currently facing a problem and would like to hear your opinion
   After we add this type, hoodie.deltastreamer.source.kafka.checkpoint.type=timestamp
   I am currently thinking, does deltastreamer.checkpoint.key maintain the status quo? The
format is still: topicName,0:123,1:456
   If we continue to maintain the above format, when we specify: for example --checkpoint
1622635064, we need to determine the relationship between commitMetadata.getMetadata(CHECKPOINT_KEY)
and --checkpoint 1622635064 in org.apache.hudi.utilities.deltastreamer.DeltaSync#readFromSource,
This seems to be contrary to the results of our discussion, do not add kafka dependent code
in DeltaSync
   Do you have any suggestions for this? thanks 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:

View raw message