hudi-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Pratyaksh Sharma (Jira)" <j...@apache.org>
Subject [jira] [Created] (HUDI-728) Support for complex record keys with TimestampBasedKeyGenerator
Date Fri, 20 Mar 2020 14:01:00 GMT
Pratyaksh Sharma created HUDI-728:
-------------------------------------

             Summary: Support for complex record keys with TimestampBasedKeyGenerator
                 Key: HUDI-728
                 URL: https://issues.apache.org/jira/browse/HUDI-728
             Project: Apache Hudi (incubating)
          Issue Type: Improvement
          Components: DeltaStreamer, Usability
            Reporter: Pratyaksh Sharma
            Assignee: Pratyaksh Sharma
             Fix For: 0.6.0


We have TimestampBasedKeyGenerator for defining custom partition paths and we have ComplexKeyGenerator
for supporting having combination of fields as record key or partition key. 
 
However we do not have support for the case where one wants to have combination of fields
as record key along with being able to define custom partition paths. This use case recently
came up at my organisation. 
 
How about having CustomTimestampBasedKeyGenerator which supports the above use case? This
class can simply extend TimestampBasedKeyGenerator and allow users to have combination of
fields as record key.
 
We will try to have the implementation as generic as possible. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message