hudi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "vbalaji@apache.org" <vbal...@apache.org>
Subject Re: Hoodie dataset write without partition
Date Mon, 24 Jun 2019 21:57:36 GMT
 
Hi Amarnath,
Apart from changing the partition extractor class, you would need to change the keyGeneratorClass
for non-partitioned table.
Use this param "--key-generator-class com.uber.hoodie.NonpartitionedKeyGenerator" as part
of DeltaStreamer command-line execution.
Also, ensure we have the following configs defined in the properties file used by delta-streamer:
hoodie.datasource.write.keygenerator.class=com.uber.hoodie.NonpartitionedKeyGeneratorhoodie.datasource.hive_sync.partition_extractor_class=com.uber.hoodie.hive.NonPartitionedExtractorWe
will eventually remove the DeltaStreamer CLI and rely on the properties config for uniform
handling.

Thanks,Balaji.V
    On Monday, June 24, 2019, 1:55:51 PM PDT, Balaji Varadarajan <v.balaji@ymail.com.INVALID>
wrote:  
 
  Hi Amarnath,
I will look into it and reply back by EOD today.
Balaji.V
    On Sunday, June 23, 2019, 8:21:51 AM PDT, Amarnath Venkataswamy <amarnath.venkataswamy@gmail.com>
wrote:  
 
 Hi 

Is there any option to write the hoodie dataset without any partition?

I tried but hive sync is failing when you sync up without any partition.

Delta streamer creates with default as partition when there is no partition column.


Sent from my iPhone    
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message