hudi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vinay Patil <vinay18.pa...@gmail.com>
Subject [DISCUSS] Create Spark and Flink utilities module
Date Sat, 17 Jul 2021 14:57:53 GMT
Hi Team,

As part of https://issues.apache.org/jira/browse/HUDI-1872, we are creating
a separate flink-utilities module. Based on our discussion on the PR,
should we even create a spark-utilities module. This would look like :

hudi-utilities
├── hudi-flink-utilities
└── hudi-spark-utilities

This would also mean to create separate utilities-bundle for Flink and
Spark,

hudi-utilities-bundle
 ├── hudi-flink-utilities-bundle
 └── hudi-spark-utilities-bundle

This is not a backward compatible change as users will have to provide an
engine specific bundle. IMO, since Hudi is supporting Flink and Spark it
will be good to have  engine specific bundle.

What do you think?

Regards,
Vinay Patil

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message