hudi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Satish Kotha <>
Subject Re: [DISCUSS] Hash Index for HUDI
Date Wed, 02 Jun 2021 17:51:22 GMT
+1.   You may want to read this thread
as well. There are minor differences between these threads, but the high
level idea is similar.

On Wed, Jun 2, 2021 at 7:42 AM 耿筱喻 <> wrote:

> Hi,
> Currently, Hudi index implementation is pluggable and provides two
> options: bloom filter and hbase. When a Hudi table becomes large, the
> performance of bloom filter degrade drastically due to the increase in
> false positive probability.
> Hash index is an efficient light-weight approach to address the
> performance issue. It is used in Hive called Bucket, which clusters the
> records whose key have the same hash value under a unique hash function.
> This pre-distribution can accelerate the sql query in some scenarios.
> Besides, Bucket in Hive offers the efficient sampling.
> I make a RFC for this
> Feel free to discuss under this thread and suggestions are welcomed.
> Regards,
> Shawy

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message