druid-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From rahul gidwani <chu...@apache.org>
Subject compression strategy concurrency
Date Tue, 14 Sep 2021 21:29:07 GMT
What is the desired thread safety of the CompressionStrategy class?  From
looking at it from an API perspective, it looks to be you:

Allocate input buffer, Allocate output buffer, compress / decompress.

The CompressionStrategyTest.testConcurrency() test if you bump the number
of threads to 100, and run it a few times, you will see there are race
conditions which will cause failures.

The quick and easy solution to make this thread safe is to synchronize the
methods.  But in reality it looks like this class is mainly used to
compress, decompress segments so that will be a a thread / segment which is

My question is I am confused as to what should be the behavior, should it
be thread safe and a generic api to compress / decompress.  If so then we
should fix the code in Compression Strategy to be thread safe, if not then
maybe remove that test and mark the class as NotThreadSafe.

Was wondering about other people's thoughts.

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message