cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Branimir Lambov (JIRA)" <>
Subject [jira] [Created] (CASSANDRA-10520) Compressed writer and reader should support non-compressed data.
Date Wed, 14 Oct 2015 09:15:05 GMT
Branimir Lambov created CASSANDRA-10520:

             Summary: Compressed writer and reader should support non-compressed data.
                 Key: CASSANDRA-10520
             Project: Cassandra
          Issue Type: Improvement
            Reporter: Branimir Lambov
             Fix For: 3.0.x

Compressing uncompressible data, as done, for instance, to write SSTables during stress-tests,
results in chunks larger than 64k which are a problem for the buffer pooling mechanisms employed
by the {{CompressedRandomAccessReader}}. This results in non-negligible performance issues
due to excessive memory allocation.

To solve this problem and avoid decompression delays in the cases where it does not provide
benefits, I think we should allow compressed files to store uncompressed chunks as alternative
to compressed data. Such a chunk could be written after compression returns a buffer larger
than, for example, 90% of the input, and would not result in additional delays in writing.
On reads it could be recognized by size (using a single global threshold constant in the compression
metadata) and data could be directly transferred into the decompressed buffer, skipping the
decompression step and ensuring a 64k buffer for compressed data always suffices.

This message was sent by Atlassian JIRA

View raw message