Mailing-List: contact commits-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@cassandra.apache.org
Date: Mon, 27 Feb 2017 03:03:45 +0000 (UTC)
From: "Stefania (JIRA)" <jira@apache.org>
To: commits@cassandra.apache.org
Message-ID: <JIRA.12904794.1444814058000.2204.1488164625563@Atlassian.JIRA>
In-Reply-To: <JIRA.12904794.1444814058000@Atlassian.JIRA>
References: <JIRA.12904794.1444814058000@Atlassian.JIRA> <JIRA.12904794.1444814058249@jira-lw-us.apache.org>
Subject: [jira] [Commented] (CASSANDRA-10520) Compressed writer and reader
 should support non-compressed data.
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Mon, 27 Feb 2017 03:03:52 -0000


    [ https://issues.apache.org/jira/browse/CASSANDRA-10520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15885063#comment-15885063 ] 

Stefania commented on CASSANDRA-10520:
--------------------------------------

If the default value of {{min_compress_ratio}} is changed in the follow-up commit, the cqlshlib tests should also be updated, see [8bfa82769|https://github.com/apache/cassandra/commit/8bfa8276968db3d25d3e305a0fad156eeb904c34]. 

[{{test_describe}}|http://cassci.datastax.com/view/trunk/job/trunk_dtest/lastCompletedBuild/testReport/cqlsh_tests.cqlsh_tests/TestCqlsh/test_describe/] and [{{test_describe_mv}}|http://cassci.datastax.com/view/trunk/job/trunk_dtest/lastCompletedBuild/testReport/cqlsh_tests.cqlsh_tests/TestCqlsh/test_describe_mv/] need fixing if not already done, thanks.

> Compressed writer and reader should support non-compressed data.
> ----------------------------------------------------------------
>
>                 Key: CASSANDRA-10520
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-10520
>             Project: Cassandra
>          Issue Type: Improvement
>          Components: Local Write-Read Paths
>            Reporter: Branimir Lambov
>            Assignee: Branimir Lambov
>              Labels: messaging-service-bump-required
>             Fix For: 4.x
>
>         Attachments: ReadWriteTestCompression.java
>
>
> Compressing uncompressible data, as done, for instance, to write SSTables during stress-tests, results in chunks larger than 64k which are a problem for the buffer pooling mechanisms employed by the {{CompressedRandomAccessReader}}. This results in non-negligible performance issues due to excessive memory allocation.
> To solve this problem and avoid decompression delays in the cases where it does not provide benefits, I think we should allow compressed files to store uncompressed chunks as alternative to compressed data. Such a chunk could be written after compression returns a buffer larger than, for example, 90% of the input, and would not result in additional delays in writing. On reads it could be recognized by size (using a single global threshold constant in the compression metadata) and data could be directly transferred into the decompressed buffer, skipping the decompression step and ensuring a 64k buffer for compressed data always suffices.


--
This message was sent by Atlassian JIRA
(v6.3.15#6346)