cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sylvain Lebresne (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-9225) CompactionAwareWriterTest is broken with compression
Date Fri, 24 Apr 2015 08:19:38 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-9225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14510636#comment-14510636
] 

Sylvain Lebresne commented on CASSANDRA-9225:
---------------------------------------------

bq. We want to split sstables based on their compressed size

Of course, I wasn't thinking clearly. Patch fixes the test with compression, +1.

> CompactionAwareWriterTest is broken with compression
> ----------------------------------------------------
>
>                 Key: CASSANDRA-9225
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-9225
>             Project: Cassandra
>          Issue Type: Bug
>            Reporter: Sylvain Lebresne
>            Assignee: Marcus Eriksson
>            Priority: Trivial
>             Fix For: 3.0
>
>         Attachments: 0001-9225.patch
>
>
> {{CompactionAwareWriterTest}} generates a relatively small sstable and expect different
types of compaction to split it at relatively precise sizes. However, the implementations
of {{CompactionAwareWriter}} use {{writer.getOnDiskFilePointer()}} to decide when to split
and with compression that value is updated only when an compressed block is written. Which
means that with compression the splitting is not as precise as the test expects it to be and
the test fails.
> Note that this imprecision in splitting is not a big deal in practice (you might even
call it a feature) since you expect to split sstables at sizes that are a lot bigger than
the compression block size (besides, we already know we are imprecise in our splitting size
since we only split at partition boundaries). But this does make things a bit harder to test.
> So I guess we could either:
> # make the test use much bigger values so it's not affected by the imprecision (with
the small downside that it will make the test slower)
> # remove that imprecision by making the {{CompactionAwareWriter}} implementations use
{{writer.getFilePointer()}}.
> I don't have much personal preference but using {{getFilePointer}} sounds like the simpler
fix, and it feels overall more predictable.
> Note that the test is not failing on trunk because unit tests do not use compression,
you have to call {{ant test-compression}} for that (in which case it will fail). We might
want to make compression one our randomized parameter of tests in the near future as I'm not
sure anyone is really running {{ant test-compression}} on a regular basis.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message