cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Pavel Yaskevich (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-3001) Make the compression algorithm and chunk length configurable
Date Tue, 30 Aug 2011 18:43:39 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-3001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13093988#comment-13093988
] 

Pavel Yaskevich commented on CASSANDRA-3001:
--------------------------------------------

Applied 0001 and 0003 and used `ant gen-thrift-java && ant avro-generate &&
ant clean`.

CompressedRandomAccessReaderTest weren't to compile at first because

Line 108 should be changed to

{noformat}
SequentialWriter writer = new CompressedSequentialWriter(file, metadata.getPath(), false,
new CompressionParameters(SnappyCompressor.instance));
{noformat}

After that modifications `ant test` succeeded.

Tried to run `ant test-compression` and various tests are failing exception similar to this:

{noformat}
    [junit] Testcase: testIndexCreate(org.apache.cassandra.db.ColumnFamilyStoreTest):	Caused
an ERROR
    [junit] error reading 1 of 1
    [junit] java.lang.RuntimeException: error reading 1 of 1
    [junit] 	at org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleSliceReader.java:83)
    [junit] 	at org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleSliceReader.java:40)
    [junit] 	at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:140)
    [junit] 	at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:135)
    [junit] 	at org.apache.cassandra.db.columniterator.SSTableSliceIterator.hasNext(SSTableSliceIterator.java:107)
    [junit] 	at org.apache.cassandra.utils.MergeIterator$Candidate.advance(MergeIterator.java:194)
    [junit] 	at org.apache.cassandra.utils.MergeIterator.<init>(MergeIterator.java:47)
    [junit] 	at org.apache.cassandra.utils.MergeIterator$ManyToOne.<init>(MergeIterator.java:142)
    [junit] 	at org.apache.cassandra.utils.MergeIterator.get(MergeIterator.java:66)
    [junit] 	at org.apache.cassandra.db.filter.QueryFilter.collateColumns(QueryFilter.java:96)
    [junit] 	at org.apache.cassandra.db.CollationController.collectAllData(CollationController.java:226)
    [junit] 	at org.apache.cassandra.db.CollationController.getTopLevelColumns(CollationController.java:61)
    [junit] 	at org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1318)
    [junit] 	at org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1171)
    [junit] 	at org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1122)
    [junit] 	at org.apache.cassandra.db.index.keys.KeysSearcher.search(KeysSearcher.java:203)
    [junit] 	at org.apache.cassandra.db.index.SecondaryIndexManager.search(SecondaryIndexManager.java:428)
    [junit] 	at org.apache.cassandra.db.ColumnFamilyStore.search(ColumnFamilyStore.java:1405)
    [junit] 	at org.apache.cassandra.db.ColumnFamilyStoreTest.queryBirthdate(ColumnFamilyStoreTest.java:485)
    [junit] 	at org.apache.cassandra.db.ColumnFamilyStoreTest.testIndexCreate(ColumnFamilyStoreTest.java:465)
    [junit] Caused by: java.nio.channels.ClosedChannelException
    [junit] 	at org.apache.cassandra.io.util.RandomAccessReader.read(RandomAccessReader.java:268)
    [junit] 	at java.io.RandomAccessFile.readByte(RandomAccessFile.java:623)
    [junit] 	at org.apache.cassandra.utils.ByteBufferUtil.readShortLength(ByteBufferUtil.java:356)
    [junit] 	at org.apache.cassandra.utils.ByteBufferUtil.readWithShortLength(ByteBufferUtil.java:367)
    [junit] 	at org.apache.cassandra.db.ColumnSerializer.deserialize(ColumnSerializer.java:87)
    [junit] 	at org.apache.cassandra.db.ColumnSerializer.deserialize(ColumnSerializer.java:82)
    [junit] 	at org.apache.cassandra.db.ColumnSerializer.deserialize(ColumnSerializer.java:72)
    [junit] 	at org.apache.cassandra.db.ColumnSerializer.deserialize(ColumnSerializer.java:36)
    [junit] 	at org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleSliceReader.java:79)
{noformat}

The same happened when I re-compiled and run `./tools/stress/bin/stress -n 10000 -I SnappyCompressor`
and then tried to read the data.

Have you tried to run the tests?

> Make the compression algorithm and chunk length configurable
> ------------------------------------------------------------
>
>                 Key: CASSANDRA-3001
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-3001
>             Project: Cassandra
>          Issue Type: Improvement
>          Components: Core
>            Reporter: Sylvain Lebresne
>            Assignee: Sylvain Lebresne
>            Priority: Minor
>              Labels: compression
>             Fix For: 1.0
>
>         Attachments: 0001-Pluggable-algorithm-and-chunk-length.patch, 0002-Thrift-crap.patch,
0003-Add-deflate-compressor.patch
>
>


--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message