cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Taylan Develioglu (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (CASSANDRA-5903) Integer overflow in OffHeapBitSet when bloomfilter > 2GB
Date Wed, 21 Aug 2013 15:08:52 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-5903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13746118#comment-13746118
] 

Taylan Develioglu edited comment on CASSANDRA-5903 at 8/21/13 3:08 PM:
-----------------------------------------------------------------------

Sadly that wasn't sufficient, there's another overflow in OffHeapBitSet.deserialize:

ERROR [SSTableBatchOpen:6] 2013-08-21 15:29:51,799 CassandraDaemon.java (line 192) Exception
in thread Thread[SSTableBatchOpen:6,5,main]
java.lang.IllegalArgumentException
        at org.apache.cassandra.io.util.Memory.allocate(Memory.java:58)
        at org.apache.cassandra.utils.obs.OffHeapBitSet.deserialize(OffHeapBitSet.java:123)
        at org.apache.cassandra.utils.BloomFilterSerializer.deserialize(BloomFilterSerializer.java:46)
        at org.apache.cassandra.utils.Murmur2BloomFilter$Murmur2BloomFilterSerializer.deserialize(Murmur2BloomFilter.java:40)
        at org.apache.cassandra.utils.FilterFactory.deserialize(FilterFactory.java:71)
        at org.apache.cassandra.io.sstable.SSTableReader.loadBloomFilter(SSTableReader.java:365)
        at org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:195)
        at org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:153)
        at org.apache.cassandra.io.sstable.SSTableReader$1.run(SSTableReader.java:258)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
        at java.util.concurrent.FutureTask.run(FutureTask.java:166)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:724)


{code}
    public static OffHeapBitSet deserialize(DataInput dis) throws IOException
    {
        int byteCount = dis.readInt() * 8;
        Memory memory = RefCountedMemory.allocate(byteCount);
        for (int i = 0; i < byteCount;)
        {
            long v = dis.readLong();
            memory.setByte(i++, (byte) (v >>> 0));
            memory.setByte(i++, (byte) (v >>> 8));
            memory.setByte(i++, (byte) (v >>> 16));
            memory.setByte(i++, (byte) (v >>> 24));
            memory.setByte(i++, (byte) (v >>> 32));
            memory.setByte(i++, (byte) (v >>> 40));
            memory.setByte(i++, (byte) (v >>> 48));
            memory.setByte(i++, (byte) (v >>> 56));
        }
        return new OffHeapBitSet(memory);
    }
{code}
                
      was (Author: tdevelioglu):
    Sadly that wasn't sufficient, there's another overflow in OffHeapBitSet.deserialize:

{code}
    public static OffHeapBitSet deserialize(DataInput dis) throws IOException
    {
        int byteCount = dis.readInt() * 8;
        Memory memory = RefCountedMemory.allocate(byteCount);
        for (int i = 0; i < byteCount;)
        {
            long v = dis.readLong();
            memory.setByte(i++, (byte) (v >>> 0));
            memory.setByte(i++, (byte) (v >>> 8));
            memory.setByte(i++, (byte) (v >>> 16));
            memory.setByte(i++, (byte) (v >>> 24));
            memory.setByte(i++, (byte) (v >>> 32));
            memory.setByte(i++, (byte) (v >>> 40));
            memory.setByte(i++, (byte) (v >>> 48));
            memory.setByte(i++, (byte) (v >>> 56));
        }
        return new OffHeapBitSet(memory);
    }
{code}
                  
> Integer overflow in OffHeapBitSet when bloomfilter > 2GB
> --------------------------------------------------------
>
>                 Key: CASSANDRA-5903
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-5903
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>            Reporter: Taylan Develioglu
>            Assignee: Vijay
>             Fix For: 1.2.9
>
>         Attachments: 0001-CASSANDRA-5903.patch
>
>
> In org.apache.cassandra.utils.obs.OffHeapBitSet.
> byteCount overflows and causes an IllegalArgument exception in Memory.allocate when bloomfilter
is > 2GB.
> Suggest changing byteCount to long.
> {code:title=OffHeapBitSet.java}
>     public OffHeapBitSet(long numBits)
>     {
>         // OpenBitSet.bits2words calculation is there for backward compatibility.
>         int byteCount = OpenBitSet.bits2words(numBits) * 8;
>         bytes = RefCountedMemory.allocate(byteCount);
>         // flush/clear the existing memory.
>         clear();
>     }
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message