geode-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dan Smith (JIRA)" <j...@apache.org>
Subject [jira] [Created] (GEODE-3232) Race condition serializing FilterProfile
Date Mon, 17 Jul 2017 23:35:00 GMT
Dan Smith created GEODE-3232:
--------------------------------

             Summary: Race condition serializing FilterProfile
                 Key: GEODE-3232
                 URL: https://issues.apache.org/jira/browse/GEODE-3232
             Project: Geode
          Issue Type: Bug
          Components: client queues
            Reporter: Dan Smith


We saw an error we can't reproduce deserializing a FilterProfile.

{noformat}
[severe 2017/07/07 21:31:28.538 PDT bridgegemfire4_rs-myFullRegr-client-23_13409 <P2P message
reader for rs-myFullRegr-client-23(bridgegemfire1_rs-myFullRegr-client-23_13408:13408)<ec><v7>:1025
shared unordered uid=3 port=56290> tid=0x5c] Error deserializing message
org.apache.geode.SerializationException: Could not create an instance of  org.apache.geode.internal.cache.CreateRegionProcessor$CreateRegionReplyMessage
.
        at org.apache.geode.internal.InternalDataSerializer.invokeFromData(InternalDataSerializer.java:2381)
        at org.apache.geode.internal.DSFIDFactory.create(DSFIDFactory.java:981)
        at org.apache.geode.internal.InternalDataSerializer.readDSFID(InternalDataSerializer.java:2573)
        at org.apache.geode.internal.tcp.Connection.processNIOBuffer(Connection.java:3530)
        at org.apache.geode.internal.tcp.Connection.runNioReader(Connection.java:1814)
        at org.apache.geode.internal.tcp.Connection.run(Connection.java:1675)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.geode.SerializationException: Could not create an instance of  org.apache.geode.internal.cache.partitioned.RegionAdvisor$PartitionProfile
.
        at org.apache.geode.internal.InternalDataSerializer.invokeFromData(InternalDataSerializer.java:2381)
        at org.apache.geode.internal.DSFIDFactory.create(DSFIDFactory.java:981)
        at org.apache.geode.internal.InternalDataSerializer.basicReadObject(InternalDataSerializer.java:2691)
        at org.apache.geode.DataSerializer.readObject(DataSerializer.java:2961)
        at org.apache.geode.internal.cache.CreateRegionProcessor$CreateRegionReplyMessage.fromData(CreateRegionProcessor.java:798)
        at org.apache.geode.internal.InternalDataSerializer.invokeFromData(InternalDataSerializer.java:2370)
        ... 8 more
Caused by: org.apache.geode.SerializationException: Could not create an instance of  org.apache.geode.internal.cache.FilterProfile
.
        at org.apache.geode.internal.InternalDataSerializer.invokeFromData(InternalDataSerializer.java:2381)
        at org.apache.geode.internal.DSFIDFactory.create(DSFIDFactory.java:981)
        at org.apache.geode.internal.InternalDataSerializer.basicReadObject(InternalDataSerializer.java:2691)
        at org.apache.geode.DataSerializer.readObject(DataSerializer.java:2961)
        at org.apache.geode.internal.cache.CacheDistributionAdvisor$CacheProfile.fromData(CacheDistributionAdvisor.java:867)
        at org.apache.geode.internal.cache.partitioned.RegionAdvisor$PartitionProfile.fromData(RegionAdvisor.java:586)
        at org.apache.geode.internal.InternalDataSerializer.invokeFromData(InternalDataSerializer.java:2370)
        ... 13 more
Caused by: java.nio.BufferUnderflowException
        at java.nio.Buffer.nextGetIndex(Buffer.java:506)
        at java.nio.DirectByteBuffer.getInt(DirectByteBuffer.java:681)
        at org.apache.geode.internal.tcp.ByteBufferInputStream$ByteBufferByteSource.getInt(ByteBufferInputStream.java:267)
        at org.apache.geode.internal.tcp.ByteBufferInputStream.readInt(ByteBufferInputStream.java:963)
        at org.apache.geode.internal.InternalDataSerializer.readSetOfLongs(InternalDataSerializer.java:1775)
        at org.apache.geode.internal.cache.FilterProfile.fromData(FilterProfile.java:1469)
        at org.apache.geode.internal.InternalDataSerializer.invokeFromData(InternalDataSerializer.java:2370)
        ... 19 more
{noformat}


>From code inspection ,it looks like FilterProfile stores allKeyClients and some other
fields CopyOnWriteHashSets, which makes me think they can be concurrently modified.

However, FilterProfile.toData does this to write out this hash set.

{code}
    InternalDataSerializer.writeSetOfLongs(this.allKeyClients, this.clientMap.hasLongID, out);
{code}

Within writeSetOfLongs, it first reads the size and writes it to the stream, and then gets
an iterator on the set. Between those two calls the set could be modified.

I think the code needs to do something like this for all of these sets to be threadsafe:

{code}
    InternalDataSerializer.writeSetOfLongs(this.allKeyClients.getSnapshot(), this.clientMap.hasLongID,
out);
{code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message