geode-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF subversion and git services (JIRA)" <>
Subject [jira] [Commented] (GEODE-478) Message class length field overflows if size is > 2GB
Date Mon, 22 Feb 2016 21:43:18 GMT


ASF subversion and git services commented on GEODE-478:

Commit 1e3f89ddcd4bfebe0e3e9c3e0f61478826d5fdde in incubator-geode's branch refs/heads/feature/GEODE-870
from [~bschuchardt]
[;h=1e3f89d ]

GEODE-478: Message class length field overflows if size is > 2GB

Message size is now restricted to 1GB.  If this is exceeded a
MessageTooLargeException is thrown.

I think the original intent of this ticket was to have messaging handle
the carving up of the message into multiple Messages somehow, but I think
this is the correct approach.  This will ensure that the problem doesn't
cause the current connection to be terminated and the operation retried
on another server and make sure that the exception gets back to the point
of initiation.

Barry already has a way to avoid large messages in WAN senders that can
be ported to Geode.

> Message class length field overflows if size is > 2GB
> -----------------------------------------------------
>                 Key: GEODE-478
>                 URL:
>             Project: Geode
>          Issue Type: Bug
>          Components: client/server
>            Reporter: Barry Oglesby
>            Assignee: Barry Oglesby
> This bug is described in the context of a {{GatewaySender}} batch, but I think it could
also happen with a client putAll.
> If the batch size is sufficiently large, and the data size as well, it is possible that
we attempt to send a message greater than 2GB, which blows up the receiver with an exception
like this one:
> {noformat}
> [warning 2015/10/07 21:06:29.213 UTC gemfire-data-node-cer-lx-gfirep03-49002 <ServerConnection
on port 1543 Thread 2> tid=0xc3af9] Server connection from [identity(<v2>:51883,connection=13;
port=18172]: Unexpected IOException: 
> Part length ( -2,000,248,199 ) and number of parts ( 18,001 ) inconsistent
> at com.gemstone.gemfire.internal.cache.tier.sockets.Message.readPayloadFields(
> at com.gemstone.gemfire.internal.cache.tier.sockets.Message.readHeaderAndPayload(
> at
> at com.gemstone.gemfire.internal.cache.tier.sockets.Message.recv(
> at com.gemstone.gemfire.internal.cache.tier.sockets.Message.recv(
> at com.gemstone.gemfire.internal.cache.tier.sockets.BaseCommand.readRequest(
> at com.gemstone.gemfire.internal.cache.tier.sockets.ServerConnection.doNormalMsg(
> at com.gemstone.gemfire.internal.cache.tier.sockets.ServerConnection.doOneMessage(
> at
> at java.util.concurrent.ThreadPoolExecutor.runWorker(
> at java.util.concurrent.ThreadPoolExecutor$
> at com.gemstone.gemfire.internal.cache.tier.sockets.AcceptorImpl$1$
> at
> {noformat}
> Nothing on the sender side keeps track of the total size of the message that will be
sent. We need to send earlier than satisfying the batch size if the buffer gets too large.
I would suggest 1GB as being the largest we should allow.

This message was sent by Atlassian JIRA

View raw message