ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Fong <mcfong.o...@gmail.com>
Subject Re: Problem with reading incomplete payload - IGNITE-7153
Date Tue, 30 Oct 2018 03:07:11 GMT
bump :)

On Fri, Oct 26, 2018 at 4:00 PM Michael Fong <mcfong.open@gmail.com> wrote:

> Hi,
>
> Thanks for your reply. I think current implementation (based on R2.6) does
> exactly what you mentioned as of in GridNioServer
> <https://github.com/apache/ignite/blob/ignite-2.6/modules/core/src/main/java/org/apache/ignite/internal/util/nio/GridNioServer.java#L1087>.
> The problem is that in redis protocol defines message length right before
> the message context. If a huge message comes in and unfortunately Selector
> / NioServer has not read the whole payload fully. Furthermore, the
> incomplete message is passed to Redis packet parser, then logic error would
> occur - in this case BufferUnderFlowError. I wonder if this is expected
> behavior of how Ignite receive data from server perspective?
>
> Regards,
>
> Michael
>
> On Fri, Oct 26, 2018 at 1:13 PM Jörn Franke <jornfranke@gmail.com> wrote:
>
>> I have not checked the exact routine, but you need to reexecute read
>> until it returns -1 or if known that all bytes that are expected have been
>> read (if sockCh is based on some kind of input stream)
>>
>> > Am 26.10.2018 um 04:24 schrieb Michael Fong <mcfong.open@gmail.com>:
>> >
>> > Bump!
>> >
>> > Anyone who could help me solve this random issue? Thanks!
>> >
>> > Regards,
>> >
>> >
>> > Michael
>> >
>> >> On Wed, Oct 24, 2018 at 10:32 PM Michael Fong <mcfong.open@gmail.com>
>> wrote:
>> >>
>> >> Hi, all,
>> >>
>> >>
>> >> I was trying to fix  IGNITE-7153 which relates to parsing incomplete
>> REDIS
>> >> packet larger than 8192 bytes. However, I found a random problem
>> which is
>> >> reproducible on TC as well.
>> >> That said, GridNioServerRead.processRead() :
>> >>  - int cnt = sockCh.read(readBuf);
>> >>
>> >> sometimes does not read the payload fully as the length field in the
>> >> header is larger than the ByteBuffer.limit(). As the result, the
>> >> BufferUnderFlowException will be thrown.
>> >>
>> >> For example, in a erroneous round run with my IDE, a REDIS payload
>> (sent
>> >> by jedis client) looks like the following:
>> >>
>> >> 2a 33 d a 24 33 d a 53 45 54 d a 24 32 d a 62 31 d a 24 {38 31 39 32}
>> d a | 65
>> >> d a ...etc
>> >>
>> >> GridRedisProtocolParser.readBulkStr(buf) invokes elCnt(buf) which gets
>> >> {8192}. However, the limit of buf is 28 which ends at | position.
>> Obviously,
>> >> 8192 < limit(), therefore, the logic throws BufferUnderFlow soon after.
>> >>
>> >> I traced back and found  sockCh.read(readBuf) only read 28 bytes for
>> unknown reason
>> >>
>> >> The problem seems totally random. I wonder if this has anything to do
>> with other NioWorker
>> >> or Selector setting.
>> >>
>> >>
>> >> Any help would be appreciated!
>> >>
>> >>
>> >> Regards,
>> >>
>> >>
>> >> Michael
>> >>
>> >>
>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message