pulsar-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] codelipenghui commented on issue #3131: Consumer stop receive messages from broker
Date Fri, 11 Jan 2019 01:11:01 GMT
codelipenghui commented on issue #3131: Consumer stop receive messages from broker
URL: https://github.com/apache/pulsar/issues/3131#issuecomment-453328896
 
 
   ### Broker dispatch process:
   1. Consumer subscribe to a topic, consumer will send a flow request.
   2. By default, broker will read 100 entries from ledger and distribute entries to 5 consumers,
one consumer will get 20 entries, in the test case, every entries has 2000 messages by enable
the batching message in producer.
   3. MaxUnackMessagesPerConsumer in broker settings is 100 and MaxUnackMessagesPerSubscription
is 500.
   4. So when broker send messages to the first consumer, dispatcher will be blocked, because
20 entries has 40000 messages. I have to consumers with receive queue size 100, so the dispatcher
init available permits is 200, after send 40000 messages to the first consumer, available
permits is 39980.
   5. Consumer received messages and when call receive time > 50(receive queue size is
100), consumer will send permits request to the broker again, but now consumer is blocked
by un-ack messages, so broker do not send any messages to consumer, and consumer will increase
`permitsReceivedWhileConsumerBlocked`.
   6. While consumer send a redelivery request, dispatcher should release the flow control,
but the logic in the consumer, consumer will use min(`permitsReceivedWhileConsumerBlocked`,`totalRedelivery`)
to increase the available permits in the dispatcher. I think the problem is here, if totalRedelivery
< permitsReceivedWhileConsumerBlocked, consumer is already send flow request, but broker
can't release the flow control. When a consumer receive 10000 messages(5 message, batch size
is 2000), consumer will send 200 times flow request(50 messages per request),  if permitsReceivedWhileConsumerBlocked
= 9000, at this time consumer redelivery 2000 messages, dispatcher will use 2000 to increase
the available permits. after consumer send the last 1000 messages flow request, dispatcher
is until blocked.
   
   This is the test log:
   
   [stop-consume-broker.log](https://github.com/apache/pulsar/files/2747607/stop-consume-broker.log)
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

Mime
View raw message