pulsar-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [pulsar] massakam opened a new issue #6437: Backlog messages increase if no consumer is connected to replicated subscription
Date Fri, 28 Feb 2020 04:49:29 GMT
massakam opened a new issue #6437: Backlog messages increase if no consumer is connected to
replicated subscription
URL: https://github.com/apache/pulsar/issues/6437
 
 
   **Describe the bug**
   
   If no consumer is connected to a replicated subscription, the number of backlog messages
continues to increase. The same is true even if no new messages have been published by the
producers.
   
   If this state is left for a long time, the storage allocated to the topic will be exhausted
and message publishing will be blocked or backlog messages will be discarded.
   
   **To Reproduce**
   
   Steps to reproduce the behavior:
   1. Connect a consumer with `replicateSubscriptionState` set to true to a global topic
   2. Disconnect the consumer
   
   **Expected behavior**
   
   It seems that marker messages accumulate in the backlog. These messages should be cleared
automatically.
   
   **Screenshots**
   
   Topic stats:
   ```json
   {
     "msgRateIn" : 2.000001502001128,
     "msgThroughputIn" : 218.00016371812293,
     "msgRateOut" : 2.0000015018011275,
     "msgThroughputOut" : 108.00008109726089,
     "averageMsgSize" : 109.0,
     "storageSize" : 2308087,
     "backlogSize" : 1872241,
     "publishers" : [ ],
     "subscriptions" : {
       "sub1" : {
         "msgRateOut" : 0.0,
         "msgThroughputOut" : 0.0,
         "msgRateRedeliver" : 0.0,
         "msgBacklog" : 17900,
         "blockedSubscriptionOnUnackedMsgs" : false,
         "msgDelayed" : 0,
         "unackedMessages" : 0,
         "type" : "Exclusive",
         "msgRateExpired" : 0.0,
         "consumers" : [ ],
         "isReplicated" : true
       }
     },
     "replication" : {
       "dev2" : {
         "msgRateIn" : 2.000001502001128,
         "msgThroughputIn" : 218.00016371812293,
         "msgRateOut" : 2.0000015018011275,
         "msgThroughputOut" : 108.00008109726089,
         "msgRateExpired" : 0.0,
         "replicationBacklog" : 0,
         "connected" : true,
         "replicationDelayInSeconds" : 0,
         "inboundConnection" : "/xxx.xxx.xxx.xxx:42024",
         "inboundConnectedSince" : "2020-02-28T11:40:53.817+09:00",
         "outboundConnection" : "[id: 0xa2bf7f6c, L:/xxx.xxx.xxx.xxx:52110 - R:dev-broker200.pulsar.xxx.xxx.yahoo.co.jp/xxx.xxx.xxx.xxx:6651]",
         "outboundConnectedSince" : "2020-02-28T11:40:53.817+09:00"
       }
     },
     "deduplicationStatus" : "Disabled"
   }
   ```
   
   **Desktop (please complete the following information):**
    - OS: CentOS 7.7
    - Pulsar: 2.4.2

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

Mime
View raw message