Mailing-List: contact users-help@activemq.apache.org; run by ezmlm
Precedence: bulk
Reply-To: users@activemq.apache.org
Received-SPF: pass (athena.apache.org: domain of suonayi2006@163.com
 designates 220.181.13.144 as permitted sender)
Date: Thu, 18 Apr 2013 09:09:51 +0800 (CST)
From: SuoNayi  <suonayi2006@163.com>
To: users@activemq.apache.org
Subject: Re:Re: Re: Re: What can be reason of 460%+ memory usage limit
In-Reply-To: 
 <CAKS7T5Pp0b8ANnKYCtjNH8=9mUApNn8D-s6tOZE5wt6Y9WsZjg@mail.gmail.com>
References: <167f3ed7.fa59.13de4e2095e.Coremail.suonayi2006@163.com>
 <CAKS7T5ND0wfPqcteOJFR9HPS9Vrh=qTmwtMLuEUfzCdc5RvxFA@mail.gmail.com>
 <1c6ea58c.15538.13e18ca1c97.Coremail.suonayi2006@163.com>
 <CAKS7T5NGow3Yt9ucztnVn0kw9UnCKXycuDOVFDruJMUEYAEmRA@mail.gmail.com>
 <2a71ed9a.15664.13e18d6bdcc.Coremail.suonayi2006@163.com>
 <CAKS7T5Pp0b8ANnKYCtjNH8=9mUApNn8D-s6tOZE5wt6Y9WsZjg@mail.gmail.com>
Content-Type: multipart/alternative;
	boundary="----=_Part_38286_54673929.1366247391926"
MIME-Version: 1.0
Message-ID: <5849f100.27b2.13e1ab0c2b7.Coremail.suonayi2006@163.com>

------=_Part_38286_54673929.1366247391926
Content-Type: text/plain; charset=GBK
Content-Transfer-Encoding: 7bit

B/c we use the unique feature the VT provides so we won't uses the regular topic subscription.
Anyway,when you use the VT and want to disable PFC, you have to disable PFC for the VT and all the queues that are matched in runtime.
People may not be aware of this usage pattern and potentially gets into the trouble I faced.


At 2013-04-18 00:56:28,"Christian Posta" <christian.posta@gmail.com> wrote:
>Well, I believe you can consume directly from the VT using regular topic
>semantics.
>
>
>On Wed, Apr 17, 2013 at 9:32 AM, SuoNayi <suonayi2006@163.com> wrote:
>
>> yes, if we do that the retroactive consumers won't be able to get passed
>> messages.
>> Since we can always expect that there are no consumers on the virtual
>> topic,
>> is it reasonable if the topic is virtual we will do that optimization ?
>>
>>
>>
>>
>> At 2013-04-18 00:21:25,"Christian Posta" <christian.posta@gmail.com>
>> wrote:
>> >that's a good point... not sure what the value of flow controlling the
>> >producer if there are no consumers to a topic... maybe preserving
>> >retroactive consumers?
>> >
>> >
>> >On Wed, Apr 17, 2013 at 9:18 AM, SuoNayi <suonayi2006@163.com> wrote:
>> >
>> >> The message flow can not go there at the moment.
>> >> In the send method it will check if the memory is full at first, if full
>> >> then the thread will wait for space and the producer gets blocked.
>> >> I mean before checking the memory is full or not, we can see if there
>> are
>> >> consumers, if none the thread just returns.
>> >> So we can avoid blocking the producer.
>> >>
>> >>
>> >> At 2013-04-18 00:08:33,"Christian Posta" <christian.posta@gmail.com>
>> >> wrote:
>> >> >Check the dispatch method in Topic.java... we do just that:
>> >> >
>> >> >            synchronized (consumers) {
>> >> >                if (consumers.isEmpty()) {
>> >> >                    onMessageWithNoConsumers(context, message);
>> >> >                    return;
>> >> >                }
>> >> >            }
>> >> >
>> >> >onMessageWithNoConsumers really just sends advisories for specific
>> cases
>> >> >
>> >> >
>> >> >On Sun, Apr 7, 2013 at 7:24 AM, SuoNayi <suonayi2006@163.com> wrote:
>> >> >
>> >> >> Hi,we're using the virtual topic to scatter messages to 12
>> >> >> queues/consumers.
>> >> >>
>> >> >> When one consumer of one queue becomes very slow since it's located
>> in
>> >> >> London,
>> >> >> far from our data center in china, our single producer becomes very
>> slow
>> >> >> to publish messages.
>> >> >> What phenomenon we observed is that:
>> >> >> When the slow consumer located in London dequeues 200 messages the
>> >> single
>> >> >> producer can publish 200 messages or it will be blocked.
>> >> >> It seems that PFC is working but I ensure I have disabled PFC for
>> queues
>> >> >> when the broker is deployed in production.
>> >> >> But PFC for topics is enabled and I have make that disabled and given
>> >> the
>> >> >> broker a reboot.
>> >> >> After that my producer can keep publishing messages regardless of the
>> >> slow
>> >> >> consumer but the memory usage limit keeps increasing all the time.
>> >> >> Eventually I saw the 460+ memory usage limit reached which made me
>> very
>> >> >> surprised.
>> >> >> AFAIK, the pending messages in transactions can contribute to the
>> >> exceeded
>> >> >> memory usage limit.
>> >> >> Since we only have single producer and we send 100 messages in a
>> batch
>> >> >> using a transaction and every message size is less than 1kb,
>> >> >> So I can not understand how the  memory usage limit exceeds so much.
>> >> >>
>> >> >>
>> >> >> Thanks,
>> >> >> SuoNayi
>> >> >
>> >> >
>> >> >
>> >> >
>> >> >--
>> >> >*Christian Posta*
>> >> >http://www.christianposta.com/blog
>> >> >twitter: @christianposta
>> >>
>> >
>> >
>> >
>> >--
>> >*Christian Posta*
>> >http://www.christianposta.com/blog
>> >twitter: @christianposta
>>
>
>
>
>-- 
>*Christian Posta*
>http://www.christianposta.com/blog
>twitter: @christianposta

------=_Part_38286_54673929.1366247391926--