activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tim Bain <tb...@alumni.duke.edu>
Subject Re: ]: RE: : Re: WARNING log message to be removed
Date Thu, 16 Nov 2017 05:38:55 GMT
Francois,

I think you're misunderstanding a few things, so let me try to explain more
fully and then respond to your questions.

In ActiveMQ, there are three places messages are stored.

   1. The persistence store. All persistent messages are stored here. May
   be backed by a SQL database, a NoSQL database (KahaDB or the deprecated
   LevelDB), or memory.
   2. The memory store. All non-persistent messages are stored here.
   3. The temp store. Used for overflow storage of the memory store.

If you're setting the broker's persistent property to false, then all
messages received by the broker will have their persistent flag overridden
to false, and so all messages will be stored in the memory store, and
you're correct that the storeUsage limit is irrelevant in that case.
However, that has absolutely nothing to do with using a
MemoryPersistenceAdapter, which is how you choose to store persistent
messages in memory. (Not in the memory store, in memory, in a series of
HashMaps. MemoryPersistenceAdapter != memory store.) I'm not aware of any
reason to both set the broker's persistent property to false and also use a
MemoryPersistenceAdapter, since the former disables any possibility of use
of the latter.

If you're using a MemoryPersistenceAdapter, then all three limits are used.
The storeUsage controls how many persistent messages are stored in the
memory-backed persistence store, the memoryUsage controls how many messages
are stored in the memory store, and the tempUsage controls how many
messages are stored in the temp store. All three of them ALSO use the
"physical limit" in that if you try to put more messages into any of the
stores than your physical RAM or disk can hold, the broker will do awful
things and may crash. The point of the limits is to prevent this from
happening by making the broker refuse to store messages at a software level
rather than allowing it to result in a hardware failure that might kill the
broker process, but you're in charge of setting those limits appropriately
and if you screw it up, the broker can't prevent you from shooting your own
foot. Caveat emptor.

No, the broker doesn't compute any limits at all for you; setting no limit
for any of the three effectively means that the software limit is infinite
(no limit), so you'll hit the "physical limit" (with all the negatives that
come with that) if anything. It's your responsibility to protect your
broker process from doing that, and if you choose not to, that's on you.

You've probably realized this already from what I wrote above, but to your
final question, no, if you use a MemoryPersistenceAdapter (without setting
the broker's persistent property to false, presumably, otherwise what's the
point?), you control everything by setting all three limits, not just
memoryUsage and tempUsage.

Tim

On Mon, Nov 13, 2017 at 2:37 PM, COURTAULT Francois <
Francois.Courtault@gemalto.com> wrote:

> Hello Tim,
>
> Try to summarize:
>        - If we use a MemoryPersistenceAdapter that is to say that the
> Broker persistent property is set to false, only the memoryUsage and
> tempUsage limit  could be set.
>           The storeUsage limit is useless in this case, right ?
>        - If we use a MemoryPersistenceAdapter  and if one of the limits,
> either the memoryUsage or the tempUsage, is used, the other will use the
> physical limit.
>           That is to say:
>                    * if we only set the memoryUsage limit then the
> tempUsage limit will be the physical available disk space.
>                    * if we only set the tempUsage limit then the
> memoryUsage limit will be the physical available memory (maybe limited by
> the Xmx JVM settings).
>        - If we use a MemoryPersistenceAdapter and none of the limits are
> set (neither memoryUsage nor the tempUsage  limits) then the Broker will
> compute these limits
>           using some formula, right ?
>        - If we use a MemoryPersistenceAdapter, the way to control
> everything is to set the memoryUsage and tempUsage  limits, right ?
>
> Best Regards.
>
> -----Original Message-----
> From: tbain98@gmail.com [mailto:tbain98@gmail.com] On Behalf Of Tim Bain
> Sent: jeudi 9 novembre 2017 06:17
> To: ActiveMQ Users <users@activemq.apache.org>
> Subject: Re: [++SPAM++]: RE: : Re: WARNING log message to be removed
>
> On Wed, Nov 8, 2017 at 7:29 AM, COURTAULT Francois <
> Francois.Courtault@gemalto.com> wrote:
>
> > Hello Tim,
> >
> > Understood for the temp store: it looks like a swap mechanism in case
> > there is no more room in memory, right ?
> >
>
> Just to clarify: it's a swap mechanism in case there is no more room in
> *the memory store*, not in memory overall. This is not for paging generic
> heap content out to disk if the JVM runs out of heap, etc., and I believe
> it's also not for paging out content if you overfill a persistence store
> that happens to be backed by memory (though I've not investigated that
> specific question). But yes, you understand the overall concept.
>
>
> > Let's say that, in our configuration, we only set the memoryUsage
> > limit  without specifying any storeUsage and tempUsage limit .
> > In such case, does it exist a formula to calculate the tempUsage limit
> > ( eg tempUsage limit=f(memoryUsage limit) )?
> >
>
> In that case, no limit is applied, so you're allowed to write content to
> disk right up to the point where the write fails (maybe spectacularly)
> because there is no more available space on the volume.
>
>
> > I ask you the question, because if we only set the memoryUsage limit
> > to
> > 128 mb (no storeUsage and tempUsage limits are set), the WARNING
> > message disappears with the same available space on the disk.
> >
>
> Yes, the warning message disappears because you've taken the safety off
> and told the broker that you're on your own, so it allows you to shoot
> yourself in the foot if you so choose without trying to help you avoid
> doing so. The net effect for you is the same (the broker won't prevent you
> from filling the disk in either configuration), but in one case it tells
> you that there's a potential for doing that, and in the other it doesn't
> attempt to warn you.
>
> Tim
> ________________________________
>  This message and any attachments are intended solely for the addressees
> and may contain confidential information. Any unauthorized use or
> disclosure, either whole or partial, is prohibited.
> E-mails are susceptible to alteration. Our company shall not be liable for
> the message if altered, changed or falsified. If you are not the intended
> recipient of this message, please delete it and notify the sender.
> Although all reasonable efforts have been made to keep this transmission
> free from viruses, the sender will not be liable for damages caused by a
> transmitted virus.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message