activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From COURTAULT Francois <Francois.Courta...@gemalto.com>
Subject RE: ]: RE: : Re: WARNING log message to be removed
Date Thu, 16 Nov 2017 14:37:05 GMT
Hello Tim,

Sorry to come back to you.
I think I have understood the purpose of the 3 stores (persistent, memory and temp).

But, I remind you that I used TomEE 7.0.4 plus flavor with ActiveMQ 5.14.5 with the following
settings:
- no activemq.xml file
- BrokerXmlConfig =  broker:(tcp://0.0.0.0:61616)?useJmx=false&amp;persistent=false

In the TomEE server statup log, I was able to see:
01-Nov-2017 14:16:38.142 INFO [ActiveMQFactory start and checkpoint] org.apache.activemq.broker.BrokerService.doStartPersistenceAdapter
 Using Persistence Adapter: MemoryPersistenceAdapter

So, my understanding is that by having set  the persistence property to false, ActiveMQ runtime
set the PersistenceAdapter to a MemoryPersistenceAdapter: is my understanding right or wrong
(eg persistence=false => set the PersistenceAdapter to a MemoryPersistenceAdapter) ?
In such case, the persistence storage is not used, right ?

As I don't have any activemq.xml: no value has been set for memory and temp store limits.
Nevertheless, I have 2 WARNING messages:
01-Nov-2017 14:16:38.140 WARNING [ActiveMQFactory start and checkpoint] org.apache.activemq.broker.BrokerService.checkMemorySystemUsageLimit
s Memory Usage for the Broker (1024mb) is more than the maximum available for the JVM: 726
mb - resetting to 70% of maximum available: 508 mb
....
01-Nov-2017 14:16:38.360 WARNING [ActiveMQFactory start and checkpoint] org.apache.activemq.broker.BrokerService.checkUsageLimit
Temporary Store limit is 51200 mb (current store usage is 0 mb). The data directory: /opt/gemalto/bin
only has 16571 mb of usable space. - resetting to maximum available disk space: 16571 mb

Our goal is to remove these WARNING messages.

If we use an activemq.xml file with the following content:
         <systemUsage>
             <systemUsage sendFailIfNoSpace="true">
                 <memoryUsage>
                     <memoryUsage limit="128 mb"/>
                 </memoryUsage>
             </systemUsage>
         </systemUsage>
The 2 WARNING messages disappears but the drawback of this solution is that we have to customize
TomEE  for that (add some spring jars) and we want to avoid that.
So, according to this test (usage of actvemq.xml and setting only the memoryUsage limit),
we are able to remove these WARNING messages. There is a TomEE jira already submitted for
setting a memoryUsage limit in the BrokerXmlConfig.

This is why my understanding is the following:
     - setting the persistence property to false, activemq runtime set the PersistenceAdapter
to a MemoryPersistenceAdapter, right ?
     - having the persistence set to false:
                     * if no store limit is used (eg no activemq.xml), the activemq runtime
will compute these limits thank to some checking for Memory usage and Temporary usage (as
we can see in the WARNING logs), right ?
                     * if we set only a Memory usage limit as we did by using the above activemq.xml,
then the tempUsage limit is set to unlimited, right ?
                     * in a symmetric way, if we set only a tempUsage limit, then the memoryUsage
limit is set to unlimited, right ?

Best Regards.

-----Original Message-----
From: tbain98@gmail.com [mailto:tbain98@gmail.com] On Behalf Of Tim Bain
Sent: jeudi 16 novembre 2017 06:39
To: ActiveMQ Users <users@activemq.apache.org>
Subject: Re: ]: RE: : Re: WARNING log message to be removed

Francois,

I think you're misunderstanding a few things, so let me try to explain more fully and then
respond to your questions.

In ActiveMQ, there are three places messages are stored.

   1. The persistence store. All persistent messages are stored here. May
   be backed by a SQL database, a NoSQL database (KahaDB or the deprecated
   LevelDB), or memory.
   2. The memory store. All non-persistent messages are stored here.
   3. The temp store. Used for overflow storage of the memory store.

If you're setting the broker's persistent property to false, then all messages received by
the broker will have their persistent flag overridden to false, and so all messages will be
stored in the memory store, and you're correct that the storeUsage limit is irrelevant in
that case.
However, that has absolutely nothing to do with using a MemoryPersistenceAdapter, which is
how you choose to store persistent messages in memory. (Not in the memory store, in memory,
in a series of HashMaps. MemoryPersistenceAdapter != memory store.) I'm not aware of any reason
to both set the broker's persistent property to false and also use a MemoryPersistenceAdapter,
since the former disables any possibility of use of the latter.

If you're using a MemoryPersistenceAdapter, then all three limits are used.
The storeUsage controls how many persistent messages are stored in the memory-backed persistence
store, the memoryUsage controls how many messages are stored in the memory store, and the
tempUsage controls how many messages are stored in the temp store. All three of them ALSO
use the "physical limit" in that if you try to put more messages into any of the stores than
your physical RAM or disk can hold, the broker will do awful things and may crash. The point
of the limits is to prevent this from happening by making the broker refuse to store messages
at a software level rather than allowing it to result in a hardware failure that might kill
the broker process, but you're in charge of setting those limits appropriately and if you
screw it up, the broker can't prevent you from shooting your own foot. Caveat emptor.

No, the broker doesn't compute any limits at all for you; setting no limit for any of the
three effectively means that the software limit is infinite (no limit), so you'll hit the
"physical limit" (with all the negatives that come with that) if anything. It's your responsibility
to protect your broker process from doing that, and if you choose not to, that's on you.

You've probably realized this already from what I wrote above, but to your final question,
no, if you use a MemoryPersistenceAdapter (without setting the broker's persistent property
to false, presumably, otherwise what's the point?), you control everything by setting all
three limits, not just memoryUsage and tempUsage.

Tim

On Mon, Nov 13, 2017 at 2:37 PM, COURTAULT Francois < Francois.Courtault@gemalto.com>
wrote:

> Hello Tim,
>
> Try to summarize:
>        - If we use a MemoryPersistenceAdapter that is to say that the
> Broker persistent property is set to false, only the memoryUsage and
> tempUsage limit  could be set.
>           The storeUsage limit is useless in this case, right ?
>        - If we use a MemoryPersistenceAdapter  and if one of the
> limits, either the memoryUsage or the tempUsage, is used, the other
> will use the physical limit.
>           That is to say:
>                    * if we only set the memoryUsage limit then the
> tempUsage limit will be the physical available disk space.
>                    * if we only set the tempUsage limit then the
> memoryUsage limit will be the physical available memory (maybe limited
> by the Xmx JVM settings).
>        - If we use a MemoryPersistenceAdapter and none of the limits
> are set (neither memoryUsage nor the tempUsage  limits) then the
> Broker will compute these limits
>           using some formula, right ?
>        - If we use a MemoryPersistenceAdapter, the way to control
> everything is to set the memoryUsage and tempUsage  limits, right ?
>
> Best Regards.
>
> -----Original Message-----
> From: tbain98@gmail.com [mailto:tbain98@gmail.com] On Behalf Of Tim
> Bain
> Sent: jeudi 9 novembre 2017 06:17
> To: ActiveMQ Users <users@activemq.apache.org>
> Subject: Re: [++SPAM++]: RE: : Re: WARNING log message to be removed
>
> On Wed, Nov 8, 2017 at 7:29 AM, COURTAULT Francois <
> Francois.Courtault@gemalto.com> wrote:
>
> > Hello Tim,
> >
> > Understood for the temp store: it looks like a swap mechanism in
> > case there is no more room in memory, right ?
> >
>
> Just to clarify: it's a swap mechanism in case there is no more room
> in *the memory store*, not in memory overall. This is not for paging
> generic heap content out to disk if the JVM runs out of heap, etc.,
> and I believe it's also not for paging out content if you overfill a
> persistence store that happens to be backed by memory (though I've not
> investigated that specific question). But yes, you understand the overall concept.
>
>
> > Let's say that, in our configuration, we only set the memoryUsage
> > limit  without specifying any storeUsage and tempUsage limit .
> > In such case, does it exist a formula to calculate the tempUsage
> > limit ( eg tempUsage limit=f(memoryUsage limit) )?
> >
>
> In that case, no limit is applied, so you're allowed to write content
> to disk right up to the point where the write fails (maybe
> spectacularly) because there is no more available space on the volume.
>
>
> > I ask you the question, because if we only set the memoryUsage limit
> > to
> > 128 mb (no storeUsage and tempUsage limits are set), the WARNING
> > message disappears with the same available space on the disk.
> >
>
> Yes, the warning message disappears because you've taken the safety
> off and told the broker that you're on your own, so it allows you to
> shoot yourself in the foot if you so choose without trying to help you
> avoid doing so. The net effect for you is the same (the broker won't
> prevent you from filling the disk in either configuration), but in one
> case it tells you that there's a potential for doing that, and in the
> other it doesn't attempt to warn you.
>
> Tim
> ________________________________
>  This message and any attachments are intended solely for the
> addressees and may contain confidential information. Any unauthorized
> use or disclosure, either whole or partial, is prohibited.
> E-mails are susceptible to alteration. Our company shall not be liable
> for the message if altered, changed or falsified. If you are not the
> intended recipient of this message, please delete it and notify the sender.
> Although all reasonable efforts have been made to keep this
> transmission free from viruses, the sender will not be liable for
> damages caused by a transmitted virus.
>
________________________________
 This message and any attachments are intended solely for the addressees and may contain confidential
information. Any unauthorized use or disclosure, either whole or partial, is prohibited.
E-mails are susceptible to alteration. Our company shall not be liable for the message if
altered, changed or falsified. If you are not the intended recipient of this message, please
delete it and notify the sender.
Although all reasonable efforts have been made to keep this transmission free from viruses,
the sender will not be liable for damages caused by a transmitted virus.
Mime
View raw message