activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tim Bain <tb...@alumni.duke.edu>
Subject Re: ]: RE: : Re: WARNING log message to be removed
Date Fri, 17 Nov 2017 06:26:38 GMT
Francois,

I have two apologize, I've misled you on two fronts.

First, I had wrongly assumed that setting persistent=false on the broker
resulted in setting persistent=false on the incoming messages, but based on
the information in your logs I took a look into the code (specifically
BrokerService.createPersistenceAdapter()) and I see that it's as you said:
when persistent=false, the broker creates a MemoryPersistenceAdapter in
place of a persistence adapter that's backed by KahaDB, LevelDB, or a SQL
database. So on that point I stand corrected, and I apologize for
misleading you.

Second, said that if no value was set, no limits were applied. I believe
that's true if you're explicitly configuring a <systemUsage> element via an
XML config file (or some other means). However, digging through the code
(specifically BrokerService.getSystemUsage()) I see that if no system usage
is explicitly set, default values are set for all three store types I
listed plus also the scheduled job store (which I forgot about, but which
isn't relevant to your questions). The default values used are 1 GB for the
memory store, 50 GB for the temp store, and 100 GB for the persistence
store), which is why your config-less broker is using those values for the
memory and temp stores. There is no output for the persistence store
(despite the fact that 100 GB is far more RAM than you've allocated to your
JVM's heap) because MemoryPersistenceAdapter.size() is hard-coded to return
0, which means the memory-backed persistence store always thinks it's empty
even when it's not (so there's nothing to protect you from running out of
memory if you try to store too many persistent messages). So for your
specific configuration, which I appreciate you clarifying, I think that
default limits are in fact being applied.

With that said, let me reply inline to the various questions and statements
you made.

On Thu, Nov 16, 2017 at 7:37 AM, COURTAULT Francois <
Francois.Courtault@gemalto.com> wrote:

> Hello Tim,
>
> Sorry to come back to you.
> I think I have understood the purpose of the 3 stores (persistent, memory
> and temp).
>
> But, I remind you that I used TomEE 7.0.4 plus flavor with ActiveMQ 5.14.5
> with the following settings:
> - no activemq.xml file
> - BrokerXmlConfig =  broker:(tcp://0.0.0.0:61616)?
> useJmx=false&amp;persistent=false
>
> In the TomEE server statup log, I was able to see:
> 01-Nov-2017 14:16:38.142 INFO [ActiveMQFactory start and checkpoint]
> org.apache.activemq.broker.BrokerService.doStartPersistenceAdapter  Using
> Persistence Adapter: MemoryPersistenceAdapter
>
> So, my understanding is that by having set  the persistence property to
> false, ActiveMQ runtime set the PersistenceAdapter to a
> MemoryPersistenceAdapter: is my understanding right or wrong (eg
> persistence=false => set the PersistenceAdapter to a
> MemoryPersistenceAdapter) ?
>

Yes, this is correct (despite my earlier statements that it was not).


> In such case, the persistence storage is not used, right ?
>

This is incorrect. The persistence store is used, it just happens to be
backed by memory and not a SQL database or a KahaDB or LevelDB engine. This
is the main point I was trying to highlight in my last message: just
because the persistence store is using memory as the place its messages are
written to doesn't mean that it *is* the memory store. It's still the
persistence store and it still holds only persistent messages, it's just a
memory-backed flavor of persistence store. The memory store is still
separate and unrelated (though the two will need to co-exist in the
available heap of the JVM).


> As I don't have any activemq.xml: no value has been set for memory and
> temp store limits.
> Nevertheless, I have 2 WARNING messages:
> 01-Nov-2017 14:16:38.140 WARNING [ActiveMQFactory start and checkpoint]
> org.apache.activemq.broker.BrokerService.checkMemorySystemUsageLimit s
> Memory Usage for the Broker (1024mb) is more than the maximum available for
> the JVM: 726 mb - resetting to 70% of maximum available: 508 mb
> ....
> 01-Nov-2017 14:16:38.360 WARNING [ActiveMQFactory start and checkpoint]
> org.apache.activemq.broker.BrokerService.checkUsageLimit Temporary Store
> limit is 51200 mb (current store usage is 0 mb). The data directory:
> /opt/gemalto/bin only has 16571 mb of usable space. - resetting to maximum
> available disk space: 16571 mb
>

The first portion of this email explains how the limits from those two log
lines are chosen and why there's not a third line for the persistence
store, so I won't re-hash that here.


> Our goal is to remove these WARNING messages.
>

To play devil's advocate: why? If your only goal is to remove the warning
messages, who cares? You already know that these messages are telling you
about something that you don't seem to consider a problem (i.e. that the
store limits are set higher than the physical capacity of the physical
resources that back them, and therefore it's possible to attempt to
overfill those physical resources with possibly fatal results), so just
ignore them, or view the log files through appropriate grep -v commands.
Why is it so critical that you have a log file that doesn't have any
WARNING lines? To be clear, I'd always prefer to have clean logs, but you
seem unwilling to use the one solution that's available to you (customizing
TomEE to provide a customer activemq.xml file) and the other one that would
be cleaner (setting a memoryUsage limit via the BrokerXmlConfig) isn't yet
available to you, so maybe just living with the current behavior is the
option you'd find least objectionable.


> If we use an activemq.xml file with the following content:
>          <systemUsage>
>              <systemUsage sendFailIfNoSpace="true">
>                  <memoryUsage>
>                      <memoryUsage limit="128 mb"/>
>                  </memoryUsage>
>              </systemUsage>
>          </systemUsage>
> The 2 WARNING messages disappears but the drawback of this solution is
> that we have to customize TomEE  for that (add some spring jars) and we
> want to avoid that.
>

To play devil's advocate again, why? Presumably you already are
"customizing" TomEE to provide it whatever WAR you want it to run; why's it
unacceptable to customize it by providing an additional config file? Also,
it might be possible to use ActiveMQ's xbean:classPathResource syntax (see
http://activemq.apache.org/broker-xbean-uri.html) to reference an
activemq.xml file that's embedded within your WAR, which would mean you're
not even customizing TomEE at all (beyond your WAR file). But I have no
experience with TomEE nor with how it interacts with ActiveMQ, so you'd
have to test to see if that actually works.


> So, according to this test (usage of actvemq.xml and setting only the
> memoryUsage limit), we are able to remove these WARNING messages. There is
> a TomEE jira already submitted for setting a memoryUsage limit in the
> BrokerXmlConfig.
>

For that JIRA to be implemented, there would have to be a change the
ActiveMQ code, specifically the code in
DefaultBrokerFactory.createBroker(), to allow creation of a complex object
(SystemUsage) from a URI that is inherently flat so it can be passed to the
BrokerService.setSystemUsage() method. That's not impossible, but there's
not a natural syntax for representing how to do it (not that I can see,
anyway), so unless you have a proposal for how to do that cleanly, the odds
are good that you're not going to get this feature implemented. So while
this would be the cleanest solution, I wouldn't advise you to hold out too
much hope for this one.


> This is why my understanding is the following:
>      - setting the persistence property to false, activemq runtime set the
> PersistenceAdapter to a MemoryPersistenceAdapter, right ?
>

Correct.


>      - having the persistence set to false:
>                      * if no store limit is used (eg no activemq.xml), the
> activemq runtime will compute these limits thank to some checking for
> Memory usage and Temporary usage (as we can see in the WARNING logs), right
> ?
>

Not quite. If no limits are set on any of the stores, then default values
will be used. Those default values are hard-coded and make no attempt to
account for the amount of JVM heap or disk space available, but if the
default values are larger than those physical limits then you'll see those
WARNINGs in the logs and the broker will not attempt to prevent you from
overrunning the physical limits of your hardware.


>                      * if we set only a Memory usage limit as we did by
> using the above activemq.xml, then the tempUsage limit is set to unlimited,
> right ?
>

Correct.


>                      * in a symmetric way, if we set only a tempUsage
> limit, then the memoryUsage limit is set to unlimited, right ?
>

Correct.


> Best Regards.
>

I hope this helps. I recognize that the ideal option (being able to set
these values with the "broker:" syntax) isn't currently available and that
there are some drawbacks to the other options, but those drawbacks don't
seem all that terrible based on what I know at the moment. And if you're
lucky, you'll be able to use the xbean:classPathResource approach to get
all the flexibility of the XML-based syntax without having to do any
additional configuration.

Tim

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message