activemq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Justin Bertram <jbert...@redhat.com>
Subject Re: org.apache.activemq.artemis.core.journal.impl.JournalImpl
Date Thu, 22 Jun 2017 13:41:03 GMT
It's also worth noting that paging is a palliative measure.  The broker is
not designed to be consistently in a state where production > consumption
such that more and more messages keep accumulating.  Paging is meant to
provide temporary relief.

If you find that production > consumption more often than not then you
might consider mixing something like a database into your architecture
somewhere to store your message data.  It's a classic anti-pattern to use a
message broker as a database [1].


Justin

On Tue, Jun 6, 2017 at 3:15 AM, Денис Кирпиченков <
denis.kirpichenkov@gmail.com> wrote:

> Miroslav, thanks for advice!
> I've decreased memory usage by tunning broker.xml configuration.
> JournalImpl is not consuming a lot of memory now. But problem still exists
> -  org.apache.activemq.artemis.core.paging.impl.PaginManagerImpl has
> become
> a problem. It holds an information about all transactions in a hashmap
> org.apache.activemq.artemis.core.paging.impl.PagingManagerImpl.
> transactions.
> And when queue has, lets say 100k messages, it may cause an
> OutOfMemoryError for apps with small heap (-Xmx256m)
>
> May be there are another configuration options I'm missing?
>
>
> On Fri, Jun 2, 2017 at 2:49 PM, Miroslav Novak <mnovak@redhat.com> wrote:
>
> > Hi,
> >
> > I think you need to configure max-size-bytes in address-settings. See [1]
> > for more information. Basically you specify how much memory can every
> > queue/topic consume. Next you can set address-full-policy to BLOCK, if
> you
> > don't want to page messages to disc if max-size-bytes was exceeded. It
> will
> > hold producers until some messages are consumed.
> >
> > I would also recommend to configure consumer-window-size to 0 if your
> > consumers are slow. It will avoid buffering of messages on client side
> > message buffer so those messages can be consumed by other consumers on
> > queue which are idle.
> >
> > I hope this helps,
> > Mirek
> >
> > [1] https://github.com/apache/activemq-artemis/blob/master/
> > docs/user-manual/en/paging.md#configuration-1
> >
> > ----- Original Message -----
> > > From: "Денис Кирпиченков" <denis.kirpichenkov@gmail.com>
> > > To: dev@activemq.apache.org
> > > Sent: Friday, June 2, 2017 11:20:12 AM
> > > Subject: org.apache.activemq.artemis.core.journal.impl.JournalImpl
> > >
> > > Hello, All!
> > >
> > > I work on an application that uses artemis in embedded mode and
> sometimes
> > > (due to unexpected external coditions) the app processes any single
> > message
> > > very slow. But in any cases artemis has constantly high rate of input
> > > messages. For example, artemis receives 100-200 messages per second,
> but
> > > due to slow consumers it could deliver only 10-20 messages per second.
> > > As you can image, described conditions cause an OutOfMemoryError,
> > > eventually.
> > > After an investigation I found that ConcurrentHashMap<Long,
> > > JournalRecord> records in
> > > org.apache.activemq.artemis.core.journal.impl.JournalImpl holds a lot
> of
> > > information about messages which is waiting to be delivered.
> > >
> > > Artemis is configured to have a persisted journal, but anyway
> > > JournalImpl.records grows on any new received message as far as JVM
> heap
> > > allows.
> > >
> > > And my questions, is there any workaround? maybe configuration tweaks?
> > > How do you think is it ok to make JournalImpl.records backed by
> > > disk-storage, to limit it's heap usage.
> > >
> > > --
> > > Best regards,
> > > Denis
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message