activemq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matan Zruya (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (AMQ-3885) ActiveMQ java client doesn't scale to thousands of queues
Date Sun, 17 Jun 2012 14:10:42 GMT

     [ https://issues.apache.org/jira/browse/AMQ-3885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Matan Zruya updated AMQ-3885:
-----------------------------

    Description: 
The ActiveMQ broker scales to tens of thousands of queues easily when using -Dorg.apache.activemq.UseDedicatedTaskRunner=false
(false by default).

A problem actually arises in the java client side, when a client is listening to X queues
using 1 connection and Y sessions per queue, using a JMS message listener, X * Y threads will
be created, when X * Y is not bounded,

This is because each ActiveMQConnection object holds a TaskRunnerFactory which in turn has
a ThreadPoolExecutor, the max pool size of the executor is defined to be Integer.MAX_VALUE,
with the combination of a SynchronousQueue it creates as many threads as it pleases.

Shouldn't this executor be bounded with a reasonable value ? (~200 or a constructor passed
value) and have the rest of the tasks queued,
where if more concurrency is necessary, the work could be distributed across more connections...

The ideal solution would be if multiple ActiveMQConnection objects could share a common TaskExecutor.

But i feel like the minimum would be limiting the executor so the OS doesn't halt completely
because of context switches.


  was:
The ActiveMQ broker scales to tens of thousands of queues easily when using -Dorg.apache.activemq.UseDedicatedTaskRunner=false
(false by default).

A problem actually arises in the java client side, when a client is listening to X queues
using 1 connection and Y sessions per queue, using a JMS message listener, X * Y threads will
be created, when X * Y is not bounded,

This is because each ActiveMQConnection object holds a TaskRunnerFactory which in turn has
a ThreadPoolExecutor, the max pool size of the executor is defined to be Integer.MAX_VALUE.

Shouldn't this number be bounded with a reasonable value ? (~200 or a constructor passed value)
where if more concurrency is necessary, the work could be distributed across more connections...

The ideal solution would be if multiple ActiveMQConnection objects could share a common TaskExecutor.

But i feel like the minimum would be limiting the executor so the OS doesn't halt completely
because of context switches.


    
> ActiveMQ java client doesn't scale to thousands of queues
> ---------------------------------------------------------
>
>                 Key: AMQ-3885
>                 URL: https://issues.apache.org/jira/browse/AMQ-3885
>             Project: ActiveMQ
>          Issue Type: Bug
>          Components: Connector
>    Affects Versions: 5.6.0
>         Environment: core i7 laptop running win7 64bit with 8gb of ram
>            Reporter: Matan Zruya
>              Labels: ActiveMQConnection, TaskRunnerFactory, client, java
>
> The ActiveMQ broker scales to tens of thousands of queues easily when using -Dorg.apache.activemq.UseDedicatedTaskRunner=false
(false by default).
> A problem actually arises in the java client side, when a client is listening to X queues
using 1 connection and Y sessions per queue, using a JMS message listener, X * Y threads will
be created, when X * Y is not bounded,
> This is because each ActiveMQConnection object holds a TaskRunnerFactory which in turn
has a ThreadPoolExecutor, the max pool size of the executor is defined to be Integer.MAX_VALUE,
with the combination of a SynchronousQueue it creates as many threads as it pleases.
> Shouldn't this executor be bounded with a reasonable value ? (~200 or a constructor passed
value) and have the rest of the tasks queued,
> where if more concurrency is necessary, the work could be distributed across more connections...
> The ideal solution would be if multiple ActiveMQConnection objects could share a common
TaskExecutor.
> But i feel like the minimum would be limiting the executor so the OS doesn't halt completely
because of context switches.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message