activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gaurav Sharma <gaurav.cs.sha...@gmail.com>
Subject Re: 100% CPU
Date Wed, 03 Apr 2013 16:21:42 GMT
So, here're a few of my observations and suggestions based on the provided info:
1. use a newer java than v1.7.0_03
2. switch to amq v5.7 or v5.8
3. keep xmx and xms the same and preferably a multiple of 512 megs
4. where are you getting server grade machines with 2 and 15 gig ram
5. when you notice 100% util, take a few spaced apart thread dumps and post them on here
6. on a 4 core machine, load should not just be 100% but higher than 1. can you post 'top'
output as well
7. use the amq pool for producers, sessions, connections where possible
8. test your existing config and load without kahadb (non-persistent mode) and compare results


On Apr 3, 2013, at 7:09, nielsbaloe <niels@geoxplore.nl> wrote:

Hi,

-Xms512m -Xmx1500m

but before we've set anything, it went wrong as well, and setting it
higher also doesn't help. Note that there are only a few messages going
through every hour, this is not yet fully production, so it should not
even hit the 1% continuously...

Thanks!,
Niels

> 
> Niels,
> 
> I am not one of the experts here, but I am a new user of ActiveMQ via
> TomEE, and I like to listen in on the mail topics/questions/responses
> here.
> 
> Since I listen in on Tomcat user list as well, I would say that this
> sounds
> like a GC (garbage collection) issue, but I might be mistaking.
> 
> Can you reply with the java options of your app/container? What is your
> java options for the Windows 7 laptops and the Ubuntu Linux server/machine
> that has 12GB and 2GB?
> 
> I assume you have smaller memory settings on your Windows 7 (developer)
> laptops and probably larger memory settings on the Ubuntu Linux
> server/machine. Right? If yes, then GC may be the reason why you are
> always
> experiencing 100% CPU.
> 
> This is just a guess/hunch, but since you provided such a detailed
> question, please do not leave out the java (memory) options on your
> Windows
> 7 laptops as well as Ubuntu Linux server(s).
> 
> Howard
> 
> 
> On Wed, Apr 3, 2013 at 8:51 AM, nielsbaloe <niels@geoxplore.nl> wrote:
> 
>> Hi all,
>> 
>> We are using activeMQ successfully for two projects now, but we
>> accidentely
>> discovered that both the broker and the worker/consumer machines are
>> hitting
>> 100% CPU continuously. We do not have this issue on our developers
>> machines
>> (all Windows 7 laptops). This occurs even when no events are being
>> processed.
>> 
>> I couldn't find any clues for this issue, except setting the prefetch
>> size.
>> I've set the prefetch size to 10, as we have 10 consumers at the
>> worker/consumer machine. We have a broker machine and a worker/consumer
>> machine, which are both configured like below. In the near future we
>> will
>> add more worker/consumer machines.
>> 
>> OS: Ubuntu Linux 12.04 (headless)
>> Memory: 12GB and 2GB
>> CPU: Intel Xeon 3.06GHz 4core
>> Java: "1.7.0_03", OpenJDK Runtime Environment (IcedTea7 2.1.1pre)
>> (7~u3-2.1.1~pre1-1ubuntu3)
>> Webcontainer: none, java-standalone
>> ActiveMQ: 5.6.0
>> 
>> The broker uses the internal KahaDB database.
>> 
>> We are using one queue, to which the worker/consumer machine is
>> listening
>> and posting to, say about 100 messages a day. We also use about 4
>> scheduled
>> messages for every 'modem' (our internal subject) which results in about
>> 40
>> scheduled messages or so which generates an event once every 30 minuts.
>> Nothing spectacular so to say.
>> 
>> Thanks for any clues in advance. For completeness, I will post our
>> Broker,
>> Consumer and Producer code (without comments), this might show any wrong
>> assumptions on our side.
>> 
>> Best,
>> Niels Baloe
>> 
>> 
>>> -----------------------
>> 
>> public class Consumer implements ExceptionListener {
>> 
>>        private Session session;
>>        private MessageConsumer messageConsumer;
>>        private static Logger LOG =
>> Logger.getLogger(Consumer.class.getName());
>> 
>>        public Consumer(String brokerServer, String queueName,
>>                        MessageListener messageListener) throws
>> JMSException,
>>                        FileNotFoundException, IOException {
>>                this(Broker.getSession(brokerServer), queueName,
>> messageListener);
>>        }
>> 
>>        public Consumer(Session session, String queueName,
>>                        MessageListener messageListener) throws
>> JMSException {
>>                this.session = session;
>> 
>>                Queue queue = session.createQueue(queueName);
>>                messageConsumer = session.createConsumer(queue);
>>                messageConsumer.setMessageListener(messageListener);
>>        }
>> 
>>        public void close() {
>>                try {
>>                        messageConsumer.close();
>>                } catch (JMSException e) {
>>                }
>>                try {
>>                        session.close();
>>                } catch (JMSException e) {
>>                }
>>        }
>> 
>>        @Override
>>        public void onException(JMSException je) {
>>                LOG.log(Level.SEVERE, je.getMessage(), je);
>>        }
>> 
>> }
>> 
>> public class Producer {
>> 
>>        private Session session;
>>        private MessageProducer producer;
>>        public Producer(String brokerUrl, String queue) throws
>> JMSException,
>>                        FileNotFoundException, IOException {
>>                this(Broker.getSession(brokerUrl), queue);
>>        }
>> 
>>        public Producer(Session session, String queue) throws
>> JMSException
>> {
>>                this.session = session;
>>                Destination destination = session.createQueue(queue);
>>                producer = session.createProducer(destination);
>>                producer.setDeliveryMode(DeliveryMode.PERSISTENT);
>>        }
>> 
>>        public void close() {
>>                try {
>>                        producer.close();
>>                } catch (JMSException e) {
>>                }
>>                try {
>>                        session.close();
>>                } catch (JMSException je) {
>>                }
>>        }
>> 
>>        public Message getMessageText(String text) throws JMSException {
>>                return session.createTextMessage(text);
>>        }
>> 
>>        public Message getMessageObject() throws JMSException {
>>                return session.createObjectMessage();
>>        }
>> 
>>        public void send(Message message) throws JMSException {
>>                producer.send(message);
>>        }
>> 
>>        public void sendScheduled(Message message, String cron) throws
>> JMSException
>> {
>> 
>> message.setStringProperty(ScheduledMessage.AMQ_SCHEDULED_CRON, cron);
>>                producer.send(message);
>>        }
>> 
>> }
>> 
>> public class Broker {
>> 
>>        private BrokerService broker;
>> 
>>        public Broker(String host, int port, String brokerName) throws
>> Exception {
>>                broker = new BrokerService();
>>                broker.setUseJmx(true);
>>                broker.setBrokerName(brokerName);
>>                broker.addConnector("tcp://" + host + ":" + port);
>>                broker.setSchedulerSupport(true);
>>                broker.start();
>>        }
>> 
>>        public URI getNameTCP() {
>>                return broker.getVmConnectorURI();
>>        }
>> 
>>        public void close() {
>>                try {
>>                        broker.stop();
>>                        broker.waitUntilStopped();
>>                } catch (Exception e) {
>>                }
>>        }
>> 
>>        public static void closeConnection() {
>>                if (connection != null) {
>>                        try {
>>                                connection.close();
>>                        } catch (JMSException e) {
>>                        }
>>                }
>>        }
>> 
>>        private static Connection connection;
>> 
>>        private static Session getSessionWithoutRetry(String
>> brokerServer)
>>                        throws JMSException, FileNotFoundException,
>> IOException {
>>                if (connection == null) { // does not work when broker
>> is
>> local
>>                        ActiveMQConnectionFactory connectionFactory =
>> new
>> ActiveMQConnectionFactory(
>>                                        brokerServer);
>>                        connectionFactory.setAlwaysSessionAsync(true);
>> 
>>                        // Prefetch size
>>                        String prefetch =
>> NoImportUtils.getSettings().getProperty(
>>                                        "broker.prefetchSize");
>> 
>>                        connectionFactory.getPrefetchPolicy().setAll(
>>                                        Integer.parseInt(prefetch));
>>                        connection =
>> connectionFactory.createConnection();
>>                        connection.start();
>>                }
>>                return connection.createSession(false,
>> Session.AUTO_ACKNOWLEDGE);
>>        }
>> 
>>        public static Session getSession(String brokerServer) throws
>> JMSException,
>>                        FileNotFoundException, IOException {
>>                try {
>>                        return getSessionWithoutRetry(brokerServer);
>>                } catch (ConnectionFailedException e) {
>>                        // Retry once when connection failed
>>                        closeConnection();
>>                        return getSessionWithoutRetry(brokerServer);
>>                }
>>        }
>> 
>> }
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> --
>> View this message in context:
>> http://activemq.2283324.n4.nabble.com/100-CPU-tp4665414.html
>> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
> 
> 
> 
> 
> _______________________________________________
> If you reply to this email, your message will be added to the discussion
> below:
> http://activemq.2283324.n4.nabble.com/100-CPU-tp4665414p4665415.html
> 
> To unsubscribe from 100% CPU, visit
> http://activemq.2283324.n4.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=4665414&code=bmllbHNAZ2VveHBsb3JlLm5sfDQ2NjU0MTR8NTk0NzAwMTI4






--
View this message in context: http://activemq.2283324.n4.nabble.com/100-CPU-tp4665414p4665420.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Mime
View raw message