activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hellweek <bwins...@tfutures.com>
Subject Re: ActiveMQ thoughts
Date Wed, 12 Dec 2007 13:42:10 GMT

The Test programs I provideded will demonstrate the issue.  To be fair and
honest the issue seems to be an issue with using the C# API and C++ API.  

Get the three test programs and you will see the the issue very clear.


Roger Hoover wrote:
> 
> Can anyone confirm or deny the second issue mentioned (what seems like the
> slow consumer problem)?
> 
> "2. A Producer that producers more then 200 messages per sec locks up the
> Broker when the Broker has only one client connected.  This one was the
> biggest issue to accept and the one issue that caused us to say ActiveMQ
> is
> not ready for a production environment.  The most basic and simple task of
> the Message Broker is not working as expected and makes the ActiveMQ
> unusable in a environment where peak message Generation can exceed 200
> messages per second.  To be honest we never even get close to 100 messages
> as it seems we die after 50 messages are fired in the same second.  The
> only
> time I am able to have producers producing without locking up or crashing
> is
> if I don't have any consumers listening.  Having a messaging system that
> works without consumers is not a valid solution."
> 
> On 12/11/07, Nathan Mittler <nathan.mittler@gmail.com> wrote:
>>
>> Appreciate your feedback and helping to identify this problem!  I've
>> captured this in a JIRA issue here:
>>
>> https://issues.apache.org/activemq/browse/AMQCPP-157
>>
>> We'll do our best to get this resolved soon!
>>
>> Regards,
>> Nate
>>
>> On Dec 11, 2007, at 9:10 AM, Hellweek wrote:
>>
>> >
>> > As promised I have created a c++ test program (TestProducerBug) that
>> > will
>> > create up to X producers. The class that does the work is
>> > (TestProducers.cpp).
>> >
>> > I am created a C# test program (TestConsumerBugCSharp) that will
>> > create up
>> > to X consumers using a MessageListener.  The class that does the
>> > work is
>> > (TestConsumers.cs).
>> >
>> > I have created a C++ test program (TestConsumerBug) that will create
>> > up to X
>> > consumers.  The class that does the work(TestConsumers.cpp).
>> >
>> > Here is some information on my setup.
>> >
>> > Compiler MS 2005.
>> >
>> > ActiveMQ
>> > Running ActiveMQ 5.0 Dated Dec 7th 2007.  It is running on windows
>> > 2003
>> > Server 64 Bit.
>> > Running Java 1.6.0_02 this version of Java is 64 bit. (Problem
>> > happens even
>> > on a 32 bit version of JAVA).
>> >
>> > ActiveMQ Settings
>> > Broker Settings (persistent="false" advisorySupport="false")
>> >
>> > Topic Policy
>> > <policyEntry topic="Test.>" producerFlowControl="true">
>> >
>> >            <!-- lets force old messages to be discarded for slow
>> > consumers
>> > -->
>> >            <pendingMessageLimitStrategy>
>> >              <constantPendingMessageLimitStrategy limit="5"/>
>> >            </pendingMessageLimitStrategy>
>> >  <messageEvictionStrategy>
>> >  <oldestMessageEvictionStrategy />
>> >  </messageEvictionStrategy>
>> >
>> >          </policyEntry>
>> >
>> >
>> > Client API's
>> >
>> > CPP activemq-cpp-2.1.2-src
>> > C# ApacheActiveMQ (Not sure the version but latest trunk).
>> >
>> >
>> > When running these test remember to stop and restart the broker each
>> > test as
>> > the test can and will cause the broker to hang.
>> >
>> > Tests 1 -3 will show what is happening between the CPP and C# API.
>> >
>> > Test 4 will show what happens to a producer when a consumer is in a
>> > break
>> > point in the MessageListener.
>> >
>> > Test 1
>> > To recreate the issue build and run
>> > TestProducerBug
>> > TestConsumerBugCSharp.
>> >
>> > If you set the number of producers and clients to 10 you should see
>> > the
>> > problem happen in less then 5 min (About 2,000 messages per consumer).
>> > The producer will throw an exception place a breakpoint on the catch
>> > block
>> > in the ThreadProc.  you will see the following information.
>> > No valid response received for command: Begin Class =
>> > ActiveMQBytesMessage
>> >
>> > Begin Class = ActiveMQMessageBase
>> >
>> > Value of ackHandler = 00000000
>> >
>> > Value of redeliveryCount = 0
>> >
>> > Value of properties = Begin Class PrimitiveMap:
>> >
>> > Begin Class PrimitiveMap:
>> >
>> > Begin Class = Message
>> >
>> > Value of Message::ID_MESSAGE = 0
>> >
>> > Value of ProducerId is Below:
>> >
>> > Begin Class = ProducerId
>> >
>> > Value of ProducerId::ID_PRODUCERID = 123
>> >
>> > Value of ConnectionId = 752afa01-c256-45c2-84ad-c74b0578f199
>> >
>> > Value of Value = 19
>> >
>> > Value of SessionId = 0
>> >
>> > No Data for Class BaseDataStructure
>> >
>> > End Class = ProducerId
>> >
>> > Value of Destination is Below:
>> >
>> > Begin Class = ActiveMQTopic
>> >
>> > Begin Class = ActiveMQDestination
>> >
>> > Value of exclusive = false
>> >
>> > Value of ordered = false
>> >
>> > Value of advisory = false
>> >
>> > Value of orderedTarget = coordinator
>> >
>> > Value of physicalName = Test.20
>> >
>> > Value of options = Begin Class activemq::util::Properties:
>> >
>> > End Class activemq::util::Properties:
>> >
>> > No Data for Class BaseDataStructure
>> >
>> > End Class = ActiveMQDestination
>> >
>> > End Class = ActiveMQTopic
>> >
>> > Value of TransactionId is Below:
>> >
>> > Object is NULL
>> >
>> > Value of OriginalDestination is Below:
>> >
>> > Object is NULL
>> >
>> > Value of MessageId is Below:
>> >
>> > Begin Class = MessageId
>> >
>> > Value of MessageId::ID_MESSAGEID = 110
>> >
>> > Value of ProducerId is Below:
>> >
>> > Begin Class = ProducerId
>> >
>> > Value of ProducerId::ID_PRODUCERID = 123
>> >
>> > Value of ConnectionId = 752afa01-c256-45c2-84ad-c74b0578f199
>> >
>> > Value of Value = 19
>> >
>> > Value of SessionId = 0
>> >
>> > No Data for Class BaseDataStructure
>> >
>> > End Class = ProducerId
>> >
>> > Value of ProducerSequenceId = 19025
>> >
>> > Value of BrokerSequenceId = 0
>> >
>> > No Data for Class BaseDataStructure
>> >
>> > End Class = MessageId
>> >
>> > Value of OriginalTransactionId is Below:
>> >
>> > Object is NULL
>> >
>> > Value of GroupID =
>> >
>> > Value of GroupSequence = 0
>> >
>> > Value of CorrelationId =
>> >
>> > Value of Persistent = 0
>> >
>> > Value of Expiration = 1197392556357
>> >
>> > Value of Priority = 4
>> >
>> > Value of ReplyTo is Below:
>> >
>> > Object is NULL
>> >
>> > Value of Timestamp = 1197392551357
>> >
>> > Value of Type =
>> >
>> > Value of Content[0] =
>> >
>> > Value of Content[1] = , check broker.
>> >
>> > FILE: ..\src\main\activemq\transport\filters\ResponseCorrelator.cpp,
>> > LINE:
>> > 146
>> >
>> > FILE: ..\src\main\activemq\transport\filters\ResponseCorrelator.cpp,
>> > LINE:
>> > 154
>> >
>> > FILE: ..\src\main\activemq\connector\openwire
>> > \OpenWireFormatNegotiator.cpp,
>> > LINE: 105
>> >
>> > FILE: ..\src\main\activemq\connector\openwire\OpenWireConnector.cpp,
>> > LINE:
>> > 1371
>> >
>> > FILE: ..\src\main\activemq\connector\openwire\OpenWireConnector.cpp,
>> > LINE:
>> > 848
>> >
>> > FILE: ..\src\main\activemq\core\ActiveMQSession.cpp, LINE: 675
>> >
>> > FILE: ..\src\main\activemq\core\ActiveMQProducer.cpp, LINE: 194
>> >
>> > FILE: ..\src\main\activemq\core\ActiveMQProducer.cpp, LINE: 149
>> >
>> > FILE: ..\src\main\activemq\core\ActiveMQProducer.cpp, LINE: 108
>> >
>> >
>> > Test 2
>> > Now if you build and run
>> > TestProducerBug
>> > TestConsumerBug
>> >
>> > These tests both use the C++ API and works as expected
>> >
>> >
>> > Test 3
>> > In the CPP program TestProducerBug you will find a sleep commented
>> > out in
>> > the ThreadProc uncomment this line.  Build Program.
>> > Build TestConsumerCSharp.
>> >
>> > You will find with the 100 ms sleep the application is stable.
>> >
>> >
>> > Test 4
>> > Build TestProducerBug remember to comment out the sleep
>> > Build TestConsumerCSharp.
>> >
>> > Place a breakpoint on the MessageListner in the C# program.
>> >
>> > In very little time the producer will throw an exception.
>> >
>> > Hellweek wrote:
>> >>
>> >>
>> >> Hello,
>> >>
>> >> I know what I am about to post will upset a few people, however I
>> >> think it
>> >> is important that I document my experience with ActiveMQ in the
>> >> hopes that
>> >> others like me can have an understanding of the issues that you
>> >> will face.
>> >>
>> >> A little history.
>> >>
>> >> I am not new to Open Source projects, have been involved in them
>> >> and have
>> >> sponsored the use of open source for many years.
>> >>
>> >> I have been working with various message brokers for a few years.  My
>> >> first experience was with TIBCO EMS.  Needless to say I was very
>> >> impressed
>> >> with the stability and functionality of this fine EMS.  Next I had
>> >> the
>> >> opportunity to work with Sonic EMS.  Again I was impressed with this
>> >> product and was even happier with its low cost of ownership.
>> >>
>> >> Over the last 6 weeks it has been my job to evaluate for our
>> >> Trading firm
>> >> an internal messaging system.  We wanted to use a EMS solution for
>> >> dissemination of pricing data to our in-house applications as well as
>> >> external clients of ours.  The messaging systems we are
>> >> evaluating.  TIBCO
>> >> EMS, MSMQ 3.0, SONIC EMS, ACTIVEMQ 4.1.1 or ActieMQ 5.0.
>> >>
>> >> How did each product fair?
>> >> 1. Tibco EMS no issues with any of the stress tests and performance
>> >> tests.
>> >> 2. MSMQ don't even get me started with this POS.
>> >> 3. SONIC EMS no issues with any of the stress tests and performance
>> >> tests.
>> >> 4. ActiveMQ can not make it past any stress tests.  See issues
>> >> below for
>> >> an understanding of what we saw.
>> >>
>> >>
>> >> I have watched ActiveMQ for well over 2 years and 2 years ago the
>> >> project
>> >> was so filled with issues that I knew I would never be able to
>> >> recommend
>> >> it to the owners of the company.  2 Years later and I was in the
>> >> position
>> >> of trying ActiveMQ again and hoping that it would be stable.
>> >>
>> >> I was very pleased to see that many of the issues I saw with
>> >> ActiveMQ had
>> >> been resolved and was committed to giving ActiveMQ a chance at
>> >> being our
>> >> EMS solution for the future.  However, I can say after weeks of
>> >> testing
>> >> ActiveMQ Is still not ready for production use by myself and the
>> >> firm I
>> >> work for.  If you have high message throughput with high number of
>> >> subscribers ActiveMQ is not well suited for your needs.
>> >>
>> >> Lets take some time to examine the issues.
>> >>
>> >> CPP ActiveMQ Client
>> >> 1. A fast producer with slow clients can and will take down the
>> >> producer.
>> >> From what I have seen in testing a slow client can bring the
>> >> producer down
>> >> and in some cases can bring the broker down.  A miss-behaved
>> >> producer or
>> >> client should never ever take the broker down.
>> >>
>> >> 2. A Producer that producers more then 200 messages per sec locks
>> >> up the
>> >> Broker when the Broker has only one client connected.  This one was
>> >> the
>> >> biggest issue to accept and the one issue that caused us to say
>> >> ActiveMQ
>> >> is not ready for a production environment.  The most basic and
>> >> simple task
>> >> of the Message Broker is not working as expected and makes the
>> >> ActiveMQ
>> >> unusable in a environment where peak message Generation can exceed
>> >> 200
>> >> messages per second.  To be honest we never even get close to 100
>> >> messages
>> >> as it seems we die after 50 messages are fired in the same second.
>> >> The
>> >> only time I am able to have producers producing without locking up or
>> >> crashing is if I don't have any consumers listening.  Having a
>> >> messaging
>> >> system that works without consumers is not a valid solution.
>> >>
>> >> Again important to note.  As long as no consumers are connected I can
>> >> produce massive amounts of messages.  Once you connect a client
>> >> massive
>> >> issues start to happen.
>> >>
>> >> 3. Producers and consumers created on the same connection can cause
>> >> deadlocks.  This is a major issue and the main solution is to not
>> >> do this.
>> >> However, this is an unacceptable solution as it is my understanding
>> >> this
>> >> is an acceptable practice.
>> >>
>> >> 4. A fast producer with a fast consumer leads to resource creep.  I
>> >> don't
>> >> want to say it is a memory leak because it is not a leak it is just
>> >> a very
>> >> very slow release of the memory.  I should not have to put sleeps
>> >> in a
>> >> program just to insure that memory gets released correctly.  In my
>> >> test I
>> >> had to sleep for 20 MS between each message being sent to keep the
>> >> ActiveMQ consumer running.
>> >>
>> >> 5. Placing a breakpoint on the message listener on a consumer will
>> >> cause
>> >> out of memory errors in the producer.  Why me setting a breakpoint
>> >> on a
>> >> consumer can cause the producer to throw an exception is
>> >> unacceptable and
>> >> leads me to think that a slow consumer can and will take the broker
>> >> and or
>> >> producer down.
>> >>
>> >> 6. Very confusing to determine what version of ActiveMQ will work
>> >> with
>> >> what version of the client.  Example ActiveMQ 5.0 was released this
>> >> week.
>> >> However, no new client was released and no information on when new
>> >> client
>> >> will be released.  The CPP client just released a 2.1.3 version that
>> >> claims it should be paired with 4.1.1 of the ActiveMQ broker.
>> >> Where is
>> >> the CPP client that is to work with the new features of 5.0?
>> >>
>> >> With all the issues I have I will not be able to go to a production
>> >> environment with ActiveMQ, this is a shame as the people that have
>> >> been
>> >> working this project are talented people and should be commended
>> >> for the
>> >> work that has been done.
>> >>
>> > http://www.nabble.com/file/p14278412/ActiveMQ%2BIssue.ZIP ActiveMQ
>> > +Issue.ZIP
>> > --
>> > View this message in context:
>> http://www.nabble.com/ActiveMQ-thoughts-tp14262131s2354p14278412.html
>> > Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>> >
>>
>>
> 
> 

-- 
View this message in context: http://www.nabble.com/ActiveMQ-thoughts-tp14262131s2354p14295149.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.


Mime
View raw message