activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nathan Mittler <nathan.mitt...@gmail.com>
Subject Re: ActiveMQ thoughts
Date Thu, 13 Dec 2007 14:54:44 GMT
Hey Rob,
Agreed - this certainly smells of a flow control issue!

The tricky bit is that two C++ clients can talk to each other without  
issue, whereas C#<->C++ doesn't work.  Given that both clients are  
talking to the broker, you would think this wouldn't make any  
difference :)

Regards,
Nate

On Dec 11, 2007, at 11:22 PM, Rob Davies wrote:

> Hi Nathan,
>
> I think we just need ensure that the C/C++/C# clients handle flow  
> control with the broker
>
> cheers,
>
> Rob
> On Dec 12, 2007, at 4:05 AM, Nathan Mittler wrote:
>
>> Appreciate your feedback and helping to identify this problem!   
>> I've captured this in a JIRA issue here:
>>
>> https://issues.apache.org/activemq/browse/AMQCPP-157
>>
>> We'll do our best to get this resolved soon!
>>
>> Regards,
>> Nate
>>
>> On Dec 11, 2007, at 9:10 AM, Hellweek wrote:
>>
>>>
>>> As promised I have created a c++ test program (TestProducerBug)  
>>> that will
>>> create up to X producers. The class that does the work is
>>> (TestProducers.cpp).
>>>
>>> I am created a C# test program (TestConsumerBugCSharp) that will  
>>> create up
>>> to X consumers using a MessageListener.  The class that does the  
>>> work is
>>> (TestConsumers.cs).
>>>
>>> I have created a C++ test program (TestConsumerBug) that will  
>>> create up to X
>>> consumers.  The class that does the work(TestConsumers.cpp).
>>>
>>> Here is some information on my setup.
>>>
>>> Compiler MS 2005.
>>>
>>> ActiveMQ
>>> Running ActiveMQ 5.0 Dated Dec 7th 2007.  It is running on windows  
>>> 2003
>>> Server 64 Bit.
>>> Running Java 1.6.0_02 this version of Java is 64 bit. (Problem  
>>> happens even
>>> on a 32 bit version of JAVA).
>>>
>>> ActiveMQ Settings
>>> Broker Settings (persistent="false" advisorySupport="false")
>>>
>>> Topic Policy
>>> <policyEntry topic="Test.>" producerFlowControl="true">
>>>
>>>          <!-- lets force old messages to be discarded for slow  
>>> consumers
>>> -->
>>>          <pendingMessageLimitStrategy>
>>>            <constantPendingMessageLimitStrategy limit="5"/>
>>>          </pendingMessageLimitStrategy>
>>> <messageEvictionStrategy>
>>> <oldestMessageEvictionStrategy />
>>> </messageEvictionStrategy>
>>>
>>>        </policyEntry>
>>>
>>>
>>> Client API's
>>>
>>> CPP activemq-cpp-2.1.2-src
>>> C# ApacheActiveMQ (Not sure the version but latest trunk).
>>>
>>>
>>> When running these test remember to stop and restart the broker  
>>> each test as
>>> the test can and will cause the broker to hang.
>>>
>>> Tests 1 -3 will show what is happening between the CPP and C# API.
>>>
>>> Test 4 will show what happens to a producer when a consumer is in  
>>> a break
>>> point in the MessageListener.
>>>
>>> Test 1
>>> To recreate the issue build and run
>>> TestProducerBug
>>> TestConsumerBugCSharp.
>>>
>>> If you set the number of producers and clients to 10 you should  
>>> see the
>>> problem happen in less then 5 min (About 2,000 messages per  
>>> consumer).
>>> The producer will throw an exception place a breakpoint on the  
>>> catch block
>>> in the ThreadProc.  you will see the following information.
>>> No valid response received for command: Begin Class =  
>>> ActiveMQBytesMessage
>>>
>>> Begin Class = ActiveMQMessageBase
>>>
>>> Value of ackHandler = 00000000
>>>
>>> Value of redeliveryCount = 0
>>>
>>> Value of properties = Begin Class PrimitiveMap:
>>>
>>> Begin Class PrimitiveMap:
>>>
>>> Begin Class = Message
>>>
>>> Value of Message::ID_MESSAGE = 0
>>>
>>> Value of ProducerId is Below:
>>>
>>> Begin Class = ProducerId
>>>
>>> Value of ProducerId::ID_PRODUCERID = 123
>>>
>>> Value of ConnectionId = 752afa01-c256-45c2-84ad-c74b0578f199
>>>
>>> Value of Value = 19
>>>
>>> Value of SessionId = 0
>>>
>>> No Data for Class BaseDataStructure
>>>
>>> End Class = ProducerId
>>>
>>> Value of Destination is Below:
>>>
>>> Begin Class = ActiveMQTopic
>>>
>>> Begin Class = ActiveMQDestination
>>>
>>> Value of exclusive = false
>>>
>>> Value of ordered = false
>>>
>>> Value of advisory = false
>>>
>>> Value of orderedTarget = coordinator
>>>
>>> Value of physicalName = Test.20
>>>
>>> Value of options = Begin Class activemq::util::Properties:
>>>
>>> End Class activemq::util::Properties:
>>>
>>> No Data for Class BaseDataStructure
>>>
>>> End Class = ActiveMQDestination
>>>
>>> End Class = ActiveMQTopic
>>>
>>> Value of TransactionId is Below:
>>>
>>> Object is NULL
>>>
>>> Value of OriginalDestination is Below:
>>>
>>> Object is NULL
>>>
>>> Value of MessageId is Below:
>>>
>>> Begin Class = MessageId
>>>
>>> Value of MessageId::ID_MESSAGEID = 110
>>>
>>> Value of ProducerId is Below:
>>>
>>> Begin Class = ProducerId
>>>
>>> Value of ProducerId::ID_PRODUCERID = 123
>>>
>>> Value of ConnectionId = 752afa01-c256-45c2-84ad-c74b0578f199
>>>
>>> Value of Value = 19
>>>
>>> Value of SessionId = 0
>>>
>>> No Data for Class BaseDataStructure
>>>
>>> End Class = ProducerId
>>>
>>> Value of ProducerSequenceId = 19025
>>>
>>> Value of BrokerSequenceId = 0
>>>
>>> No Data for Class BaseDataStructure
>>>
>>> End Class = MessageId
>>>
>>> Value of OriginalTransactionId is Below:
>>>
>>> Object is NULL
>>>
>>> Value of GroupID =
>>>
>>> Value of GroupSequence = 0
>>>
>>> Value of CorrelationId =
>>>
>>> Value of Persistent = 0
>>>
>>> Value of Expiration = 1197392556357
>>>
>>> Value of Priority = 4
>>>
>>> Value of ReplyTo is Below:
>>>
>>> Object is NULL
>>>
>>> Value of Timestamp = 1197392551357
>>>
>>> Value of Type =
>>>
>>> Value of Content[0] = 
>>>
>>> Value of Content[1] = , check broker.
>>>
>>> FILE: ..\src\main\activemq\transport\filters 
>>> \ResponseCorrelator.cpp, LINE:
>>> 146
>>>
>>> FILE: ..\src\main\activemq\transport\filters 
>>> \ResponseCorrelator.cpp, LINE:
>>> 154
>>>
>>> FILE: ..\src\main\activemq\connector\openwire 
>>> \OpenWireFormatNegotiator.cpp,
>>> LINE: 105
>>>
>>> FILE: ..\src\main\activemq\connector\openwire 
>>> \OpenWireConnector.cpp, LINE:
>>> 1371
>>>
>>> FILE: ..\src\main\activemq\connector\openwire 
>>> \OpenWireConnector.cpp, LINE:
>>> 848
>>>
>>> FILE: ..\src\main\activemq\core\ActiveMQSession.cpp, LINE: 675
>>>
>>> FILE: ..\src\main\activemq\core\ActiveMQProducer.cpp, LINE: 194
>>>
>>> FILE: ..\src\main\activemq\core\ActiveMQProducer.cpp, LINE: 149
>>>
>>> FILE: ..\src\main\activemq\core\ActiveMQProducer.cpp, LINE: 108
>>>
>>>
>>> Test 2
>>> Now if you build and run
>>> TestProducerBug
>>> TestConsumerBug
>>>
>>> These tests both use the C++ API and works as expected
>>>
>>>
>>> Test 3
>>> In the CPP program TestProducerBug you will find a sleep commented  
>>> out in
>>> the ThreadProc uncomment this line.  Build Program.
>>> Build TestConsumerCSharp.
>>>
>>> You will find with the 100 ms sleep the application is stable.
>>>
>>>
>>> Test 4
>>> Build TestProducerBug remember to comment out the sleep
>>> Build TestConsumerCSharp.
>>>
>>> Place a breakpoint on the MessageListner in the C# program.
>>>
>>> In very little time the producer will throw an exception.
>>>
>>> Hellweek wrote:
>>>>
>>>>
>>>> Hello,
>>>>
>>>> I know what I am about to post will upset a few people, however I  
>>>> think it
>>>> is important that I document my experience with ActiveMQ in the  
>>>> hopes that
>>>> others like me can have an understanding of the issues that you  
>>>> will face.
>>>>
>>>> A little history.
>>>>
>>>> I am not new to Open Source projects, have been involved in them  
>>>> and have
>>>> sponsored the use of open source for many years.
>>>>
>>>> I have been working with various message brokers for a few  
>>>> years.  My
>>>> first experience was with TIBCO EMS.  Needless to say I was very  
>>>> impressed
>>>> with the stability and functionality of this fine EMS.  Next I  
>>>> had the
>>>> opportunity to work with Sonic EMS.  Again I was impressed with  
>>>> this
>>>> product and was even happier with its low cost of ownership.
>>>>
>>>> Over the last 6 weeks it has been my job to evaluate for our  
>>>> Trading firm
>>>> an internal messaging system.  We wanted to use a EMS solution for
>>>> dissemination of pricing data to our in-house applications as  
>>>> well as
>>>> external clients of ours.  The messaging systems we are  
>>>> evaluating.  TIBCO
>>>> EMS, MSMQ 3.0, SONIC EMS, ACTIVEMQ 4.1.1 or ActieMQ 5.0.
>>>>
>>>> How did each product fair?
>>>> 1. Tibco EMS no issues with any of the stress tests and  
>>>> performance tests.
>>>> 2. MSMQ don't even get me started with this POS.
>>>> 3. SONIC EMS no issues with any of the stress tests and  
>>>> performance tests.
>>>> 4. ActiveMQ can not make it past any stress tests.  See issues  
>>>> below for
>>>> an understanding of what we saw.
>>>>
>>>>
>>>> I have watched ActiveMQ for well over 2 years and 2 years ago the  
>>>> project
>>>> was so filled with issues that I knew I would never be able to  
>>>> recommend
>>>> it to the owners of the company.  2 Years later and I was in the  
>>>> position
>>>> of trying ActiveMQ again and hoping that it would be stable.
>>>>
>>>> I was very pleased to see that many of the issues I saw with  
>>>> ActiveMQ had
>>>> been resolved and was committed to giving ActiveMQ a chance at  
>>>> being our
>>>> EMS solution for the future.  However, I can say after weeks of  
>>>> testing
>>>> ActiveMQ Is still not ready for production use by myself and the  
>>>> firm I
>>>> work for.  If you have high message throughput with high number of
>>>> subscribers ActiveMQ is not well suited for your needs.
>>>>
>>>> Lets take some time to examine the issues.
>>>>
>>>> CPP ActiveMQ Client
>>>> 1. A fast producer with slow clients can and will take down the  
>>>> producer.
>>>> From what I have seen in testing a slow client can bring the  
>>>> producer down
>>>> and in some cases can bring the broker down.  A miss-behaved  
>>>> producer or
>>>> client should never ever take the broker down.
>>>>
>>>> 2. A Producer that producers more then 200 messages per sec locks  
>>>> up the
>>>> Broker when the Broker has only one client connected.  This one  
>>>> was the
>>>> biggest issue to accept and the one issue that caused us to say  
>>>> ActiveMQ
>>>> is not ready for a production environment.  The most basic and  
>>>> simple task
>>>> of the Message Broker is not working as expected and makes the  
>>>> ActiveMQ
>>>> unusable in a environment where peak message Generation can  
>>>> exceed 200
>>>> messages per second.  To be honest we never even get close to 100  
>>>> messages
>>>> as it seems we die after 50 messages are fired in the same  
>>>> second.  The
>>>> only time I am able to have producers producing without locking  
>>>> up or
>>>> crashing is if I don't have any consumers listening.  Having a  
>>>> messaging
>>>> system that works without consumers is not a valid solution.
>>>>
>>>> Again important to note.  As long as no consumers are connected I  
>>>> can
>>>> produce massive amounts of messages.  Once you connect a client  
>>>> massive
>>>> issues start to happen.
>>>>
>>>> 3. Producers and consumers created on the same connection can cause
>>>> deadlocks.  This is a major issue and the main solution is to not  
>>>> do this.
>>>> However, this is an unacceptable solution as it is my  
>>>> understanding this
>>>> is an acceptable practice.
>>>>
>>>> 4. A fast producer with a fast consumer leads to resource creep.   
>>>> I don't
>>>> want to say it is a memory leak because it is not a leak it is  
>>>> just a very
>>>> very slow release of the memory.  I should not have to put sleeps  
>>>> in a
>>>> program just to insure that memory gets released correctly.  In  
>>>> my test I
>>>> had to sleep for 20 MS between each message being sent to keep the
>>>> ActiveMQ consumer running.
>>>>
>>>> 5. Placing a breakpoint on the message listener on a consumer  
>>>> will cause
>>>> out of memory errors in the producer.  Why me setting a  
>>>> breakpoint on a
>>>> consumer can cause the producer to throw an exception is  
>>>> unacceptable and
>>>> leads me to think that a slow consumer can and will take the  
>>>> broker and or
>>>> producer down.
>>>>
>>>> 6. Very confusing to determine what version of ActiveMQ will work  
>>>> with
>>>> what version of the client.  Example ActiveMQ 5.0 was released  
>>>> this week.
>>>> However, no new client was released and no information on when  
>>>> new client
>>>> will be released.  The CPP client just released a 2.1.3 version  
>>>> that
>>>> claims it should be paired with 4.1.1 of the ActiveMQ broker.   
>>>> Where is
>>>> the CPP client that is to work with the new features of 5.0?
>>>>
>>>> With all the issues I have I will not be able to go to a production
>>>> environment with ActiveMQ, this is a shame as the people that  
>>>> have been
>>>> working this project are talented people and should be commended  
>>>> for the
>>>> work that has been done.
>>>>
>>> http://www.nabble.com/file/p14278412/ActiveMQ%2BIssue.ZIP ActiveMQ 
>>> +Issue.ZIP
>>> -- 
>>> View this message in context: http://www.nabble.com/ActiveMQ-thoughts-tp14262131s2354p14278412.html
>>> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>>>
>>
>


Mime
View raw message