activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hellweek <bwins...@tfutures.com>
Subject Re: ActiveMQ thoughts
Date Mon, 17 Dec 2007 13:57:21 GMT

When performing the test with C# consumers the  CPP producers fail.
The CPP consumers do not fail with C# producers.

Hiram Chirino wrote:
> 
> Which one is the one that fails the C# side or the C++ side.  And if
> you reverse the consumer / producer role is it still the same?
> 
> On Dec 14, 2007 4:50 PM, Hellweek <bwinslow@tfutures.com> wrote:
>>
>> thats my thought also.  however C# to C# work C++ to c++ works but C# to
>> C++
>> wont.  Seems very odd.
>>
>>
>> nmittler wrote:
>> >
>> > Hey Rob,
>> > Agreed - this certainly smells of a flow control issue!
>> >
>> > The tricky bit is that two C++ clients can talk to each other without
>> > issue, whereas C#<->C++ doesn't work.  Given that both clients are
>> > talking to the broker, you would think this wouldn't make any
>> > difference :)
>> >
>> > Regards,
>> > Nate
>> >
>> > On Dec 11, 2007, at 11:22 PM, Rob Davies wrote:
>> >
>> >> Hi Nathan,
>> >>
>> >> I think we just need ensure that the C/C++/C# clients handle flow
>> >> control with the broker
>> >>
>> >> cheers,
>> >>
>> >> Rob
>> >> On Dec 12, 2007, at 4:05 AM, Nathan Mittler wrote:
>> >>
>> >>> Appreciate your feedback and helping to identify this problem!
>> >>> I've captured this in a JIRA issue here:
>> >>>
>> >>> https://issues.apache.org/activemq/browse/AMQCPP-157
>> >>>
>> >>> We'll do our best to get this resolved soon!
>> >>>
>> >>> Regards,
>> >>> Nate
>> >>>
>> >>> On Dec 11, 2007, at 9:10 AM, Hellweek wrote:
>> >>>
>> >>>>
>> >>>> As promised I have created a c++ test program (TestProducerBug)
>> >>>> that will
>> >>>> create up to X producers. The class that does the work is
>> >>>> (TestProducers.cpp).
>> >>>>
>> >>>> I am created a C# test program (TestConsumerBugCSharp) that will
>> >>>> create up
>> >>>> to X consumers using a MessageListener.  The class that does the
>> >>>> work is
>> >>>> (TestConsumers.cs).
>> >>>>
>> >>>> I have created a C++ test program (TestConsumerBug) that will
>> >>>> create up to X
>> >>>> consumers.  The class that does the work(TestConsumers.cpp).
>> >>>>
>> >>>> Here is some information on my setup.
>> >>>>
>> >>>> Compiler MS 2005.
>> >>>>
>> >>>> ActiveMQ
>> >>>> Running ActiveMQ 5.0 Dated Dec 7th 2007.  It is running on windows
>> >>>> 2003
>> >>>> Server 64 Bit.
>> >>>> Running Java 1.6.0_02 this version of Java is 64 bit. (Problem
>> >>>> happens even
>> >>>> on a 32 bit version of JAVA).
>> >>>>
>> >>>> ActiveMQ Settings
>> >>>> Broker Settings (persistent="false" advisorySupport="false")
>> >>>>
>> >>>> Topic Policy
>> >>>> <policyEntry topic="Test.>" producerFlowControl="true">
>> >>>>
>> >>>>          <!-- lets force old messages to be discarded for slow
>> >>>> consumers
>> >>>> -->
>> >>>>          <pendingMessageLimitStrategy>
>> >>>>            <constantPendingMessageLimitStrategy limit="5"/>
>> >>>>          </pendingMessageLimitStrategy>
>> >>>> <messageEvictionStrategy>
>> >>>> <oldestMessageEvictionStrategy />
>> >>>> </messageEvictionStrategy>
>> >>>>
>> >>>>        </policyEntry>
>> >>>>
>> >>>>
>> >>>> Client API's
>> >>>>
>> >>>> CPP activemq-cpp-2.1.2-src
>> >>>> C# ApacheActiveMQ (Not sure the version but latest trunk).
>> >>>>
>> >>>>
>> >>>> When running these test remember to stop and restart the broker
>> >>>> each test as
>> >>>> the test can and will cause the broker to hang.
>> >>>>
>> >>>> Tests 1 -3 will show what is happening between the CPP and C# API.
>> >>>>
>> >>>> Test 4 will show what happens to a producer when a consumer is in
>> >>>> a break
>> >>>> point in the MessageListener.
>> >>>>
>> >>>> Test 1
>> >>>> To recreate the issue build and run
>> >>>> TestProducerBug
>> >>>> TestConsumerBugCSharp.
>> >>>>
>> >>>> If you set the number of producers and clients to 10 you should
>> >>>> see the
>> >>>> problem happen in less then 5 min (About 2,000 messages per
>> >>>> consumer).
>> >>>> The producer will throw an exception place a breakpoint on the
>> >>>> catch block
>> >>>> in the ThreadProc.  you will see the following information.
>> >>>> No valid response received for command: Begin Class =
>> >>>> ActiveMQBytesMessage
>> >>>>
>> >>>> Begin Class = ActiveMQMessageBase
>> >>>>
>> >>>> Value of ackHandler = 00000000
>> >>>>
>> >>>> Value of redeliveryCount = 0
>> >>>>
>> >>>> Value of properties = Begin Class PrimitiveMap:
>> >>>>
>> >>>> Begin Class PrimitiveMap:
>> >>>>
>> >>>> Begin Class = Message
>> >>>>
>> >>>> Value of Message::ID_MESSAGE = 0
>> >>>>
>> >>>> Value of ProducerId is Below:
>> >>>>
>> >>>> Begin Class = ProducerId
>> >>>>
>> >>>> Value of ProducerId::ID_PRODUCERID = 123
>> >>>>
>> >>>> Value of ConnectionId = 752afa01-c256-45c2-84ad-c74b0578f199
>> >>>>
>> >>>> Value of Value = 19
>> >>>>
>> >>>> Value of SessionId = 0
>> >>>>
>> >>>> No Data for Class BaseDataStructure
>> >>>>
>> >>>> End Class = ProducerId
>> >>>>
>> >>>> Value of Destination is Below:
>> >>>>
>> >>>> Begin Class = ActiveMQTopic
>> >>>>
>> >>>> Begin Class = ActiveMQDestination
>> >>>>
>> >>>> Value of exclusive = false
>> >>>>
>> >>>> Value of ordered = false
>> >>>>
>> >>>> Value of advisory = false
>> >>>>
>> >>>> Value of orderedTarget = coordinator
>> >>>>
>> >>>> Value of physicalName = Test.20
>> >>>>
>> >>>> Value of options = Begin Class activemq::util::Properties:
>> >>>>
>> >>>> End Class activemq::util::Properties:
>> >>>>
>> >>>> No Data for Class BaseDataStructure
>> >>>>
>> >>>> End Class = ActiveMQDestination
>> >>>>
>> >>>> End Class = ActiveMQTopic
>> >>>>
>> >>>> Value of TransactionId is Below:
>> >>>>
>> >>>> Object is NULL
>> >>>>
>> >>>> Value of OriginalDestination is Below:
>> >>>>
>> >>>> Object is NULL
>> >>>>
>> >>>> Value of MessageId is Below:
>> >>>>
>> >>>> Begin Class = MessageId
>> >>>>
>> >>>> Value of MessageId::ID_MESSAGEID = 110
>> >>>>
>> >>>> Value of ProducerId is Below:
>> >>>>
>> >>>> Begin Class = ProducerId
>> >>>>
>> >>>> Value of ProducerId::ID_PRODUCERID = 123
>> >>>>
>> >>>> Value of ConnectionId = 752afa01-c256-45c2-84ad-c74b0578f199
>> >>>>
>> >>>> Value of Value = 19
>> >>>>
>> >>>> Value of SessionId = 0
>> >>>>
>> >>>> No Data for Class BaseDataStructure
>> >>>>
>> >>>> End Class = ProducerId
>> >>>>
>> >>>> Value of ProducerSequenceId = 19025
>> >>>>
>> >>>> Value of BrokerSequenceId = 0
>> >>>>
>> >>>> No Data for Class BaseDataStructure
>> >>>>
>> >>>> End Class = MessageId
>> >>>>
>> >>>> Value of OriginalTransactionId is Below:
>> >>>>
>> >>>> Object is NULL
>> >>>>
>> >>>> Value of GroupID =
>> >>>>
>> >>>> Value of GroupSequence = 0
>> >>>>
>> >>>> Value of CorrelationId =
>> >>>>
>> >>>> Value of Persistent = 0
>> >>>>
>> >>>> Value of Expiration = 1197392556357
>> >>>>
>> >>>> Value of Priority = 4
>> >>>>
>> >>>> Value of ReplyTo is Below:
>> >>>>
>> >>>> Object is NULL
>> >>>>
>> >>>> Value of Timestamp = 1197392551357
>> >>>>
>> >>>> Value of Type =
>> >>>>
>> >>>> Value of Content[0] =
>> >>>>
>> >>>> Value of Content[1] = , check broker.
>> >>>>
>> >>>> FILE: ..\src\main\activemq\transport\filters
>> >>>> \ResponseCorrelator.cpp, LINE:
>> >>>> 146
>> >>>>
>> >>>> FILE: ..\src\main\activemq\transport\filters
>> >>>> \ResponseCorrelator.cpp, LINE:
>> >>>> 154
>> >>>>
>> >>>> FILE: ..\src\main\activemq\connector\openwire
>> >>>> \OpenWireFormatNegotiator.cpp,
>> >>>> LINE: 105
>> >>>>
>> >>>> FILE: ..\src\main\activemq\connector\openwire
>> >>>> \OpenWireConnector.cpp, LINE:
>> >>>> 1371
>> >>>>
>> >>>> FILE: ..\src\main\activemq\connector\openwire
>> >>>> \OpenWireConnector.cpp, LINE:
>> >>>> 848
>> >>>>
>> >>>> FILE: ..\src\main\activemq\core\ActiveMQSession.cpp, LINE: 675
>> >>>>
>> >>>> FILE: ..\src\main\activemq\core\ActiveMQProducer.cpp, LINE: 194
>> >>>>
>> >>>> FILE: ..\src\main\activemq\core\ActiveMQProducer.cpp, LINE: 149
>> >>>>
>> >>>> FILE: ..\src\main\activemq\core\ActiveMQProducer.cpp, LINE: 108
>> >>>>
>> >>>>
>> >>>> Test 2
>> >>>> Now if you build and run
>> >>>> TestProducerBug
>> >>>> TestConsumerBug
>> >>>>
>> >>>> These tests both use the C++ API and works as expected
>> >>>>
>> >>>>
>> >>>> Test 3
>> >>>> In the CPP program TestProducerBug you will find a sleep commented
>> >>>> out in
>> >>>> the ThreadProc uncomment this line.  Build Program.
>> >>>> Build TestConsumerCSharp.
>> >>>>
>> >>>> You will find with the 100 ms sleep the application is stable.
>> >>>>
>> >>>>
>> >>>> Test 4
>> >>>> Build TestProducerBug remember to comment out the sleep
>> >>>> Build TestConsumerCSharp.
>> >>>>
>> >>>> Place a breakpoint on the MessageListner in the C# program.
>> >>>>
>> >>>> In very little time the producer will throw an exception.
>> >>>>
>> >>>> Hellweek wrote:
>> >>>>>
>> >>>>>
>> >>>>> Hello,
>> >>>>>
>> >>>>> I know what I am about to post will upset a few people, however
I
>> >>>>> think it
>> >>>>> is important that I document my experience with ActiveMQ in
the
>> >>>>> hopes that
>> >>>>> others like me can have an understanding of the issues that
you
>> >>>>> will face.
>> >>>>>
>> >>>>> A little history.
>> >>>>>
>> >>>>> I am not new to Open Source projects, have been involved in
them
>> >>>>> and have
>> >>>>> sponsored the use of open source for many years.
>> >>>>>
>> >>>>> I have been working with various message brokers for a few
>> >>>>> years.  My
>> >>>>> first experience was with TIBCO EMS.  Needless to say I was
very
>> >>>>> impressed
>> >>>>> with the stability and functionality of this fine EMS.  Next
I
>> >>>>> had the
>> >>>>> opportunity to work with Sonic EMS.  Again I was impressed with
>> >>>>> this
>> >>>>> product and was even happier with its low cost of ownership.
>> >>>>>
>> >>>>> Over the last 6 weeks it has been my job to evaluate for our
>> >>>>> Trading firm
>> >>>>> an internal messaging system.  We wanted to use a EMS solution
for
>> >>>>> dissemination of pricing data to our in-house applications as
>> >>>>> well as
>> >>>>> external clients of ours.  The messaging systems we are
>> >>>>> evaluating.  TIBCO
>> >>>>> EMS, MSMQ 3.0, SONIC EMS, ACTIVEMQ 4.1.1 or ActieMQ 5.0.
>> >>>>>
>> >>>>> How did each product fair?
>> >>>>> 1. Tibco EMS no issues with any of the stress tests and
>> >>>>> performance tests.
>> >>>>> 2. MSMQ don't even get me started with this POS.
>> >>>>> 3. SONIC EMS no issues with any of the stress tests and
>> >>>>> performance tests.
>> >>>>> 4. ActiveMQ can not make it past any stress tests.  See issues
>> >>>>> below for
>> >>>>> an understanding of what we saw.
>> >>>>>
>> >>>>>
>> >>>>> I have watched ActiveMQ for well over 2 years and 2 years ago
the
>> >>>>> project
>> >>>>> was so filled with issues that I knew I would never be able
to
>> >>>>> recommend
>> >>>>> it to the owners of the company.  2 Years later and I was in
the
>> >>>>> position
>> >>>>> of trying ActiveMQ again and hoping that it would be stable.
>> >>>>>
>> >>>>> I was very pleased to see that many of the issues I saw with
>> >>>>> ActiveMQ had
>> >>>>> been resolved and was committed to giving ActiveMQ a chance
at
>> >>>>> being our
>> >>>>> EMS solution for the future.  However, I can say after weeks
of
>> >>>>> testing
>> >>>>> ActiveMQ Is still not ready for production use by myself and
the
>> >>>>> firm I
>> >>>>> work for.  If you have high message throughput with high number
of
>> >>>>> subscribers ActiveMQ is not well suited for your needs.
>> >>>>>
>> >>>>> Lets take some time to examine the issues.
>> >>>>>
>> >>>>> CPP ActiveMQ Client
>> >>>>> 1. A fast producer with slow clients can and will take down
the
>> >>>>> producer.
>> >>>>> From what I have seen in testing a slow client can bring the
>> >>>>> producer down
>> >>>>> and in some cases can bring the broker down.  A miss-behaved
>> >>>>> producer or
>> >>>>> client should never ever take the broker down.
>> >>>>>
>> >>>>> 2. A Producer that producers more then 200 messages per sec
locks
>> >>>>> up the
>> >>>>> Broker when the Broker has only one client connected.  This
one
>> >>>>> was the
>> >>>>> biggest issue to accept and the one issue that caused us to
say
>> >>>>> ActiveMQ
>> >>>>> is not ready for a production environment.  The most basic and
>> >>>>> simple task
>> >>>>> of the Message Broker is not working as expected and makes the
>> >>>>> ActiveMQ
>> >>>>> unusable in a environment where peak message Generation can
>> >>>>> exceed 200
>> >>>>> messages per second.  To be honest we never even get close to
100
>> >>>>> messages
>> >>>>> as it seems we die after 50 messages are fired in the same
>> >>>>> second.  The
>> >>>>> only time I am able to have producers producing without locking
>> >>>>> up or
>> >>>>> crashing is if I don't have any consumers listening.  Having
a
>> >>>>> messaging
>> >>>>> system that works without consumers is not a valid solution.
>> >>>>>
>> >>>>> Again important to note.  As long as no consumers are connected
I
>> >>>>> can
>> >>>>> produce massive amounts of messages.  Once you connect a client
>> >>>>> massive
>> >>>>> issues start to happen.
>> >>>>>
>> >>>>> 3. Producers and consumers created on the same connection can
cause
>> >>>>> deadlocks.  This is a major issue and the main solution is to
not
>> >>>>> do this.
>> >>>>> However, this is an unacceptable solution as it is my
>> >>>>> understanding this
>> >>>>> is an acceptable practice.
>> >>>>>
>> >>>>> 4. A fast producer with a fast consumer leads to resource creep.
>> >>>>> I don't
>> >>>>> want to say it is a memory leak because it is not a leak it
is
>> >>>>> just a very
>> >>>>> very slow release of the memory.  I should not have to put sleeps
>> >>>>> in a
>> >>>>> program just to insure that memory gets released correctly.
 In
>> >>>>> my test I
>> >>>>> had to sleep for 20 MS between each message being sent to keep
the
>> >>>>> ActiveMQ consumer running.
>> >>>>>
>> >>>>> 5. Placing a breakpoint on the message listener on a consumer
>> >>>>> will cause
>> >>>>> out of memory errors in the producer.  Why me setting a
>> >>>>> breakpoint on a
>> >>>>> consumer can cause the producer to throw an exception is
>> >>>>> unacceptable and
>> >>>>> leads me to think that a slow consumer can and will take the
>> >>>>> broker and or
>> >>>>> producer down.
>> >>>>>
>> >>>>> 6. Very confusing to determine what version of ActiveMQ will
work
>> >>>>> with
>> >>>>> what version of the client.  Example ActiveMQ 5.0 was released
>> >>>>> this week.
>> >>>>> However, no new client was released and no information on when
>> >>>>> new client
>> >>>>> will be released.  The CPP client just released a 2.1.3 version
>> >>>>> that
>> >>>>> claims it should be paired with 4.1.1 of the ActiveMQ broker.
>> >>>>> Where is
>> >>>>> the CPP client that is to work with the new features of 5.0?
>> >>>>>
>> >>>>> With all the issues I have I will not be able to go to a production
>> >>>>> environment with ActiveMQ, this is a shame as the people that
>> >>>>> have been
>> >>>>> working this project are talented people and should be commended
>> >>>>> for the
>> >>>>> work that has been done.
>> >>>>>
>> >>>> http://www.nabble.com/file/p14278412/ActiveMQ%2BIssue.ZIP ActiveMQ
>> >>>> +Issue.ZIP
>> >>>> --
>> >>>> View this message in context:
>> >>>>
>> http://www.nabble.com/ActiveMQ-thoughts-tp14262131s2354p14278412.html
>> >>>> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>> >>>>
>> >>>
>> >>
>> >
>> >
>> >
>>
>> --
>> View this message in context:
>> http://www.nabble.com/ActiveMQ-thoughts-tp14262131s2354p14339819.html
>>
>> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>>
>>
> 
> 
> 
> -- 
> Regards,
> Hiram
> 
> Blog: http://hiramchirino.com
> 
> Open Source SOA
> http://open.iona.com
> 
> 

-- 
View this message in context: http://www.nabble.com/ActiveMQ-thoughts-tp14262131s2354p14370300.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.


Mime
View raw message