qpid-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Praveen M <lefthandma...@gmail.com>
Subject Re: JMS Client usage - best practices?
Date Wed, 11 Jan 2012 19:02:06 GMT
ah, my bad... I expect to have 20 consumer threads per instance. Sorry, so
that'd be 600 consumer threads in the environment.

On Wed, Jan 11, 2012 at 10:14 AM, Praveen M <lefthandmagic@gmail.com> wrote:

> Great, that helps.
>
> Would you have a similar recommendation for the consumer side?
>
> In my environment, I expect about 200 consumer threads per instance. Total
> (30 instance * 200 threads/per instance) - 6000 consumer threads
>
> I was thinking I'd just use one connection per instance  and then spin off
> 200 sessions/consumers.
>
> my onMessage() callback is expected to take atleast 1 second to process (I
> don't see a reason why a message should be submitted for async processing
> if it takes lesser than that to process)
> , so I assumed I must be ok with  one connection.
>
> A  latency is totally acceptable before a message gets picked up.
> Is there any recommendations towards connection management per
> instance for the consumer side of things? I'd like to hear what you think
> about it. If I'm right to assume 1 connection per
> instance should be good?
>
> Also,
> Since the consumer side would add more parallel connections to the broker,
> I'm curious upon the limits I could push the broker.
> Is there any tested limit in terms of the number of connections that the
> broker could support?
> We are looking at 180 concurrent connections to the broker if I say 1
> connection per instance for all the consumers and 5 connections per
> instance for enqueues Should that be ok?
>
> Also, are there any cons for having a lot of connections to the broker
> which I should be aware of?
>
> Thanks a lot,
> Praveen
>
>
> On Wed, Jan 11, 2012 at 9:52 AM, Robbie Gemmell <robbie.gemmell@gmail.com>wrote:
>
>> 150 connections should be no problem.
>>
>> I would expect any locking on the client to be of little significance
>> to overall performance when using persistent messages, hitting the
>> disk is going to dominate overall performance. The issue with using
>> multiple Sessions on a Connection with persistent messages is that it
>> the Sessions are ultimately multiplexed onto the same TCP connection
>> and their data processed sequentially on arrival at the broker. If for
>> example you were to do 5 concurrent sends on 5 Sessions on the same
>> Connection, they will be processed onto the connection sequentially
>> and thus into the store sequentially upon arrival at the broker,
>> whereas if you do 5 concurrent sends for Sessions on individual
>> Connections there is scope for them to be processed into the store
>> concurrently and enable some/all of them to be write-combined and
>> synced to disk at the same time, reducing the required synchronous
>> disk IO to perform the operations and thus providing increased overall
>> performance/scalability by effectively 'doing more with less' so to
>> speak.
>>
>> Robbie
>>
>> On 11 January 2012 16:24, Praveen M <lefthandmagic@gmail.com> wrote:
>> > Hi Robbie,
>> >
>> > Thanks for writing. I just had a few followup questions. Please see
>> in-line.
>> >
>> > Thanks a lot for helping out with this,
>> > Praveen
>> >
>> > On Wed, Jan 11, 2012 at 6:58 AM, Robbie Gemmell <
>> robbie.gemmell@gmail.com>wrote:
>> >
>> >> Hi Praveen,
>> >>
>> >> If you are predominantly going to use persistent messages, then I
>> >> would actually look at your publishers using a single single session
>> >> per connection for the most part.
>> >>
>> >> The broker is currently able to best extract performance scaling from
>> >> the persistent stores (more so the BDBstore as per Robs recent
>> >> testing...though the results of which are again slightly out of date
>> >> now due to the upgrade to BDB5 earlier today) by performing actions on
>> >> a per-connection basis, so it can extract parrallism over commits to
>> >> the store. Using a lower number of connections with 5-10 sessions each
>> >> could be noticably slower at peak when using 150-300 publishers.
>> >>
>> >
>> > that totally makes sense. But now that I'd have approximately (30
>> instance
>> > * 5 connections) = 150
>> > connections to the broker, I wouldn't be overwhelming the broker right?
>> I'm
>> > guessing it shouldn't be a problem, But just want to run it by you?
>> >
>> > I understand that having separate connections is significantly faster as
>> > each
>> > connection would operate on it's own socket.
>> > Just out of curiosity I'd like to know, what are the performance
>> > degradation
>> > causes when using sessions sharing the same connection? I know for one
>> that
>> > the session has to hold on creation locks (for creation) and connection
>> > failover locks on send.
>> > Do you think this will be the cause of the major bottle necks or is
>> there
>> > something else too?
>> > (I'm sure you know the code back to front to answer this better) :)
>> >
>> >
>> >
>> >> If you are sending to a huge variety of Destinations then the way you
>> >> are using your Producers seems appropriate; creating producers isnt
>> >> necessarily that heavyweight an operation, but not doing so is
>> >> undeniably far more efficient.
>> >>
>> >
>> > ah, yep. that makes sense. I think I will stick to creating producers on
>> > connection pooling
>> > and creating the session
>> >
>> >
>> >>
>> >> I have actually never personally used sync_publish, but given that
>> >> both the underlying sync() call and session.commit() hit the broker
>> >> synchronously I would guess their performance should be pretty similar
>> >> for persistent messages with a batch size of 1, in which case I would
>> >> probably just go with transactions for 'pure JMS' purposes. If you
>> >> ever were using batches of >1 message that would seem to favor use of
>> >> transactions due to lowering the number of synchronous round trips to
>> >> the broker which would be required, increasing performance.
>> >>
>> >> yep, I totally agree with you on using transactions over the
>> sync_publish
>> > option
>> > for pure JMS purposes.
>> >
>> >
>> >> Robbie
>> >>
>> >> On 10 January 2012 17:48, Praveen M <lefthandmagic@gmail.com> wrote:
>> >> > Hi Robbie,
>> >> >
>> >> > Thanks for writing.
>> >> >
>> >> > Here is some more context for my 2nd question.
>> >> >
>> >> > I'm expected to have about 30 client instances connecting to one
>> broker.
>> >> > About 90% of the messages to the broker are expected to be persistent
>> >> > messages.
>> >> >
>> >> > I haven't decided whether to use transactions or use the option
>> >> > sync_publish. The
>> >> > guarantee that we'd expect is that the broker receives the message
>> before
>> >> > proceeding to
>> >> > send the next message. I'd be happy to hear your recommendations
>> around
>> >> > this.
>> >> >
>> >> > From the code I read, it looked like sync_publish was working to that
>> >> > effect. But one thing I
>> >> > saw was the sync() call in the client held to a connection failover
>> lock,
>> >> > and I wasn't sure if the sync() call was blocking.
>> >> > Anyways, that said do you have a recommendation between transacted
vs
>> >> > sync_publish in a multi client environment (things going in
>> parallel).
>> >> I'd
>> >> > really like to hear what you think about this.
>> >> >
>> >> > There will not be any batch enqueues (a top of my head). Almost all
>> our
>> >> > traffic will be having messages
>> >> > enqueued one at a time.
>> >> >
>> >> > The number of destinations we have could be about 4000-5000. But yes,
>> >> there
>> >> > is a strong likelihood of one
>> >> > queue being more popular and all the traffic getting routed there.
We
>> >> > sometimes receive bursty enqueues of a certain type
>> >> > and is fair to assume that 5 or more enqueues of the same type to the
>> >> same
>> >> > queue is possible to happen concurrently.
>> >> >
>> >> > We plan to have own framework on top of Qpid which will have the
>> ability
>> >> to
>> >> > load balance between the destinations
>> >> > if one queue goes over a certain threshold (some hand-picked
>> number..not
>> >> > decided on yet) and then route traffic to another destination.
>> >> >
>> >> > And the overall traffic expected is somewhere between 10 million -
20
>> >> > million messages a day maybe.
>> >> >
>> >> > Please do let me know if you'd like to hear any other information.
>> >> >
>> >> > Thank you,
>> >> > Praveen
>> >> >
>> >> >
>> >> > But yes, we'd be using one of the two (whichever would perform best
>> with
>> >> > multiple concurrent clients).
>> >> >
>> >> > On Tue, Jan 10, 2012 at 6:41 AM, Robbie Gemmell <
>> >> robbie.gemmell@gmail.com>wrote:
>> >> >
>> >> >> Hi Praveen,
>> >> >>
>> >> >> Using either JNDI or Session.createQueue should work fine, but
the
>> >> >> obvious limitation of the latter is that the (possibly
>> >> >> provider-specific) configuration is then part of your code (unless
>> you
>> >> >> look the information up somewhere before using it) and cant be
>> changed
>> >> >> without modifying the code.
>> >> >>
>> >> >> To better answer the other question, it would be good to first
know
>> >> >> more specifics about the application and its messaging model, eg:
>> >> >> How many instances of the application are you likely to be running?
>> >> >> Will all your messages be persistent?
>> >> >> Will you always use transactions (and if so will it always of batch
>> >> >> size 1 like below)?
>> >> >> You mention having maybe 5-10 concurrent publishers, but how many
>> >> >> different Destinations are you likely to be sending to in total?
>> >> >> Are any of the above Destinations likely to be used a lot more
than
>> the
>> >> >> others?
>> >> >>
>> >> >> Robbie
>> >> >>
>> >> >> On 10 January 2012 00:27, Praveen M <lefthandmagic@gmail.com>
>> wrote:
>> >> >> > Hi,
>> >> >> >
>> >> >> >   I'm writing a JMS Client and have a few best practices
>> questions.
>> >> >> >
>> >> >> > 1) I understand that there are different ways ways to create
a
>> >> reference
>> >> >> to
>> >> >> > a destination to enqueue a message (using JMS createQueue,
using
>> JNDI
>> >> >> > lookup).
>> >> >> >    What would be the recommended way to use?
>> >> >> >
>> >> >> > 2) I create a connection to the broker on my application startup,
>> and
>> >> >> > create a session pool. on every enqueue, i checkout a session
>> from my
>> >> >> pool
>> >> >> > use it and return to pool on enqueue.
>> >> >> >    Does this sound reasonable?  (code sample below)
>> >> >> >
>> >> >> >    What would be the pattern that is typically used/recommended
>> for
>> >> >> > session management/connection management in the client side?
>> >> >> >
>> >> >> >    Or is this an overkill to do? Is CreateSession() and
>> >> CreateProducer()
>> >> >> > really cheap that it is just done on each enqueue?
>> >> >> >    In my use case, I'd expect quite some concurrency (assume
n
>> (maybe
>> >> >> > 5-10) threads doing enqueues at the same time) and I'd prefer
it
>> to
>> >> not
>> >> >> > lock/block and go through asap.
>> >> >> >
>> >> >> >    I do remember from reading the qpid client code that there
is a
>> >> lock
>> >> >> > held on the connection object whenever a new session is created
on
>> >> that
>> >> >> > connection. I'm assuming it is one of the places where my
enqueue
>> can
>> >> >> slow
>> >> >> > down
>> >> >> >    when going in parallel if it has to create a session for
every
>> >> enqueue
>> >> >> > of a message. There might be more such blocks, which I'm not
>> aware of.
>> >> >> >
>> >> >> >   Taking performance into mind, what would be the best design
for
>> >> >> > session/connection management?
>> >> >> >
>> >> >> >    A snippet of what I have currently is below,
>> >> >> >
>> >> >> >    // a thread safe queue with the available session producer
>> pairs
>> >> which
>> >> >> > can be used to create a message and enqueue.
>> >> >> >    ConcurrentLinkedQueue<Pair<Session, MessageProducer>>
>> >> >> > availableSessionProducerPairs = new
>> >> ConcurrentLinkedQueue<Pair<Session,
>> >> >> > MessageProducer>> ();
>> >> >> >
>> >> >> >   // initialize the sessionProducerPairs, create upto n pairs
>> >> >> >    initializePool(Connection qpidConnection) {
>> >> >> >         for (int i = 1 ; i <= SESSION_PRODUCER_PAIRS_COUNT;
i ++
>> ) {
>> >> >> >            Session session = qpidConnection.createSession(true,
>> >> >> > Session.SESSION_TRANSACTED);
>> >> >> >            MessageProducer producer =
>> session.createProducer(null);
>> >> >> >            availableSessionProducerPairs.add(Pair.newPair(session,
>> >> >> > producer));
>> >> >> >        }
>> >> >> >    }
>> >> >> >
>> >> >> > // on enqueue we checkout a session/producer pair use it for
the
>> >> enqueue
>> >> >> > and then return it back to the session/produer pair pool.
>> >> >> >   enqueueMessage(String queueName, byte[] message) {
>> >> >> >            // remove the session producer pair
>> >> >> >             Pair<Session, MessageProducer> sessionProducerPair
=
>> >> >> > availableSessionProducerPairs.poll();
>> >> >> >        if(sessionProducerPair == null) {
>> >> >> >            // gack here and return null.
>> >> >> >        }
>> >> >> >        Session session = sessionProducerPair.getFirst();
>> >> >> >        MessageProducer producer = sessionProducerPair.getSecond();
>> >> >> >        BytesMessage jmsMessage = session.createBytesMessage();
>> >> >> >        jmsMessage.writeBytes(message);
>> >> >> >        Queue queueRef = queueNameToReferenceMap.get(queueName);
>> >> >> >        producer.send(queueRef, jmsMessage,
>> DeliveryMode.PERSISTENT, 0,
>> >> >> 0);
>> >> >> >        session.commit();
>> >> >> >        availableSessionProducersPair.add()
>> >> >> >  }
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> > Thank you,
>> >> >> > -Praveen
>> >> >>
>> >> >>
>> ---------------------------------------------------------------------
>> >> >> Apache Qpid - AMQP Messaging Implementation
>> >> >> Project:      http://qpid.apache.org
>> >> >> Use/Interact: mailto:users-subscribe@qpid.apache.org
>> >> >>
>> >> >>
>> >> >
>> >> >
>> >> > --
>> >> > -Praveen
>> >>
>> >> ---------------------------------------------------------------------
>> >> Apache Qpid - AMQP Messaging Implementation
>> >> Project:      http://qpid.apache.org
>> >> Use/Interact: mailto:users-subscribe@qpid.apache.org
>> >>
>> >>
>> >
>> >
>> > --
>> > -Praveen
>>
>> ---------------------------------------------------------------------
>> Apache Qpid - AMQP Messaging Implementation
>> Project:      http://qpid.apache.org
>> Use/Interact: mailto:users-subscribe@qpid.apache.org
>>
>>
>
>
> --
> -Praveen
>



-- 
-Praveen

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message