qpid-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Praveen M <lefthandma...@gmail.com>
Subject Re: JMS Client usage - best practices?
Date Wed, 11 Jan 2012 18:14:54 GMT
Great, that helps.

Would you have a similar recommendation for the consumer side?

In my environment, I expect about 200 consumer threads per instance. Total
(30 instance * 200 threads/per instance) - 6000 consumer threads

I was thinking I'd just use one connection per instance  and then spin off
200 sessions/consumers.

my onMessage() callback is expected to take atleast 1 second to process (I
don't see a reason why a message should be submitted for async processing
if it takes lesser than that to process)
, so I assumed I must be ok with  one connection.

A  latency is totally acceptable before a message gets picked up.
Is there any recommendations towards connection management per instance for
the consumer side of things? I'd like to hear what you think about it. If
I'm right to assume 1 connection per
instance should be good?

Also,
Since the consumer side would add more parallel connections to the broker,
I'm curious upon the limits I could push the broker.
Is there any tested limit in terms of the number of connections that the
broker could support?
We are looking at 180 concurrent connections to the broker if I say 1
connection per instance for all the consumers and 5 connections per
instance for enqueues Should that be ok?

Also, are there any cons for having a lot of connections to the broker
which I should be aware of?

Thanks a lot,
Praveen


On Wed, Jan 11, 2012 at 9:52 AM, Robbie Gemmell <robbie.gemmell@gmail.com>wrote:

> 150 connections should be no problem.
>
> I would expect any locking on the client to be of little significance
> to overall performance when using persistent messages, hitting the
> disk is going to dominate overall performance. The issue with using
> multiple Sessions on a Connection with persistent messages is that it
> the Sessions are ultimately multiplexed onto the same TCP connection
> and their data processed sequentially on arrival at the broker. If for
> example you were to do 5 concurrent sends on 5 Sessions on the same
> Connection, they will be processed onto the connection sequentially
> and thus into the store sequentially upon arrival at the broker,
> whereas if you do 5 concurrent sends for Sessions on individual
> Connections there is scope for them to be processed into the store
> concurrently and enable some/all of them to be write-combined and
> synced to disk at the same time, reducing the required synchronous
> disk IO to perform the operations and thus providing increased overall
> performance/scalability by effectively 'doing more with less' so to
> speak.
>
> Robbie
>
> On 11 January 2012 16:24, Praveen M <lefthandmagic@gmail.com> wrote:
> > Hi Robbie,
> >
> > Thanks for writing. I just had a few followup questions. Please see
> in-line.
> >
> > Thanks a lot for helping out with this,
> > Praveen
> >
> > On Wed, Jan 11, 2012 at 6:58 AM, Robbie Gemmell <
> robbie.gemmell@gmail.com>wrote:
> >
> >> Hi Praveen,
> >>
> >> If you are predominantly going to use persistent messages, then I
> >> would actually look at your publishers using a single single session
> >> per connection for the most part.
> >>
> >> The broker is currently able to best extract performance scaling from
> >> the persistent stores (more so the BDBstore as per Robs recent
> >> testing...though the results of which are again slightly out of date
> >> now due to the upgrade to BDB5 earlier today) by performing actions on
> >> a per-connection basis, so it can extract parrallism over commits to
> >> the store. Using a lower number of connections with 5-10 sessions each
> >> could be noticably slower at peak when using 150-300 publishers.
> >>
> >
> > that totally makes sense. But now that I'd have approximately (30
> instance
> > * 5 connections) = 150
> > connections to the broker, I wouldn't be overwhelming the broker right?
> I'm
> > guessing it shouldn't be a problem, But just want to run it by you?
> >
> > I understand that having separate connections is significantly faster as
> > each
> > connection would operate on it's own socket.
> > Just out of curiosity I'd like to know, what are the performance
> > degradation
> > causes when using sessions sharing the same connection? I know for one
> that
> > the session has to hold on creation locks (for creation) and connection
> > failover locks on send.
> > Do you think this will be the cause of the major bottle necks or is there
> > something else too?
> > (I'm sure you know the code back to front to answer this better) :)
> >
> >
> >
> >> If you are sending to a huge variety of Destinations then the way you
> >> are using your Producers seems appropriate; creating producers isnt
> >> necessarily that heavyweight an operation, but not doing so is
> >> undeniably far more efficient.
> >>
> >
> > ah, yep. that makes sense. I think I will stick to creating producers on
> > connection pooling
> > and creating the session
> >
> >
> >>
> >> I have actually never personally used sync_publish, but given that
> >> both the underlying sync() call and session.commit() hit the broker
> >> synchronously I would guess their performance should be pretty similar
> >> for persistent messages with a batch size of 1, in which case I would
> >> probably just go with transactions for 'pure JMS' purposes. If you
> >> ever were using batches of >1 message that would seem to favor use of
> >> transactions due to lowering the number of synchronous round trips to
> >> the broker which would be required, increasing performance.
> >>
> >> yep, I totally agree with you on using transactions over the
> sync_publish
> > option
> > for pure JMS purposes.
> >
> >
> >> Robbie
> >>
> >> On 10 January 2012 17:48, Praveen M <lefthandmagic@gmail.com> wrote:
> >> > Hi Robbie,
> >> >
> >> > Thanks for writing.
> >> >
> >> > Here is some more context for my 2nd question.
> >> >
> >> > I'm expected to have about 30 client instances connecting to one
> broker.
> >> > About 90% of the messages to the broker are expected to be persistent
> >> > messages.
> >> >
> >> > I haven't decided whether to use transactions or use the option
> >> > sync_publish. The
> >> > guarantee that we'd expect is that the broker receives the message
> before
> >> > proceeding to
> >> > send the next message. I'd be happy to hear your recommendations
> around
> >> > this.
> >> >
> >> > From the code I read, it looked like sync_publish was working to that
> >> > effect. But one thing I
> >> > saw was the sync() call in the client held to a connection failover
> lock,
> >> > and I wasn't sure if the sync() call was blocking.
> >> > Anyways, that said do you have a recommendation between transacted vs
> >> > sync_publish in a multi client environment (things going in parallel).
> >> I'd
> >> > really like to hear what you think about this.
> >> >
> >> > There will not be any batch enqueues (a top of my head). Almost all
> our
> >> > traffic will be having messages
> >> > enqueued one at a time.
> >> >
> >> > The number of destinations we have could be about 4000-5000. But yes,
> >> there
> >> > is a strong likelihood of one
> >> > queue being more popular and all the traffic getting routed there. We
> >> > sometimes receive bursty enqueues of a certain type
> >> > and is fair to assume that 5 or more enqueues of the same type to the
> >> same
> >> > queue is possible to happen concurrently.
> >> >
> >> > We plan to have own framework on top of Qpid which will have the
> ability
> >> to
> >> > load balance between the destinations
> >> > if one queue goes over a certain threshold (some hand-picked
> number..not
> >> > decided on yet) and then route traffic to another destination.
> >> >
> >> > And the overall traffic expected is somewhere between 10 million - 20
> >> > million messages a day maybe.
> >> >
> >> > Please do let me know if you'd like to hear any other information.
> >> >
> >> > Thank you,
> >> > Praveen
> >> >
> >> >
> >> > But yes, we'd be using one of the two (whichever would perform best
> with
> >> > multiple concurrent clients).
> >> >
> >> > On Tue, Jan 10, 2012 at 6:41 AM, Robbie Gemmell <
> >> robbie.gemmell@gmail.com>wrote:
> >> >
> >> >> Hi Praveen,
> >> >>
> >> >> Using either JNDI or Session.createQueue should work fine, but the
> >> >> obvious limitation of the latter is that the (possibly
> >> >> provider-specific) configuration is then part of your code (unless
> you
> >> >> look the information up somewhere before using it) and cant be
> changed
> >> >> without modifying the code.
> >> >>
> >> >> To better answer the other question, it would be good to first know
> >> >> more specifics about the application and its messaging model, eg:
> >> >> How many instances of the application are you likely to be running?
> >> >> Will all your messages be persistent?
> >> >> Will you always use transactions (and if so will it always of batch
> >> >> size 1 like below)?
> >> >> You mention having maybe 5-10 concurrent publishers, but how many
> >> >> different Destinations are you likely to be sending to in total?
> >> >> Are any of the above Destinations likely to be used a lot more than
> the
> >> >> others?
> >> >>
> >> >> Robbie
> >> >>
> >> >> On 10 January 2012 00:27, Praveen M <lefthandmagic@gmail.com>
wrote:
> >> >> > Hi,
> >> >> >
> >> >> >   I'm writing a JMS Client and have a few best practices questions.
> >> >> >
> >> >> > 1) I understand that there are different ways ways to create a
> >> reference
> >> >> to
> >> >> > a destination to enqueue a message (using JMS createQueue, using
> JNDI
> >> >> > lookup).
> >> >> >    What would be the recommended way to use?
> >> >> >
> >> >> > 2) I create a connection to the broker on my application startup,
> and
> >> >> > create a session pool. on every enqueue, i checkout a session
from
> my
> >> >> pool
> >> >> > use it and return to pool on enqueue.
> >> >> >    Does this sound reasonable?  (code sample below)
> >> >> >
> >> >> >    What would be the pattern that is typically used/recommended
for
> >> >> > session management/connection management in the client side?
> >> >> >
> >> >> >    Or is this an overkill to do? Is CreateSession() and
> >> CreateProducer()
> >> >> > really cheap that it is just done on each enqueue?
> >> >> >    In my use case, I'd expect quite some concurrency (assume n
> (maybe
> >> >> > 5-10) threads doing enqueues at the same time) and I'd prefer
it to
> >> not
> >> >> > lock/block and go through asap.
> >> >> >
> >> >> >    I do remember from reading the qpid client code that there
is a
> >> lock
> >> >> > held on the connection object whenever a new session is created
on
> >> that
> >> >> > connection. I'm assuming it is one of the places where my enqueue
> can
> >> >> slow
> >> >> > down
> >> >> >    when going in parallel if it has to create a session for every
> >> enqueue
> >> >> > of a message. There might be more such blocks, which I'm not aware
> of.
> >> >> >
> >> >> >   Taking performance into mind, what would be the best design
for
> >> >> > session/connection management?
> >> >> >
> >> >> >    A snippet of what I have currently is below,
> >> >> >
> >> >> >    // a thread safe queue with the available session producer
pairs
> >> which
> >> >> > can be used to create a message and enqueue.
> >> >> >    ConcurrentLinkedQueue<Pair<Session, MessageProducer>>
> >> >> > availableSessionProducerPairs = new
> >> ConcurrentLinkedQueue<Pair<Session,
> >> >> > MessageProducer>> ();
> >> >> >
> >> >> >   // initialize the sessionProducerPairs, create upto n pairs
> >> >> >    initializePool(Connection qpidConnection) {
> >> >> >         for (int i = 1 ; i <= SESSION_PRODUCER_PAIRS_COUNT;
i ++ )
> {
> >> >> >            Session session = qpidConnection.createSession(true,
> >> >> > Session.SESSION_TRANSACTED);
> >> >> >            MessageProducer producer = session.createProducer(null);
> >> >> >            availableSessionProducerPairs.add(Pair.newPair(session,
> >> >> > producer));
> >> >> >        }
> >> >> >    }
> >> >> >
> >> >> > // on enqueue we checkout a session/producer pair use it for the
> >> enqueue
> >> >> > and then return it back to the session/produer pair pool.
> >> >> >   enqueueMessage(String queueName, byte[] message) {
> >> >> >            // remove the session producer pair
> >> >> >             Pair<Session, MessageProducer> sessionProducerPair
=
> >> >> > availableSessionProducerPairs.poll();
> >> >> >        if(sessionProducerPair == null) {
> >> >> >            // gack here and return null.
> >> >> >        }
> >> >> >        Session session = sessionProducerPair.getFirst();
> >> >> >        MessageProducer producer = sessionProducerPair.getSecond();
> >> >> >        BytesMessage jmsMessage = session.createBytesMessage();
> >> >> >        jmsMessage.writeBytes(message);
> >> >> >        Queue queueRef = queueNameToReferenceMap.get(queueName);
> >> >> >        producer.send(queueRef, jmsMessage,
> DeliveryMode.PERSISTENT, 0,
> >> >> 0);
> >> >> >        session.commit();
> >> >> >        availableSessionProducersPair.add()
> >> >> >  }
> >> >> >
> >> >> >
> >> >> >
> >> >> > Thank you,
> >> >> > -Praveen
> >> >>
> >> >> ---------------------------------------------------------------------
> >> >> Apache Qpid - AMQP Messaging Implementation
> >> >> Project:      http://qpid.apache.org
> >> >> Use/Interact: mailto:users-subscribe@qpid.apache.org
> >> >>
> >> >>
> >> >
> >> >
> >> > --
> >> > -Praveen
> >>
> >> ---------------------------------------------------------------------
> >> Apache Qpid - AMQP Messaging Implementation
> >> Project:      http://qpid.apache.org
> >> Use/Interact: mailto:users-subscribe@qpid.apache.org
> >>
> >>
> >
> >
> > --
> > -Praveen
>
> ---------------------------------------------------------------------
> Apache Qpid - AMQP Messaging Implementation
> Project:      http://qpid.apache.org
> Use/Interact: mailto:users-subscribe@qpid.apache.org
>
>


-- 
-Praveen

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message