qpid-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Praveen M <lefthandma...@gmail.com>
Subject Re: JMS Client usage - best practices?
Date Tue, 10 Jan 2012 17:48:01 GMT
Hi Robbie,

Thanks for writing.

Here is some more context for my 2nd question.

I'm expected to have about 30 client instances connecting to one broker.
About 90% of the messages to the broker are expected to be persistent
messages.

I haven't decided whether to use transactions or use the option
sync_publish. The
guarantee that we'd expect is that the broker receives the message before
proceeding to
send the next message. I'd be happy to hear your recommendations around
this.

>From the code I read, it looked like sync_publish was working to that
effect. But one thing I
saw was the sync() call in the client held to a connection failover lock,
and I wasn't sure if the sync() call was blocking.
Anyways, that said do you have a recommendation between transacted vs
sync_publish in a multi client environment (things going in parallel). I'd
really like to hear what you think about this.

There will not be any batch enqueues (a top of my head). Almost all our
traffic will be having messages
enqueued one at a time.

The number of destinations we have could be about 4000-5000. But yes, there
is a strong likelihood of one
queue being more popular and all the traffic getting routed there. We
sometimes receive bursty enqueues of a certain type
and is fair to assume that 5 or more enqueues of the same type to the same
queue is possible to happen concurrently.

We plan to have own framework on top of Qpid which will have the ability to
load balance between the destinations
if one queue goes over a certain threshold (some hand-picked number..not
decided on yet) and then route traffic to another destination.

And the overall traffic expected is somewhere between 10 million - 20
million messages a day maybe.

Please do let me know if you'd like to hear any other information.

Thank you,
Praveen


But yes, we'd be using one of the two (whichever would perform best with
multiple concurrent clients).

On Tue, Jan 10, 2012 at 6:41 AM, Robbie Gemmell <robbie.gemmell@gmail.com>wrote:

> Hi Praveen,
>
> Using either JNDI or Session.createQueue should work fine, but the
> obvious limitation of the latter is that the (possibly
> provider-specific) configuration is then part of your code (unless you
> look the information up somewhere before using it) and cant be changed
> without modifying the code.
>
> To better answer the other question, it would be good to first know
> more specifics about the application and its messaging model, eg:
> How many instances of the application are you likely to be running?
> Will all your messages be persistent?
> Will you always use transactions (and if so will it always of batch
> size 1 like below)?
> You mention having maybe 5-10 concurrent publishers, but how many
> different Destinations are you likely to be sending to in total?
> Are any of the above Destinations likely to be used a lot more than the
> others?
>
> Robbie
>
> On 10 January 2012 00:27, Praveen M <lefthandmagic@gmail.com> wrote:
> > Hi,
> >
> >   I'm writing a JMS Client and have a few best practices questions.
> >
> > 1) I understand that there are different ways ways to create a reference
> to
> > a destination to enqueue a message (using JMS createQueue, using JNDI
> > lookup).
> >    What would be the recommended way to use?
> >
> > 2) I create a connection to the broker on my application startup, and
> > create a session pool. on every enqueue, i checkout a session from my
> pool
> > use it and return to pool on enqueue.
> >    Does this sound reasonable?  (code sample below)
> >
> >    What would be the pattern that is typically used/recommended for
> > session management/connection management in the client side?
> >
> >    Or is this an overkill to do? Is CreateSession() and CreateProducer()
> > really cheap that it is just done on each enqueue?
> >    In my use case, I'd expect quite some concurrency (assume n (maybe
> > 5-10) threads doing enqueues at the same time) and I'd prefer it to not
> > lock/block and go through asap.
> >
> >    I do remember from reading the qpid client code that there is a lock
> > held on the connection object whenever a new session is created on that
> > connection. I'm assuming it is one of the places where my enqueue can
> slow
> > down
> >    when going in parallel if it has to create a session for every enqueue
> > of a message. There might be more such blocks, which I'm not aware of.
> >
> >   Taking performance into mind, what would be the best design for
> > session/connection management?
> >
> >    A snippet of what I have currently is below,
> >
> >    // a thread safe queue with the available session producer pairs which
> > can be used to create a message and enqueue.
> >    ConcurrentLinkedQueue<Pair<Session, MessageProducer>>
> > availableSessionProducerPairs = new ConcurrentLinkedQueue<Pair<Session,
> > MessageProducer>> ();
> >
> >   // initialize the sessionProducerPairs, create upto n pairs
> >    initializePool(Connection qpidConnection) {
> >         for (int i = 1 ; i <= SESSION_PRODUCER_PAIRS_COUNT; i ++ ) {
> >            Session session = qpidConnection.createSession(true,
> > Session.SESSION_TRANSACTED);
> >            MessageProducer producer = session.createProducer(null);
> >            availableSessionProducerPairs.add(Pair.newPair(session,
> > producer));
> >        }
> >    }
> >
> > // on enqueue we checkout a session/producer pair use it for the enqueue
> > and then return it back to the session/produer pair pool.
> >   enqueueMessage(String queueName, byte[] message) {
> >            // remove the session producer pair
> >             Pair<Session, MessageProducer> sessionProducerPair =
> > availableSessionProducerPairs.poll();
> >        if(sessionProducerPair == null) {
> >            // gack here and return null.
> >        }
> >        Session session = sessionProducerPair.getFirst();
> >        MessageProducer producer = sessionProducerPair.getSecond();
> >        BytesMessage jmsMessage = session.createBytesMessage();
> >        jmsMessage.writeBytes(message);
> >        Queue queueRef = queueNameToReferenceMap.get(queueName);
> >        producer.send(queueRef, jmsMessage, DeliveryMode.PERSISTENT, 0,
> 0);
> >        session.commit();
> >        availableSessionProducersPair.add()
> >  }
> >
> >
> >
> > Thank you,
> > -Praveen
>
> ---------------------------------------------------------------------
> Apache Qpid - AMQP Messaging Implementation
> Project:      http://qpid.apache.org
> Use/Interact: mailto:users-subscribe@qpid.apache.org
>
>


-- 
-Praveen

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message