qpid-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From conflue...@apache.org
Subject [CONF] Apache Qpid > New Page
Date Thu, 20 Aug 2009 10:22:00 GMT
<html>
<head>
    <base href="http://cwiki.apache.org/confluence">
            <link rel="stylesheet" href="/confluence/s/1519/1/1/_/styles/combined.css?spaceKey=qpid&amp;forWysiwyg=true"
type="text/css">
    </head>
<body style="background-color: white" bgcolor="white">
<div id="pageContent">
<div id="notificationFormat">
<div class="wiki-content">
<div class="email">
     <h2><a href="http://cwiki.apache.org/confluence/display/qpid/New+Page">New
Page</a></h2>
     <h4>Page <b>edited</b> by             <a href="http://cwiki.apache.org/confluence/display/~ritchiem">Martin
Ritchie</a>
    </h4>
     
          <br/>
     <div class="notificationGreySide">
         <h2><a name="NewPage-Purpose"></a>Purpose</h2>

<p>This page is intended to outline the known use cases for running multiple Java Brokers,
addressing logged issues and limitations of the current implementation (as of V0.5). It is
<b>not</b> about clustering proper.</p>

<h2><a name="NewPage-UseCases"></a>Use Cases</h2>

<h3><a name="NewPage-HighVolumeTransientBroker"></a>High Volume Transient
Broker</h3>

<p><b>Description</b><br/>
The key characteristics of this application scenario are:</p>
<ol>
	<li>Broker availability is key</li>
	<li>Traffic is pub-sub</li>
	<li>Message Order is not important</li>
</ol>


<p>This use case relates to applications with a high residual message load i.e. where
message data on the broker remains in memory for some time or consumption lags production
such that a backlog is constantly present in the broker queues. </p>

<p>This paradigm is reasonably common, partly because publication threads are generally
handling only the simple publish call where we often see consumption threads handling writes
to RDBMS or other time expensive processing. Thus a rate gap opens up, and creates a data
tailback.</p>

<p>In this scenario, particularly for deployments on a 32bit VM, the broker can exhaust
a 3GB heap or start to perform poorly as it approaches max heap.</p>

<p><b>Result</b></p>

<p>Broker side OoM or performance degradation requiring bounce. Messages in flight not
processed, client applications experience connection loss.</p>

<p><b>Possible Solution A</b></p>

<p>In this scenario, it would be possible to use 2 brokers. Publishing clients would
use 2 connections and publish alternately to each broker, providing a simplistic load balancing
solution. Consuming clients would then consume from the 2 brokers, using the same topic name
etc. The consumer could choose to consume in parallel, thus potentially speeding up processing
time or by taking messages singly from the two sources alternately.</p>

<ul>
	<li>Possible Solution B *</li>
</ul>


<p>Again, using 2 brokers but this time working with the assumption that the application
data flows across the broker can be separated by source/destination. An easy solution for
this is to simply divide the required traffic by source or destination and put an amount on
each broker. </p>

<p>This may necessitate multiple consuming connections (to each of the brokers) on the
client side where there are multiple sources feeding the same client. Alternatively, for some
applications, the clients can be segmented in pairs of publisher-consumer by flow.</p>

<ul>
	<li>Possible Solution C *</li>
</ul>



<ul>
	<li>Notes *</li>
</ul>


<p>Note that encouraging applications to use round robin consumers in this paradigm
may also pay dividends, particularly where a secondart write can be run in parallel i.e. inserts
to an RDBMS are not mutually exclusive.</p>


<h3><a name="NewPage-LowVolumePersistentBroker"></a>Low Volume Persistent
Broker</h3>

<p><b>Description</b></p>


<p><b>Result</b></p>


<p><b>Possible Solution</b></p>


     </div>
     <div id="commentsSection" class="wiki-content pageSection">
       <div style="float: right;">
            <a href="http://cwiki.apache.org/confluence/users/viewnotifications.action"
class="grey">Change Notification Preferences</a>
       </div>

       <a href="http://cwiki.apache.org/confluence/display/qpid/New+Page">View Online</a>
       |
       <a href="http://cwiki.apache.org/confluence/pages/diffpagesbyversion.action?pageId=2851639&revisedVersion=3&originalVersion=2">View
Change</a>
              |
       <a href="http://cwiki.apache.org/confluence/display/qpid/New+Page?showComments=true&amp;showCommentArea=true#addcomment">Add
Comment</a>
            </div>
</div>
</div>
</div>
</div>
</body>
</html>

---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project:      http://qpid.apache.org
Use/Interact: mailto:commits-subscribe@qpid.apache.org


Mime
View raw message