activemq-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From chir...@apache.org
Subject svn commit: r961447 - in /activemq/sandbox/activemq-apollo-actor: apollo-website/src/architecture.md apollo-website/src/architecture.page apollo-website/src/performance-scaling.md apollo-website/src/performance-scaling.page readme.md
Date Wed, 07 Jul 2010 18:03:54 GMT
Author: chirino
Date: Wed Jul  7 18:03:54 2010
New Revision: 961447

URL: http://svn.apache.org/viewvc?rev=961447&view=rev
Log:
improving doco and readme.

Added:
    activemq/sandbox/activemq-apollo-actor/apollo-website/src/architecture.md
      - copied, changed from r961435, activemq/sandbox/activemq-apollo-actor/apollo-website/src/architecture.page
    activemq/sandbox/activemq-apollo-actor/apollo-website/src/performance-scaling.md
      - copied, changed from r961435, activemq/sandbox/activemq-apollo-actor/apollo-website/src/performance-scaling.page
Modified:
    activemq/sandbox/activemq-apollo-actor/apollo-website/src/architecture.page
    activemq/sandbox/activemq-apollo-actor/apollo-website/src/performance-scaling.page
    activemq/sandbox/activemq-apollo-actor/readme.md

Copied: activemq/sandbox/activemq-apollo-actor/apollo-website/src/architecture.md (from r961435,
activemq/sandbox/activemq-apollo-actor/apollo-website/src/architecture.page)
URL: http://svn.apache.org/viewvc/activemq/sandbox/activemq-apollo-actor/apollo-website/src/architecture.md?p2=activemq/sandbox/activemq-apollo-actor/apollo-website/src/architecture.md&p1=activemq/sandbox/activemq-apollo-actor/apollo-website/src/architecture.page&r1=961435&r2=961447&rev=961447&view=diff
==============================================================================
--- activemq/sandbox/activemq-apollo-actor/apollo-website/src/architecture.page (original)
+++ activemq/sandbox/activemq-apollo-actor/apollo-website/src/architecture.md Wed Jul  7 18:03:54
2010
@@ -1,31 +1,28 @@
----
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements.  See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License.  You may obtain a copy of the License at
-#
-#      http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-title: Architecture
---- name:overview
-
-{project_slogan:}
-
---- name:content
-## Architectural Changes
+<!--
+  Licensed to the Apache Software Foundation (ASF) under one or more
+  contributor license agreements.  See the NOTICE file distributed with
+  this work for additional information regarding copyright ownership.
+  The ASF licenses this file to You under the Apache License, Version 2.0
+  (the "License"); you may not use this file except in compliance with
+  the License.  You may obtain a copy of the License at
+  
+       http://www.apache.org/licenses/LICENSE-2.0
+  
+  Unless required by applicable law or agreed to in writing, software
+  distributed under the License is distributed on an "AS IS" BASIS,
+  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+  See the License for the specific language governing permissions and
+  limitations under the License.
+  Architecture
+-->
+# Architecture
 
 Apollo started as an experiment to see what it would take to make ActiveMQ
 work better on machines with higher core counts. It has resulted in broker
 that is much more deterministic, stable, and scaleable.
 
+## Architectural Changes
+
 The major fundamental architectural changes it brings are:
 
 * Reactor Based Thread Model

Modified: activemq/sandbox/activemq-apollo-actor/apollo-website/src/architecture.page
URL: http://svn.apache.org/viewvc/activemq/sandbox/activemq-apollo-actor/apollo-website/src/architecture.page?rev=961447&r1=961446&r2=961447&view=diff
==============================================================================
--- activemq/sandbox/activemq-apollo-actor/apollo-website/src/architecture.page (original)
+++ activemq/sandbox/activemq-apollo-actor/apollo-website/src/architecture.page Wed Jul  7
18:03:54 2010
@@ -20,82 +20,5 @@ title: Architecture
 {project_slogan:}
 
 --- name:content
-## Architectural Changes
-
-Apollo started as an experiment to see what it would take to make ActiveMQ
-work better on machines with higher core counts. It has resulted in broker
-that is much more deterministic, stable, and scaleable.
-
-The major fundamental architectural changes it brings are:
-
-* Reactor Based Thread Model
-* Scala 2.8 Implementation
-* Protocol Agnostic
-* REST Based Management
-
-### Reactor Based Thread Model
-
-Apollo uses [HawtDispatch][] to implement the sever using a multi-threaded
-non-blocking variation of the [reactor][] design pattern. HawtDispatch is a
-Java clone of [libdispatch][] (aka [Grand Central Dispatch][gcd]). It uses a
-fixed sized thread pool and only executes non-blocking tasks on those
-threads.
-
-[reactor]:http://en.wikipedia.org/wiki/Reactor_pattern
-[libdispatch]:http://en.wikipedia.org/wiki/Libdispatch
-[HawtDispatch]:http://hawtdispatch.fusesource.org/
-[gcd]:http://images.apple.com/macosx/technology/docs/GrandCentral_TB_brief_20090903.pdf
-
-
-The thread model allows Apollo to reach very high levels of scaleability and
-efficiency, but it places a huge restriction on the developer: all the tasks
-it executes must be non-blocking and ideally lock-free and wait free. This
-means that the previous ActiveMQ broker architecture had to go through a
-major overhaul. All synchronous broker interfaces had to be changed
-so that they would instead return results via asynchronous callbacks.
-
-### Scala 2.8 Implementation
-
-Even though Apollo started as a fork of ActiveMQ 5.x, the new reactor design
-restrictions required major changes from the network IO handling, to the
-flow control design, all the way to the Store interfaces. Scala provided a
-much more concise way to express callbacks, namely by it's support for
-partial functions and closures.
-
-### Protocol Agnostic
-
-ActiveMQ has supported multiple protocols for many years, but under the
-covers what it's doing is converting all the protocols to use the
-[OpenWire][] messaging protocol. Naturally, this approach is not optimal if
-you want to efficiently support other protocols and not be limited by what
-OpenWire can support.
-
-[OpenWire]:http://activemq.apache.org/openwire.html
-
-The Apollo server is much more modular and protocol agnostic. All protocols
-are equal and are built as a plugin to the the broker which just make use of
-exposed broker services for routing, flow control, queueing services etc.
-For example, this means that messages will be persisted in a Store in the
-original protocol's encoding. There is no protocol conversion occurring
-under the covers unless it is required.
-
-### REST Based Management
-
-ActiveMQ choose to exposed it's management interfaces via JMX. JMX was a
-natural choice since ActiveMQ is Java based, but JMX has a couple of
-serious limitations:
-
-* No cross language support
-* Not scaleable for exposing many objects. Registering and unregistering
-  management objects in JMX can become a bottle neck.
-* Rich data types are hard to expose
-
-Apollo exposes a rich and detailed state of the sever using REST based JSON
-or XML services.
-
-* A management client can easily be implemented in any language.
-* There is very little management overhead since there is no special
-  registration with the management system. The JAX-RS based management web
-  application knows how to navigate the internal structure of a broker to
-  access all the need status and statistics.
 
+{include_file: {filename: src/architecture.md, process_output: true, escape_html: false}}
\ No newline at end of file

Copied: activemq/sandbox/activemq-apollo-actor/apollo-website/src/performance-scaling.md (from
r961435, activemq/sandbox/activemq-apollo-actor/apollo-website/src/performance-scaling.page)
URL: http://svn.apache.org/viewvc/activemq/sandbox/activemq-apollo-actor/apollo-website/src/performance-scaling.md?p2=activemq/sandbox/activemq-apollo-actor/apollo-website/src/performance-scaling.md&p1=activemq/sandbox/activemq-apollo-actor/apollo-website/src/performance-scaling.page&r1=961435&r2=961447&rev=961447&view=diff
==============================================================================
--- activemq/sandbox/activemq-apollo-actor/apollo-website/src/performance-scaling.page (original)
+++ activemq/sandbox/activemq-apollo-actor/apollo-website/src/performance-scaling.md Wed Jul
 7 18:03:54 2010
@@ -1,21 +1,21 @@
----
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements.  See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License.  You may obtain a copy of the License at
-#
-#      http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-title: Performance and Scaling
----
+<!--
+  Licensed to the Apache Software Foundation (ASF) under one or more
+  contributor license agreements.  See the NOTICE file distributed with
+  this work for additional information regarding copyright ownership.
+  The ASF licenses this file to You under the Apache License, Version 2.0
+  (the "License"); you may not use this file except in compliance with
+  the License.  You may obtain a copy of the License at
+  
+       http://www.apache.org/licenses/LICENSE-2.0
+  
+  Unless required by applicable law or agreed to in writing, software
+  distributed under the License is distributed on an "AS IS" BASIS,
+  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+  See the License for the specific language governing permissions and
+  limitations under the License.
+  Architecture
+-->
+# Performance and Scaling
 
 ## Performance
 

Modified: activemq/sandbox/activemq-apollo-actor/apollo-website/src/performance-scaling.page
URL: http://svn.apache.org/viewvc/activemq/sandbox/activemq-apollo-actor/apollo-website/src/performance-scaling.page?rev=961447&r1=961446&r2=961447&view=diff
==============================================================================
--- activemq/sandbox/activemq-apollo-actor/apollo-website/src/performance-scaling.page (original)
+++ activemq/sandbox/activemq-apollo-actor/apollo-website/src/performance-scaling.page Wed
Jul  7 18:03:54 2010
@@ -15,89 +15,10 @@
 # limitations under the License.
 
 title: Performance and Scaling
----
+--- name:overview
 
-## Performance
+{project_slogan:}
 
-Apollo's performance is awesome considering it is the text based protocol
-Stomp. The source distribution includes a couple of benchmarks that are
-useful for getting an idea of it's performance potential.
+--- name:content
 
-The benchmark clients access the server as follows:
-
-* Clients and server run on the same machine. 
-* Clients access the server using STOMP over TCP. 
-* Producers send non-persistent messages
-* Messages contain a 20 byte payload 
-* Producers do not wait for a broker ack before sending the next message. 
-* Consumers use auto ack.
-
-The following benchmark results were run on a Mac Pro with:
-
-* Dual socket 3 ghz Quad-Core Intel Xeon (8 total cores) with 12 MB L2
-  cache per processor .
-* 8 GB of RAM
-
-### Queue Cases
-
-* 1 producer sending to 1 consumer via 1 queue can hit rates of 250,000
-  messages/second.
-* 8 producers sending to 8 consumers via 8 queues can hit rates of 540,000
-  messages/second.
-* 1 producer sending to 10 consumers via 1 queue can hit rates of 230,000
-  messages/second.
-* 10 producers sending to 1 consumers via 1 queue can hit rates of 280,000
-  messages/second.
-
-### Topic Cases
-
-Rates reported are the total consumer rate.
-
-* 1 producer sending to 1 consumer via 1 topic can hit a rates of 420,000
-  messages/second.
-* 8 producers sending to 8 consumers via 8 topics can hit rates of 810,000
-  messages/second.
-* 10 producer sending to 10 consumers via 1 topic can hit rates of
-  1,3000,000 messages/second.
-
-## Scaling Characteristics
-
-There are many different extreme ways that a messaging system can be used. 
-Some folks like to:
-
-* connect a large number of clients.
-* hold a large number of messages in their queues.  
-* move around large messages.
-
-Apollo aims to support all those usage scenarios.
-
-### Scaling the Number of Connected Clients
-
-Apollo uses non blocking IO and a reactor thread model. This means that a
-running broker uses a constant number of threads no matter how many clients
-are connected to it.
-
-### Scaling the Number of Queued Messages
-
-Queues will swap messages out of memory when there are no consumers that
-are likely to need the message soon. Once a message is swapped, the queue
-will replace the message with a reference pointer. When the queue builds a
-large number (more than 10,000) of these swapped out reference pointers,
-they then get consolidated into a single "range entry" and the pointers are
-dropped. 
-
-When a consumer comes along that needs some of the swapped out messages,
-it will convert previously consolidated "range entries" to a set of
-reference pointers. The next reference pointers that the consumer is
-interested in get swapped back as regular messages. This allows Apollo's
-queues to hold millions of messages without much impact on JVM memory
-usage.
-
-### Scaling the Message Size
-
-Big messages don't even make it into the JVM's memory management. The
-contents of big messages received and sent using buffers allocated in a
-memory mapped file. This allows the OS to efficiently manage swapping those
-large messages to disk and it avoids churning the eden garbage collection
-generation in the JVM. If 1 producer is sending 100 MB messages to a
-consumer the JVM will not report any significant memory usage.
\ No newline at end of file
+{include_file: {filename: src/performance-scaling.md, process_output: true, escape_html:
false}}

Modified: activemq/sandbox/activemq-apollo-actor/readme.md
URL: http://svn.apache.org/viewvc/activemq/sandbox/activemq-apollo-actor/readme.md?rev=961447&r1=961446&r2=961447&view=diff
==============================================================================
--- activemq/sandbox/activemq-apollo-actor/readme.md (original)
+++ activemq/sandbox/activemq-apollo-actor/readme.md Wed Jul  7 18:03:54 2010
@@ -20,183 +20,101 @@ broker. It is focused on simplicity, sta
 * REST based management
 * [Stomp](http://stomp.github.com/) Protocol Support
 
-## Performance
+## What makes Apollo Different?
 
-Apollo's performance is awesome considering it is the text based protocol
-Stomp. The source distribution includes a couple of benchmarks that are
-useful for getting an idea of it's performance potential.
-
-The benchmark clients access the server as follows:
-
-* Clients and server run on the same machine. 
-* Clients access the server using STOMP over TCP. 
-* Producers send non-persistent messages
-* Messages contain a 20 byte payload 
-* Producers do not wait for a broker ack before sending the next message. 
-* Consumers use auto ack.
-
-The following benchmark results were run on a Mac Pro with:
-
-* Dual socket 3 ghz Quad-Core Intel Xeon (8 total cores) with 12 MB L2
-  cache per processor .
-* 8 GB of RAM
-
-### Queue Cases
-
-* 1 producer sending to 1 consumer via 1 queue can hit rates of 250,000
-  messages/second.
-* 8 producers sending to 8 consumers via 8 queues can hit rates of 540,000
-  messages/second.
-* 1 producer sending to 10 consumers via 1 queue can hit rates of 230,000
-  messages/second.
-* 10 producers sending to 1 consumers via 1 queue can hit rates of 280,000
-  messages/second.
-
-### Topic Cases
-
-Rates reported are the total consumer rate.
-
-* 1 producer sending to 1 consumer via 1 topic can hit a rates of 420,000
-  messages/second.
-* 8 producers sending to 8 consumers via 8 topics can hit rates of 810,000
-  messages/second.
-* 10 producer sending to 10 consumers via 1 topic can hit rates of
-  1,3000,000 messages/second.
-
-## Scaling Characteristics
-
-There are many different extreme ways that a messaging system can be used. 
-Some folks like to:
-
-* connect a large number of clients.
-* hold a large number of messages in their queues.  
-* move around large messages.
-
-Apollo aims to support all those usage scenarios.
-
-### Scaling the Number of Connected Clients
-
-Apollo uses non blocking IO and a reactor thread model. This means that a
-running broker uses a constant number of threads no matter how many clients
-are connected to it.
-
-### Scaling the Number of Queued Messages
-
-Queues will swap messages out of memory when there are no consumers that
-are likely to need the message soon. Once a message is swapped, the queue
-will replace the message with a reference pointer. When the queue builds a
-large number (more than 10,000) of these swapped out reference pointers,
-they then get consolidated into a single "range entry" and the pointers are
-dropped. 
-
-When a consumer comes along that needs some of the swapped out messages,
-it will convert previously consolidated "range entries" to a set of
-reference pointers. The next reference pointers that the consumer is
-interested in get swapped back as regular messages. This allows Apollo's
-queues to hold millions of messages without much impact on JVM memory
-usage.
-
-### Scaling the Message Size
-
-Big messages don't even make it into the JVM's memory management. The
-contents of big messages received and sent using buffers allocated in a
-memory mapped file. This allows the OS to efficiently manage swapping those
-large messages to disk and it avoids churning the eden garbage collection
-generation in the JVM. If 1 producer is sending 100 MB messages to a
-consumer the JVM will not report any significant memory usage.
+* [Architecture](apollo-website/src/architecture.md)
+* [Performance and Scalability](apollo-website/src/performance-scaling.md)
 
 ## Building the Source Code
 
-TODO
+Prerequisites:
+
+* [Maven >= 2.2.1](http://maven.apache.org/download.html)
+* [Java JDK >= 1.6](http://java.sun.com/javase/downloads/widget/jdk6.jsp)
+
+Then run:
+
+    mvn install
 
 ## Quick Start 
 
-### Running an Apollo Broker
+We are still working on creating a binary distribution. Once that's created
+we will update these instructions to work off that distribution. Until then,
+they will be based on a built source distribution.
 
-TODO
+### Running an Apollo Broker
 
-### Running Examples
+A broker with a web based admin interface will be started by using the the
+Scala REPL console.
 
-TODO
+    $ cd apollo-web
+    $ mvn -o scala:console 
+    ... [output ommitted for brevity]
+    scala> val main = org.apache.activemq.apollo.web.Main
+    ... [output ommitted for brevity]
+    scala> main run
+    ... [output ommitted for brevity]
+    Web interface available at: http://localhost:8080/
+
+You can point your web browser at http://localhost:8080/ to explore the
+management structure of the broker. Additional status objects will become
+visible once there are connected client which cause connections and
+destination management objects to be created.
 
-## Architectural Changes
+### Running Examples
 
-Apollo started as an experiment to see what it would take to make ActiveMQ
-work better on machines with higher core counts. It has resulted in broker
-that is much more deterministic, stable, and scaleable.
-
-The major fundamental architectural changes it brings are:
-
-* Reactor Based Thread Model
-* Scala 2.8 Implementation
-* Protocol Agnostic
-* REST Based Management
-
-### Reactor Based Thread Model
-
-Apollo uses [HawtDispatch][] to implement the sever using a multi-threaded
-non-blocking variation of the [reactor][] design pattern. HawtDispatch is a
-Java clone of [libdispatch][] (aka [Grand Central Dispatch][gcd]). It uses a
-fixed sized thread pool and only executes non-blocking tasks on those
-threads.
-
-[reactor]:http://en.wikipedia.org/wiki/Reactor_pattern
-[libdispatch]:http://en.wikipedia.org/wiki/Libdispatch
-[HawtDispatch]:http://hawtdispatch.fusesource.org/
-[gcd]:http://images.apple.com/macosx/technology/docs/GrandCentral_TB_brief_20090903.pdf
-
-
-The thread model allows Apollo to reach very high levels of scaleability and
-efficiency, but it places a huge restriction on the developer: all the tasks
-it executes must be non-blocking and ideally lock-free and wait free. This
-means that the previous ActiveMQ broker architecture had to go through a
-major overhaul. All synchronous broker interfaces had to be changed
-so that they would instead return results via asynchronous callbacks.
-
-### Scala 2.8 Implementation
-
-Even though Apollo started as a fork of ActiveMQ 5.x, the new reactor design
-restrictions required major changes from the network IO handling, to the
-flow control design, all the way to the Store interfaces. Scala provided a
-much more concise way to express callbacks, namely by it's support for
-partial functions and closures.
-
-### Protocol Agnostic
-
-ActiveMQ has supported multiple protocols for many years, but under the
-covers what it's doing is converting all the protocols to use the
-[OpenWire][] messaging protocol. Naturally, this approach is not optimal if
-you want to efficiently support other protocols and not be limited by what
-OpenWire can support.
-
-[OpenWire]:http://activemq.apache.org/openwire.html
-
-The Apollo server is much more modular and protocol agnostic. All protocols
-are equal and are built as a plugin to the the broker which just make use of
-exposed broker services for routing, flow control, queueing services etc.
-For example, this means that messages will be persisted in a Store in the
-original protocol's encoding. There is no protocol conversion occurring
-under the covers unless it is required.
-
-### REST Based Management
-
-ActiveMQ choose to exposed it's management interfaces via JMX. JMX was a
-natural choice since ActiveMQ is Java based, but JMX has a couple of
-serious limitations:
-
-* No cross language support
-* Not scaleable for exposing many objects. Registering and unregistering
-  management objects in JMX can become a bottle neck.
-* Rich data types are hard to expose
-
-Apollo exposes a rich and detailed state of the sever using REST based JSON
-or XML services.
-
-* A management client can easily be implemented in any language.
-* There is very little management overhead since there is no special
-  registration with the management system. The JAX-RS based management web
-  application knows how to navigate the internal structure of a broker to
-  access all the need status and statistics.
+A stomp client will be started by using the the Scala 
+repl console.
 
+    $ cd apollo-stomp
+    $ mvn -o scala:console 
+    ... [output ommitted for brevity]
+    scala> val client = org.apache.activemq.apollo.stomp.perf.StompLoadClient        
        
+    client: org.apache.activemq.apollo.stomp.perf.StompLoadClient.type = 
+    --------------------------------------
+    StompLoadClient Properties
+    --------------------------------------
+    uri              = stomp://127.0.0.1:61613
+    destinationType  = queue
+    destinationCount = 1
+    sampleInterval   = 5000
+
+    --- Producer Properties ---
+    producers        = 1
+    messageSize      = 1024
+    persistent       = false
+    syncSend         = false
+    useContentLength = true
+    producerSleep    = 0
+    headers          = List()
+
+    --- Consumer Properties ---
+    consumers        = 1
+    consumerSleep    = 0
+    ack              = auto
+    selector         = null
+
+The above creates a client variable which allows you to customize all the
+displayed properties. Those properties control how the client applies load
+to the STOMP broker.  You could change the client configuration so that
+it uses messages with 20 byte contents and send and receive on topics instead
+of queues:
+
+    scala> client.messageSize = 20
+    scala> client.destinationType = "topic"
+
+
+Once you are happy with the client configuration, you just run it and wait
+for it to report back the producer and consumer throughput rates.
+
+    scala> client.run                                                        
+    =======================
+    Press ENTER to shutdown
+    =======================
+
+    Producer rate: 155,783.906 per second, total: 778,960
+    Consumer rate: 154,345.969 per second, total: 771,770
+    Producer rate: 165,831.141 per second, total: 1,608,210
+    Consumer rate: 165,798.734 per second, total: 1,600,858
 
 



Mime
View raw message