camel-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From SimonH <>
Subject Losing Messages in SEDA endpoint with polling consumer
Date Mon, 20 Aug 2012 13:44:22 GMT
I'm using Camel 2.8.5, and am finding that messages sent to SEDA endpoints
that have polling consumers are sometimes lost - when sending more than 1000

My example scenario is shown below.  I have a timer endpoint to kick off my
experiment.  The route then proceeds as follows:

- Generate a list of 2000 Strings out of nowhere
- Split the message into 2000 separate messages
- For each message:
-- inOnly to SEDA queue 1
-- inOnly to SEDA queue 2
-- end
- use a ConsumerTemplate to poll all messages from SEDA queue 1 until no
more messages are found; count these messages
- display the total count of messages for queue 1

Separately, I have another route to count the messages to queue 2 in an
event-driven manner.  The count for this queue is shown alongside the count
for queue 1.

The ConsumerTemplate always finds at least 1000 messages on queue 1, and
sometimes more.  But it never finds all 2000.  Typical is very close to
1000.  The event-driven counter always finds all 2000.  Why does the polling
method not find all messages?

Some extra info:
- if I start with more than 2000 messages, the polling consumer still
generally finds around 1000 messages; the event-driven consumer finds them
all still
- if I start with less than 1000 messages, both consumers find all their
- if I don't define a 'size' parameter on my polling consumer's queue, the
behaviour does not change


package com.example;

import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.atomic.AtomicInteger;

import org.apache.camel.ConsumerTemplate;
import org.apache.camel.Exchange;
import org.apache.camel.Processor;
import org.apache.camel.builder.RouteBuilder;
import org.springframework.beans.factory.annotation.Autowired;

public class ConsumerTestRouteBuilder extends RouteBuilder {

    private static final String EVENT_DRIVEN_Q = "seda:eventDrivenQ";
    private static final String POLLING_Q = "seda:pollingQ?size=10000";

    private ConsumerTemplate consumer;
 private AtomicInteger eventDrivenCount = new AtomicInteger();
 private AtomicInteger pollingCount = new AtomicInteger();

 public void configure() throws Exception {
         .process(new Processor() {
                public void process(Exchange exchange) throws Exception {
                    System.out.println("Start of test");
                    List<String> answer = new ArrayList<String>();
                    for(int i=0;i<2000;i++) {
            .process(new Processor() {
                public void process(Exchange exchange) throws Exception {
                    while(consumer.receive(POLLING_Q, 7000) !=null ) {
            .process(new Processor() {
                public void process(Exchange exchange) throws Exception {
                    System.out.println("Event-driven consumptions =
                    System.out.println("Polling consumptions =
                    System.out.println("End of test");
         .process(new Processor() {
                public void process(Exchange exchange) throws Exception {

I configure using Spring:

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns=""

 <bean id="consumerTestRouteBuilder"
class="com.example.ConsumerTestRouteBuilder" />

    <context:annotation-config />

 <camel:camelContext id="camelContext"
        <camel:consumerTemplate id="consumer" />
  <camel:routeBuilder ref="consumerTestRouteBuilder" />



View this message in context:
Sent from the Camel - Users mailing list archive at

View raw message