camel-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From elaufis <>
Subject AWS Kinesis Consumer performance improvement
Date Fri, 17 Mar 2017 10:06:00 GMT
I have narrowed down a performance bottleneck in the Processor I implemented
in a SEDA route using the AWS Kinesis component.

I tested against an "empty" Processor (i.e. that doesn't do anything) and
got the following metrics:

*For the "empty" Processor*:11:10| INFO | 107 |
type=METER, name=kinesis-records-consumed, *count=140,
16133, m1=1.6458279659887582, m5=0.43339880218285814,
m15=0.15173246327279963, rate_unit=events/second

*For the "actual" Processor*: 11:15| INFO | 107 |
type=METER, name=kinesis-records-consumed, *count=20,
mean_rate=0.3333031357470416*, m1=0.17909216046832732,
m5=0.058841979331650814, m15=0.02131567762219331, rate_unit=events/second

So, the bottleneck definitely seems to be in the "actual" Processor. 

I have increased the number of shards on the streamas well as tweaked a host
of other parameters in order to improve performance with no real effect.

I now want to try using a custom ScheduledExecutorService to use multiple
threads for the consumer. 

Is there an example of how to do this?

Also, concerning the number of records in the metrics above, how does the
Camel component determine how many records to read off the stream on the
first read? I'd expect them to be similar and maybe different on subsequent

View this message in context:
Sent from the Camel - Users mailing list archive at

View raw message