incubator-chukwa-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eric Yang <eric...@gmail.com>
Subject Re: multiple threads/HttpConnector from ChukwaAgent
Date Thu, 26 Jan 2012 20:03:45 GMT
Hi Corbin,

This is by design.  We are concatenating all data streams into in
memory queue on the agent, and establish only one http connection to
collector.  This is for horizontal scalability that we can support
more machines.  At the same time, it also ensures that agent can write
more data per HTTP post to reduce overhead of HTTP headers and
connection handshakes.

regards,
Eric

On Thu, Jan 26, 2012 at 11:51 AM, Corbin Hoenes <corbin@tynt.com> wrote:
> I am trying to do some real-time processing of the data coming into my chukwa pipeline
and notice that using a single agent I don't seem to be getting very many servlets handling
the requests. Peeking at the ChukwaAgent code it looks like the agents are limited to a single
HttpConnector.
>
> Is this by design or am I off-base in my analysis of how it works?
>
>

Mime
View raw message