flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Avinash Shahdadpuri <avinashp...@gmail.com>
Subject Question on flume log4j appender and avrosource
Date Thu, 01 Sep 2011 22:03:38 GMT

We have recently started using flume.

We have 20 servers behind a load balancer and want to see if we can avoid
running flume node on all of them.

We are looking at an option of using the flume log4j appender & avrosource &
dedicated flume nodes (machines just running flume)

1. We can use flume log4j appender to stream logs to a dedicated flume node
running flume agent/collector. In this case, if the flume node goes down, we
would lose the messages.
2.  The other option is to flume log4j appender to stream logs on the same
machine. In this case we would need an agent on flume node to read the
remote server. The avrosource agent doesn't seem to be able  to read from
remote machine? Is there something else we can do here.

Has anyone come across this and do you have any recommendations to handle

Please let me know.



View raw message