Mailing-List: contact flume-user-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: flume-user@incubator.apache.org
Received-SPF: pass (nike.apache.org: domain of avinashpuri@gmail.com
 designates 209.85.161.47 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CAOQYySBBT1O+jge_+qWtTXa1GSwi=8rkXf7j29PzJWKePeXhZQ@mail.gmail.com>
References: 
 <CAOQYySBBT1O+jge_+qWtTXa1GSwi=8rkXf7j29PzJWKePeXhZQ@mail.gmail.com>
Date: Thu, 1 Sep 2011 15:03:38 -0700
Message-ID: 
 <CAOQYySCXss_VYbbzsSa+sgKRbV_SxkNo056fgEcLyi7=w-6vVA@mail.gmail.com>
Subject: Question on flume log4j appender and avrosource
From: Avinash Shahdadpuri <avinashpuri@gmail.com>
To: flume-user@incubator.apache.org
Content-Type: multipart/alternative; boundary=001517493c14bee4e504abe86a00

--001517493c14bee4e504abe86a00
Content-Type: text/plain; charset=ISO-8859-1

Hi,

We have recently started using flume.

We have 20 servers behind a load balancer and want to see if we can avoid
running flume node on all of them.

We are looking at an option of using the flume log4j appender & avrosource &
dedicated flume nodes (machines just running flume)

1. We can use flume log4j appender to stream logs to a dedicated flume node
running flume agent/collector. In this case, if the flume node goes down, we
would lose the messages.
2.  The other option is to flume log4j appender to stream logs on the same
machine. In this case we would need an agent on flume node to read the
remote server. The avrosource agent doesn't seem to be able  to read from
remote machine? Is there something else we can do here.

Has anyone come across this and do you have any recommendations to handle
this.

Please let me know.

Thanks,

Avinash

--001517493c14bee4e504abe86a00
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<div class=3D"gmail_quote">Hi,<div><br></div><div>We have recently started =
using flume.</div><div><br></div><div>We have 20 servers behind a load bala=
ncer and want to see if we can avoid running flume node on all of them.=A0<=
/div>
<div><br></div><div>We are looking at an option of using the flume log4j ap=
pender &amp; avrosource &amp; dedicated flume nodes (machines just running =
flume)</div>
<div><br></div><div>1. We can use flume log4j appender to stream logs to a =
dedicated flume node running flume agent/collector. In this case, if the fl=
ume node goes down, we would lose the messages.</div><div>2. =A0The other o=
ption is to flume log4j appender to stream logs on the same machine. In thi=
s case we would need an agent on flume node to read the remote server. The =
avrosource agent doesn&#39;t seem to be able =A0to read from remote machine=
? Is there something else we can do here.</div>

<div><br></div><div>Has anyone come across this and do you have any recomme=
ndations to handle this.</div><div><br></div><div>Please let me know.</div>=
<div><br></div><div>Thanks,</div><div><br></div><font color=3D"#888888"><di=
v>
Avinash</div><div>
<br></div>
</font></div><br>

--001517493c14bee4e504abe86a00--