Mailing-List: contact common-user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: common-user@hadoop.apache.org
Received-SPF: pass (nike.apache.org: domain of wget.null@googlemail.com
 designates 74.125.82.176 as permitted sender)
Content-Type: text/plain; charset=iso-8859-1
Mime-Version: 1.0 (Apple Message framework v1257)
Subject: Re: Feedback on real world production experience with Flume
From: alo alt <wget.null@googlemail.com>
In-Reply-To: 
 <CANhO-s2xmKoj=LhnXX1GsZ2o4ejNv4J_w_P17oSV_F71Hu+YTQ@mail.gmail.com>
Date: Sat, 21 Apr 2012 17:42:51 +0200
Content-Transfer-Encoding: quoted-printable
Message-Id: <EFAAB2C6-FD64-4488-BC09-A4063FFB5038@gmail.com>
References: 
 <6ADFD36BCF7B7A47B17D66952CCF285A08E011FF@mbx023-w1-ca-7.exch023.domain.local>
 <CANhO-s2xmKoj=LhnXX1GsZ2o4ejNv4J_w_P17oSV_F71Hu+YTQ@mail.gmail.com>
To: common-user@hadoop.apache.org

We decided NO product and vendor advertising on apache mailing lists!=20
I do not understand why you'll put that closed source stuff from your =
employe in the room. It has nothing to do with flume or the use cases!

--
Alexander Lorenz
http://mapredit.blogspot.com

On Apr 21, 2012, at 4:06 PM, M. C. Srivas wrote:

> Karl,
>=20
> since you did ask for alternatives,  people using MapR prefer to use =
the
> NFS access to directly deposit data (or access it).  Works seamlessly =
from
> all Linuxes, Solaris, Windows, AIX and a myriad of other legacy =
systems
> without having to load any agents on those machines. And it is fully
> automatic HA
>=20
> Since compression is built-in in MapR, the data gets compressed coming =
in
> over NFS automatically without much fuss.
>=20
> Wrt to performance,  can get about 870 MB/s per node if you have =
10GigE
> attached (of course, with compression, the effective throughput will
> surpass that based on how good the data can be squeezed).
>=20
>=20
> On Fri, Apr 20, 2012 at 3:14 PM, Karl Hennig <khennig@baynote.com> =
wrote:
>=20
>> I am investigating automated methods of moving our data from the web =
tier
>> into HDFS for processing, a process that's performed periodically.
>>=20
>> I am looking for feedback from anyone who has actually used Flume in =
a
>> production setup (redundant, failover) successfully.  I understand it =
is
>> now being largely rearchitected during its incubation as Apache =
Flume-NG,
>> so I don't have full confidence in the old, stable releases.
>>=20
>> The other option would be to write our own tools.  What methods are =
you
>> using for these kinds of tasks?  Did you write your own or does Flume =
(or
>> something else) work for you?
>>=20
>> I'm also on the Flume mailing list, but I wanted to ask these =
questions
>> here because I'm interested in Flume _and_ alternatives.
>>=20
>> Thank you!
>>=20
>>=20