Mailing-List: contact user-help@avro.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@avro.apache.org
Received-SPF: pass (athena.apache.org: domain of static.void.dev@gmail.com
 designates 209.85.160.52 as permitted sender)
From: Mark <static.void.dev@gmail.com>
Content-Type: multipart/alternative;
 boundary="Apple-Mail=_AC6F04BE-306C-4EB1-A093-1B8680091F20"
Message-Id: <5232CB91-A3A9-48FF-9060-24073622D09D@gmail.com>
Mime-Version: 1.0 (Mac OS X Mail 6.3 \(1503\))
Subject: Re: Is Avro right for me?
Date: Sun, 26 May 2013 09:39:02 -0700
References: <5E1C8602-0CFE-4E42-A4BF-FF9A009AAA7D@gmail.com>
 <CAGHyZ6LQRtucasJJtMxea7xmsq_mGb-xBLEkTVSk7vLR=9bSwA@mail.gmail.com>
To: user@avro.apache.org
In-Reply-To: 
 <CAGHyZ6LQRtucasJJtMxea7xmsq_mGb-xBLEkTVSk7vLR=9bSwA@mail.gmail.com>


--Apple-Mail=_AC6F04BE-306C-4EB1-A093-1B8680091F20
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=us-ascii

Yes our central server would be Hadoop.=20

Exactly how would this work with flume? Would I write Avro to a file =
source which flume would then ship over to one of our collectors  or is =
there a better/native way? Would I have to include the schema in each =
event? FYI we would be doing this primarily from a rails application.

Does anyone ever use Avro with a message bus like RabbitMQ?=20

On May 23, 2013, at 9:16 PM, Sean Busbey <busbey@cloudera.com> wrote:

> Yep. Avro would be great at that (provided your central consumer is =
Avro friendly, like a Hadoop system).  Make sure that all of your =
schemas have default values defined for fields so that schema evolution =
will be easier in the future.
>=20
>=20
> On Thu, May 23, 2013 at 4:29 PM, Mark <static.void.dev@gmail.com> =
wrote:
> We're thinking about generating logs and events with Avro and shipping =
them to a central collector service via Flume. Is this a valid use case?
>=20
>=20
>=20
>=20
> --=20
> Sean


--Apple-Mail=_AC6F04BE-306C-4EB1-A093-1B8680091F20
Content-Transfer-Encoding: quoted-printable
Content-Type: text/html;
	charset=us-ascii

<html><head><meta http-equiv=3D"Content-Type" content=3D"text/html =
charset=3Dus-ascii"></head><body style=3D"word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">Yes =
our central server would be Hadoop.&nbsp;<div><br></div><div>Exactly how =
would this work with flume? Would I write Avro to a file source which =
flume would then ship over to one of our collectors &nbsp;or is there a =
better/native way? Would I have to include the schema in each event? FYI =
we would be doing this primarily from a rails =
application.</div><div><br></div><div>Does anyone ever use Avro with a =
message bus like =
RabbitMQ?&nbsp;</div><div><br></div><div><div><div><div>On May 23, 2013, =
at 9:16 PM, Sean Busbey &lt;<a =
href=3D"mailto:busbey@cloudera.com">busbey@cloudera.com</a>&gt; =
wrote:</div><br class=3D"Apple-interchange-newline"><blockquote =
type=3D"cite"><div dir=3D"ltr">Yep. Avro would be great at that =
(provided your central consumer is Avro friendly, like a Hadoop system). =
&nbsp;Make sure that all of your schemas have default values defined for =
fields so that schema evolution will be easier in the future.<div =
class=3D"gmail_extra">

<br><br><div class=3D"gmail_quote">On Thu, May 23, 2013 at 4:29 PM, Mark =
<span dir=3D"ltr">&lt;<a href=3D"mailto:static.void.dev@gmail.com" =
target=3D"_blank">static.void.dev@gmail.com</a>&gt;</span> =
wrote:<br><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 =
.8ex;border-left:1px #ccc solid;padding-left:1ex">

We're thinking about generating logs and events with Avro and shipping =
them to a central collector service via Flume. Is this a valid use =
case?<br>
<br>
</blockquote></div><br><br clear=3D"all"><div><br></div>-- <br>Sean<br>
</div></div>
</blockquote></div><br></div></div></body></html>=

--Apple-Mail=_AC6F04BE-306C-4EB1-A093-1B8680091F20--