Mailing-List: contact user-help@avro.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@avro.apache.org
MIME-Version: 1.0
In-Reply-To: <1488174530704-4036890.post@n3.nabble.com>
References: <1487867813856-4036886.post@n3.nabble.com> <CAKBEjTtZqKsnuWud9P=o6=ScKmjz0fdgGHGnp7-k+sHyNnrxaQ@mail.gmail.com>
 <1488174530704-4036890.post@n3.nabble.com>
From: Johannes Schulte <johannes.schulte@gmail.com>
Date: Tue, 28 Feb 2017 13:57:32 +0100
Message-ID: <CAKBEjTvs=2q_zYUrxHAr8sARLzLJhc1grJCBxMAcGcDoXYdpFw@mail.gmail.com>
Subject: Re: Avro with MultipleInputs in the new API
To: user@avro.apache.org
Content-Type: multipart/alternative; boundary=94eb2c07aadc4ad88b054996bdac
archived-at: Tue, 28 Feb 2017 12:57:44 -0000

--94eb2c07aadc4ad88b054996bdac
Content-Type: text/plain; charset=UTF-8

I suggest you use Mapper<AvroKey<Object>,NullWritable> or even
Mapper<AvroKey<SpecificRecord>,NullWritable>

You can than decide on sorting key by checking the datum() of the avro key
with instanceof or with SpecificRecord#getSchema()#getName in the latter
case.
The schema for the map output (AvroJob#setMapOutputKeySchema()) will be the
union again and the key depends on your use case. The sorting by default
should use the provided key for sorting the (key,[values])


On Mon, Feb 27, 2017 at 6:48 AM, Zephod <zephod@tlen.pl> wrote:

> Dear Johannes,
> Thank you for your suggestion :) I know the types in advance (data model is
> specific). Where could I read how the union type works? Are there any
> examples e.g.:
> -what kind of types should I use for type parameters of the mapper/reducer?
> -how the values will be sorted in the shuffle phase?
> -how do I check what type are the values I'm iterating over in the reducer?
>
>
>
>
>
> --
> View this message in context: http://apache-avro.679487.n3.
> nabble.com/Avro-with-MultipleInputs-in-the-new-API-tp4036886p4036890.html
> Sent from the Avro - Users mailing list archive at Nabble.com.
>

--94eb2c07aadc4ad88b054996bdac
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">I suggest you use Mapper&lt;AvroKey&lt;Object&gt;,NullWrit=
able&gt; or even Mapper&lt;AvroKey&lt;SpecificRecord&gt;,NullWritable&gt;<d=
iv><br></div><div>You can than decide on sorting key by checking the datum(=
) of the avro key with instanceof or with SpecificRecord#getSchema()#getNam=
e in the latter case.=C2=A0</div><div>The schema for the map output (AvroJo=
b#setMapOutputKeySchema()) will be the union again and the key depends on y=
our use case. The sorting by default should use the provided key for sortin=
g the (key,[values])</div><div><br></div><div><br></div></div><div class=3D=
"gmail_extra"><br><div class=3D"gmail_quote">On Mon, Feb 27, 2017 at 6:48 A=
M, Zephod <span dir=3D"ltr">&lt;<a href=3D"mailto:zephod@tlen.pl" target=3D=
"_blank">zephod@tlen.pl</a>&gt;</span> wrote:<br><blockquote class=3D"gmail=
_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:=
1ex">Dear Johannes,<br>
Thank you for your suggestion :) I know the types in advance (data model is=
<br>
specific). Where could I read how the union type works? Are there any<br>
examples e.g.:<br>
-what kind of types should I use for type parameters of the mapper/reducer?=
<br>
-how the values will be sorted in the shuffle phase?<br>
-how do I check what type are the values I&#39;m iterating over in the redu=
cer?<br>
<br>
<br>
<br>
<br>
<br>
--<br>
View this message in context: <a href=3D"http://apache-avro.679487.n3.nabbl=
e.com/Avro-with-MultipleInputs-in-the-new-API-tp4036886p4036890.html" rel=
=3D"noreferrer" target=3D"_blank">http://apache-avro.679487.n3.<wbr>nabble.=
com/Avro-with-<wbr>MultipleInputs-in-the-new-API-<wbr>tp4036886p4036890.htm=
l</a><br>
<div class=3D"HOEnZb"><div class=3D"h5">Sent from the Avro - Users mailing =
list archive at Nabble.com.<br>
</div></div></blockquote></div><br></div>

--94eb2c07aadc4ad88b054996bdac--