Mailing-List: contact user-help@flume.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@flume.apache.org
MIME-Version: 1.0
In-Reply-To: 
 <CACQ=AYuPaxxukaC_NqS4Xt=7Lgc3pAMVeZOz8Ro3Pzq8vf4t1g@mail.gmail.com>
References: 
 <CACQ=AYvRQneKXxAuJCrESoEmzO0wo=6M_BMimhgPcgpbS62ReQ@mail.gmail.com>
	<CAM-G-DXDMTGpUCvO3uP-N=zzF2VP7QLYbT4fmTUqzQTF_18wDQ@mail.gmail.com>
	<CACQ=AYuPaxxukaC_NqS4Xt=7Lgc3pAMVeZOz8Ro3Pzq8vf4t1g@mail.gmail.com>
Date: Tue, 10 Nov 2015 14:55:29 +0000
Message-ID: 
 <CAM-G-DVZ5fojeKa-395nWbeJHgfJxPm82idX=5VhLGAQ5VM9sg@mail.gmail.com>
Subject: Re: Hbase Sink
From: Gonzalo Herreros <gherreros@gmail.com>
To: user <user@flume.apache.org>
Content-Type: multipart/alternative; boundary=047d7b3a8764983cc0052430e6cf

--047d7b3a8764983cc0052430e6cf
Content-Type: text/plain; charset=UTF-8

I started by extending RegexHbaseEventSerializer, so I didn't have to
implement the full HbaseEventSerializer

Override (and call parent) configure(Context context) to get your custom
configuration properties

In


public List<Row> getActions() throws FlumeException

where you conver the

On 7 November 2015 at 11:07, Rani Yaroshinski <rani.yaroshinski@gmail.com>
wrote:

> Any pointers to the code, as sample ?
>
> On Sat, Nov 7, 2015 at 12:45 PM, Gonzalo Herreros <gherreros@gmail.com>
> wrote:
>
>> I did a custom serializer that parses the event an json and the top level
>> properties become columns inside a configurable column family.
>> I also have a custom property to configure which fields make up the
>> composite key (which I salt based on the number of regions).
>>
>> It shouldn't be too hard having a property that maps input fields into
>> hbase cf:column names (kinda like Hive does when you map an HBase table)
>>
>> Regards,
>> Gonzalo
>>
>> On 7 November 2015 at 09:17, Rani Yaroshinski <rani.yaroshinski@gmail.com
>> > wrote:
>>
>>> Hi,
>>>
>>> It seems a little weird, but it looks like the most simple sink you
>>> would like for hbase,
>>> which allows you to set key, and columns generally is missing.
>>> You need to write specific custom serializer for any implementation
>>> which would make
>>> a sense for any production system.
>>> Anyone with pointers to rellevant custom implementations which allows
>>> configuration for key
>>> type and value, and also for column family data.
>>>
>>> Regards,
>>> Rani
>>>
>>
>>
>

--047d7b3a8764983cc0052430e6cf
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">I started by extending=C2=A0RegexHbaseEventSerializer, so =
I didn&#39;t have to implement the full=C2=A0HbaseEventSerializer<div><br><=
/div><div>Override (and call parent) configure(Context context) to get your=
 custom configuration properties</div><div><br></div><div>In</div><div><br>=
</div><div><br></div><div>public List&lt;Row&gt; getActions() throws FlumeE=
xception<br></div><div><br></div><div>where you conver the=C2=A0</div></div=
><div class=3D"gmail_extra"><br><div class=3D"gmail_quote">On 7 November 20=
15 at 11:07, Rani Yaroshinski <span dir=3D"ltr">&lt;<a href=3D"mailto:rani.=
yaroshinski@gmail.com" target=3D"_blank">rani.yaroshinski@gmail.com</a>&gt;=
</span> wrote:<br><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .=
8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir=3D"ltr">Any point=
ers to the code, as sample ?</div><div class=3D"HOEnZb"><div class=3D"h5"><=
div class=3D"gmail_extra"><br><div class=3D"gmail_quote">On Sat, Nov 7, 201=
5 at 12:45 PM, Gonzalo Herreros <span dir=3D"ltr">&lt;<a href=3D"mailto:ghe=
rreros@gmail.com" target=3D"_blank">gherreros@gmail.com</a>&gt;</span> wrot=
e:<br><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-l=
eft:1px #ccc solid;padding-left:1ex"><div dir=3D"ltr"><div>I did a custom s=
erializer that parses the event an json and the top level properties become=
 columns inside a configurable column family.<br></div><div>I also have a c=
ustom property to configure which fields make up the composite key (which I=
 salt based on the number of regions).</div><div><br></div><div>It shouldn&=
#39;t be too hard having a property that maps input fields into hbase cf:co=
lumn names (kinda like Hive does when you map an HBase table)</div><div><br=
></div><div>Regards,</div><div>Gonzalo</div></div><div><div><div class=3D"g=
mail_extra"><br><div class=3D"gmail_quote">On 7 November 2015 at 09:17, Ran=
i Yaroshinski <span dir=3D"ltr">&lt;<a href=3D"mailto:rani.yaroshinski@gmai=
l.com" target=3D"_blank">rani.yaroshinski@gmail.com</a>&gt;</span> wrote:<b=
r><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:=
1px #ccc solid;padding-left:1ex"><div dir=3D"ltr">Hi,<div><br></div><div>It=
 seems a little weird, but it looks like the most simple sink you would lik=
e for hbase,</div><div>which allows you to set key, and columns generally i=
s missing.</div><div>You need to write specific custom serializer for any i=
mplementation which would make</div><div>a sense for any production system.=
</div><div>Anyone with pointers to rellevant custom implementations which a=
llows configuration for key</div><div>type and value, and also for column f=
amily data.</div><div><br></div><div>Regards,</div><div>Rani</div></div>
</blockquote></div><br></div>
</div></div></blockquote></div><br></div>
</div></div></blockquote></div><br></div>

--047d7b3a8764983cc0052430e6cf--