flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gonzalo Herreros <gherre...@gmail.com>
Subject Re: Hbase Sink
Date Tue, 10 Nov 2015 14:55:29 GMT
I started by extending RegexHbaseEventSerializer, so I didn't have to
implement the full HbaseEventSerializer

Override (and call parent) configure(Context context) to get your custom
configuration properties

In


public List<Row> getActions() throws FlumeException

where you conver the

On 7 November 2015 at 11:07, Rani Yaroshinski <rani.yaroshinski@gmail.com>
wrote:

> Any pointers to the code, as sample ?
>
> On Sat, Nov 7, 2015 at 12:45 PM, Gonzalo Herreros <gherreros@gmail.com>
> wrote:
>
>> I did a custom serializer that parses the event an json and the top level
>> properties become columns inside a configurable column family.
>> I also have a custom property to configure which fields make up the
>> composite key (which I salt based on the number of regions).
>>
>> It shouldn't be too hard having a property that maps input fields into
>> hbase cf:column names (kinda like Hive does when you map an HBase table)
>>
>> Regards,
>> Gonzalo
>>
>> On 7 November 2015 at 09:17, Rani Yaroshinski <rani.yaroshinski@gmail.com
>> > wrote:
>>
>>> Hi,
>>>
>>> It seems a little weird, but it looks like the most simple sink you
>>> would like for hbase,
>>> which allows you to set key, and columns generally is missing.
>>> You need to write specific custom serializer for any implementation
>>> which would make
>>> a sense for any production system.
>>> Anyone with pointers to rellevant custom implementations which allows
>>> configuration for key
>>> type and value, and also for column family data.
>>>
>>> Regards,
>>> Rani
>>>
>>
>>
>

Mime
View raw message