flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robert Metzger <rmetz...@apache.org>
Subject Re: Write Stream to HBase
Date Wed, 20 May 2015 09:15:51 GMT
There is this pending pull request which is addressing exactly the issues
I've mentioned (wrong naming, private method):
https://github.com/apache/flink/pull/521

I'll see whats blocking the PR ...

On Wed, May 20, 2015 at 11:11 AM, Robert Metzger <rmetzger@apache.org>
wrote:

> Maybe we can also use the Batch HBase OutputFormat.
>
> In the DataStream API there is a private method:
>
> private DataStreamSink<OUT> writeToFile(OutputFormat<OUT> format, long millis)
{
>
> which seems to allow batch output formats.
> The naming of the method seems weird because its called "toFile" but its
> expecting an OutputFormat instead of a FileOutputFormat.
>
> Maybe its worth trying to see how far we can get with this method.
>
>
> On Wed, May 20, 2015 at 11:00 AM, Robert Metzger <rmetzger@apache.org>
> wrote:
>
>> Hi,
>> I agree with Hilmi, Flavio's examples are for batch.
>>
>> I'm not aware of a StreamingHBaseSink for Flink yet.
>> I'll filed a JIRA for the feature request:
>> https://issues.apache.org/jira/browse/FLINK-2055
>>
>> Are you interested in implementing this?
>>
>> On Wed, May 20, 2015 at 10:50 AM, Hilmi Yildirim <
>> hilmi.yildirim@neofonie.de> wrote:
>>
>>>  Thank you Flavio,
>>> these are examples for Batch Processing. But I want to write a
>>> continuous stream into an HBase within a StreamExecutionEnvironment instead
>>> of a ExecutionEnvironment.
>>>
>>> Best Regards,
>>> Hilmi
>>>
>>>
>>> Am 20.05.2015 um 10:42 schrieb Flavio Pompermaier:
>>>
>>> I've added an example of HBase writing
>>> at flink-staging/flink-hbase/src/test/java/org/apache/flink/addons/hbase/example/HBaseWriteExample.java.
>>>
>>> Otherwise you can look at these 2 URLs:
>>>
>>>    -
>>>    https://github.com/fpompermaier/flink/blob/hbaseOutExample/flink-staging/flink-hbase/src/test/java/org/apache/flink/addons/hbase/example/HBaseWriteExample.java
>>>    -
>>>    https://github.com/apache/flink/blob/57615aaa19e9933e43ed0431a78dd231bf98b103/flink-staging/flink-hbase/src/test/java/org/apache/flink/addons/hbase/example/HBaseWriteExample.java
>>>
>>> Best,
>>>  Flavio
>>>
>>> On Wed, May 20, 2015 at 10:16 AM, Hilmi Yildirim <
>>> hilmi.yildirim@neofonie.de> wrote:
>>>
>>>> Hi,
>>>> I want to write a stream continuously into an HBase. For example, I
>>>> have 1 source and 4 workers. I want that each worker writes autonomously
>>>> into HBase. Is there a proper way to do it?
>>>>
>>>> Best Regards,
>>>>
>>>> --
>>>> --
>>>> Hilmi Yildirim
>>>> Software Developer R&D
>>>>
>>>> T: +49 30 24627-281
>>>> Hilmi.Yildirim@neofonie.de
>>>>
>>>> http://www.neofonie.de
>>>>
>>>> Besuchen Sie den Neo Tech Blog für Anwender:
>>>> http://blog.neofonie.de/
>>>>
>>>> Folgen Sie uns:
>>>> https://plus.google.com/+neofonie
>>>> http://www.linkedin.com/company/neofonie-gmbh
>>>> https://www.xing.com/companies/neofoniegmbh
>>>>
>>>> Neofonie GmbH | Robert-Koch-Platz 4 | 10115 Berlin
>>>> Handelsregister Berlin-Charlottenburg: HRB 67460
>>>> Geschäftsführung: Thomas Kitlitschko
>>>>
>>>>
>>>
>>>
>>> --
>>> --
>>> Hilmi Yildirim
>>> Software Developer R&D
>>>
>>> T: +49 30 24627-281Hilmi.Yildirim@neofonie.de
>>> http://www.neofonie.de
>>>
>>> Besuchen Sie den Neo Tech Blog für Anwender:http://blog.neofonie.de/
>>>
>>> Folgen Sie uns:https://plus.google.com/+neofoniehttp://www.linkedin.com/company/neofonie-gmbhhttps://www.xing.com/companies/neofoniegmbh
>>>
>>> Neofonie GmbH | Robert-Koch-Platz 4 | 10115 Berlin
>>> Handelsregister Berlin-Charlottenburg: HRB 67460
>>> Geschäftsführung: Thomas Kitlitschko
>>>
>>>
>>
>

Mime
View raw message