flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Márton Balassi <mbala...@apache.org>
Subject Re: Write Stream to HBase
Date Wed, 20 May 2015 10:03:48 GMT
I'm merging the pull request, it was blocked by the streaming operator
rework so it is free to go since yesterday.

I do agree that it needs some additional love before it can be on the
master, but I am positive that it should be there this week.
On May 20, 2015 11:16 AM, "Robert Metzger" <rmetzger@apache.org> wrote:

> There is this pending pull request which is addressing exactly the issues
> I've mentioned (wrong naming, private method):
> https://github.com/apache/flink/pull/521
>
> I'll see whats blocking the PR ...
>
> On Wed, May 20, 2015 at 11:11 AM, Robert Metzger <rmetzger@apache.org>
> wrote:
>
>> Maybe we can also use the Batch HBase OutputFormat.
>>
>> In the DataStream API there is a private method:
>>
>> private DataStreamSink<OUT> writeToFile(OutputFormat<OUT> format, long
millis) {
>>
>> which seems to allow batch output formats.
>> The naming of the method seems weird because its called "toFile" but its
>> expecting an OutputFormat instead of a FileOutputFormat.
>>
>> Maybe its worth trying to see how far we can get with this method.
>>
>>
>> On Wed, May 20, 2015 at 11:00 AM, Robert Metzger <rmetzger@apache.org>
>> wrote:
>>
>>> Hi,
>>> I agree with Hilmi, Flavio's examples are for batch.
>>>
>>> I'm not aware of a StreamingHBaseSink for Flink yet.
>>> I'll filed a JIRA for the feature request:
>>> https://issues.apache.org/jira/browse/FLINK-2055
>>>
>>> Are you interested in implementing this?
>>>
>>> On Wed, May 20, 2015 at 10:50 AM, Hilmi Yildirim <
>>> hilmi.yildirim@neofonie.de> wrote:
>>>
>>>>  Thank you Flavio,
>>>> these are examples for Batch Processing. But I want to write a
>>>> continuous stream into an HBase within a StreamExecutionEnvironment instead
>>>> of a ExecutionEnvironment.
>>>>
>>>> Best Regards,
>>>> Hilmi
>>>>
>>>>
>>>> Am 20.05.2015 um 10:42 schrieb Flavio Pompermaier:
>>>>
>>>> I've added an example of HBase writing
>>>> at flink-staging/flink-hbase/src/test/java/org/apache/flink/addons/hbase/example/HBaseWriteExample.java.
>>>>
>>>> Otherwise you can look at these 2 URLs:
>>>>
>>>>    -
>>>>    https://github.com/fpompermaier/flink/blob/hbaseOutExample/flink-staging/flink-hbase/src/test/java/org/apache/flink/addons/hbase/example/HBaseWriteExample.java
>>>>    -
>>>>    https://github.com/apache/flink/blob/57615aaa19e9933e43ed0431a78dd231bf98b103/flink-staging/flink-hbase/src/test/java/org/apache/flink/addons/hbase/example/HBaseWriteExample.java
>>>>
>>>> Best,
>>>>  Flavio
>>>>
>>>> On Wed, May 20, 2015 at 10:16 AM, Hilmi Yildirim <
>>>> hilmi.yildirim@neofonie.de> wrote:
>>>>
>>>>> Hi,
>>>>> I want to write a stream continuously into an HBase. For example, I
>>>>> have 1 source and 4 workers. I want that each worker writes autonomously
>>>>> into HBase. Is there a proper way to do it?
>>>>>
>>>>> Best Regards,
>>>>>
>>>>> --
>>>>> --
>>>>> Hilmi Yildirim
>>>>> Software Developer R&D
>>>>>
>>>>> T: +49 30 24627-281
>>>>> Hilmi.Yildirim@neofonie.de
>>>>>
>>>>> http://www.neofonie.de
>>>>>
>>>>> Besuchen Sie den Neo Tech Blog für Anwender:
>>>>> http://blog.neofonie.de/
>>>>>
>>>>> Folgen Sie uns:
>>>>> https://plus.google.com/+neofonie
>>>>> http://www.linkedin.com/company/neofonie-gmbh
>>>>> https://www.xing.com/companies/neofoniegmbh
>>>>>
>>>>> Neofonie GmbH | Robert-Koch-Platz 4 | 10115 Berlin
>>>>> Handelsregister Berlin-Charlottenburg: HRB 67460
>>>>> Geschäftsführung: Thomas Kitlitschko
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> --
>>>> Hilmi Yildirim
>>>> Software Developer R&D
>>>>
>>>> T: +49 30 24627-281Hilmi.Yildirim@neofonie.de
>>>> http://www.neofonie.de
>>>>
>>>> Besuchen Sie den Neo Tech Blog für Anwender:http://blog.neofonie.de/
>>>>
>>>> Folgen Sie uns:https://plus.google.com/+neofoniehttp://www.linkedin.com/company/neofonie-gmbhhttps://www.xing.com/companies/neofoniegmbh
>>>>
>>>> Neofonie GmbH | Robert-Koch-Platz 4 | 10115 Berlin
>>>> Handelsregister Berlin-Charlottenburg: HRB 67460
>>>> Geschäftsführung: Thomas Kitlitschko
>>>>
>>>>
>>>
>>
>

Mime
View raw message