flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Zach Cox <zcox...@gmail.com>
Subject Re: Availability for the ElasticSearch 2 streaming connector
Date Thu, 18 Feb 2016 13:32:43 GMT
Awesome, thanks Suneel. :D

I made the changes to support our use case, which needed flatMap behavior
(index 2 docs, or zero docs, per incoming element) instead of map, and we
also need to make either IndexRequest or UpdateRequest depending on the
element.

-Zach


On Thu, Feb 18, 2016 at 2:06 AM Suneel Marthi <smarthi@apache.org> wrote:

> Thanks Zach, I have a few minor changes too locally; I'll push a PR out
> tomorrow that has ur changes too.
>
> On Wed, Feb 17, 2016 at 5:13 PM, Zach Cox <zcox522@gmail.com> wrote:
>
>> I recently did exactly what Robert described: I copied the code from this
>> (closed) PR https://github.com/apache/flink/pull/1479, modified it a
>> bit, and just included it in my own project that uses the Elasticsearch 2
>> java api. Seems to work well. Here are the files so you can do the same:
>>
>> https://gist.github.com/zcox/59e486be7aeeca381be0
>>
>> -Zach
>>
>>
>> On Wed, Feb 17, 2016 at 4:06 PM Suneel Marthi <suneel.marthi@gmail.com>
>> wrote:
>>
>>> Hey I missed this thread, sorry about that.
>>>
>>> I have a basic connector working with ES 2.0 which I can push out.  Its
>>> not optimized yet and I don't have the time to look at it, if someone would
>>> like to take it over go ahead I can send a PR.
>>>
>>> On Wed, Feb 17, 2016 at 4:57 PM, Robert Metzger <rmetzger@apache.org>
>>> wrote:
>>>
>>>> Hi Mihail,
>>>>
>>>> It seems that nobody is actively working on the elasticsearch2
>>>> connector right now. The 1.0.0 release is already feature frozen, only bug
>>>> fixes or (some) pending pull requests go in.
>>>>
>>>> What you can always do is copy the code from our current elasticsearch
>>>> connector, set the dependency to the version you would like to use and
>>>> adopt our code to their API changes. I think it might take not much time
to
>>>> get it working.
>>>> (The reason why we usually need more time for stuff like this are
>>>> integration tests and documentation).
>>>>
>>>> Please let me know if that solution doesn't work for you.
>>>>
>>>> Regards,
>>>> Robert
>>>>
>>>>
>>>> On Tue, Feb 16, 2016 at 2:53 PM, Vieru, Mihail <mihail.vieru@zalando.de
>>>> > wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> in reference to this ticket
>>>>> https://issues.apache.org/jira/browse/FLINK-3115 when do you think
>>>>> that an ElasticSearch 2 streaming connector will become available? Will
it
>>>>> make it for the 1.0 release?
>>>>>
>>>>> That would be great, as we are planning to use that particular version
>>>>> of ElasticSearch in the very near future.
>>>>>
>>>>> Best regards,
>>>>> Mihail
>>>>>
>>>>
>>>>
>>>
>

Mime
View raw message