beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Etienne Chauchot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (BEAM-1637) Create Elasticsearch IO compatible with ES 5.x
Date Tue, 18 Jul 2017 07:41:00 GMT

    [ https://issues.apache.org/jira/browse/BEAM-1637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16091230#comment-16091230
] 

Etienne Chauchot commented on BEAM-1637:
----------------------------------------

I've created the ES v5 IO and refactored ES v2 IO. I still need to fix some of the tests.
I'll submit a PR soon. 

> Create Elasticsearch IO compatible with ES 5.x
> ----------------------------------------------
>
>                 Key: BEAM-1637
>                 URL: https://issues.apache.org/jira/browse/BEAM-1637
>             Project: Beam
>          Issue Type: New Feature
>          Components: sdk-java-extensions
>            Reporter: Etienne Chauchot
>            Assignee: Etienne Chauchot
>            Priority: Minor
>
> The current Elasticsearch IO (see https://issues.apache.org/jira/browse/BEAM-425) is
only compatible with Elasticsearch v 2.x. The aim is to have an IO compatible with ES v 5.x.
Beyond being able to address v5.x elasticsearch instances, we could also leverage the use
of the Elasticsearch pipeline API and also better split the dataset (be as close as possible
of desiredBundleSize) thanks to the new ES split API that allows ES shards splitting.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message