beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Fokko van der Wal (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (BEAM-3333) Create Elasticsearch IO compatible with ES 6.x
Date Tue, 12 Dec 2017 07:40:00 GMT

     [ https://issues.apache.org/jira/browse/BEAM-3333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Fokko van der Wal updated BEAM-3333:
------------------------------------
    Description: The current Elasticsearch IO is only compatible with Elasticsearch v 2.x
and v 5.x. The aim is to have an IO compatible with ES v 6.x. Beyond being able to address
v6.x elasticsearch instances, we could also leverage the use of the Elasticsearch pipeline
API and also better split the dataset (be as close as possible of desiredBundleSize) thanks
to the new ES split API that allows ES shards splitting.  (was: The current Elasticsearch
IO (see https://issues.apache.org/jira/browse/BEAM-425) is only compatible with Elasticsearch
v 2.x. The aim is to have an IO compatible with ES v 5.x. Beyond being able to address v5.x
elasticsearch instances, we could also leverage the use of the Elasticsearch pipeline API
and also better split the dataset (be as close as possible of desiredBundleSize) thanks to
the new ES split API that allows ES shards splitting.)

> Create Elasticsearch IO compatible with ES 6.x
> ----------------------------------------------
>
>                 Key: BEAM-3333
>                 URL: https://issues.apache.org/jira/browse/BEAM-3333
>             Project: Beam
>          Issue Type: New Feature
>          Components: sdk-java-extensions
>            Reporter: Fokko van der Wal
>            Assignee: Etienne Chauchot
>            Priority: Minor
>             Fix For: 2.2.0
>
>
> The current Elasticsearch IO is only compatible with Elasticsearch v 2.x and v 5.x. The
aim is to have an IO compatible with ES v 6.x. Beyond being able to address v6.x elasticsearch
instances, we could also leverage the use of the Elasticsearch pipeline API and also better
split the dataset (be as close as possible of desiredBundleSize) thanks to the new ES split
API that allows ES shards splitting.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message