flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-4491) Handle index.number_of_shards in the ES connector
Date Tue, 22 Nov 2016 15:38:58 GMT

    [ https://issues.apache.org/jira/browse/FLINK-4491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15687057#comment-15687057

ASF GitHub Bot commented on FLINK-4491:

Github user fpompermaier commented on the issue:

    From my point of view ElasticSearchHelper is very useful if you plan to use the Flink
Elasticsearch Sink in a real production use case, because it reduce a lot the complexity of
interfacing with ES.
    We didn't use ES java client before and it took about 2 days to write that helper...I
think it's not that obvious to rewrite that logic from the scratch.
    Ina production environment you really needs to easy customize indexing templates and mappings

> Handle index.number_of_shards in the ES connector
> -------------------------------------------------
>                 Key: FLINK-4491
>                 URL: https://issues.apache.org/jira/browse/FLINK-4491
>             Project: Flink
>          Issue Type: Improvement
>          Components: Streaming Connectors
>    Affects Versions: 1.1.0
>            Reporter: Flavio Pompermaier
>            Priority: Minor
>              Labels: elasticsearch, streaming
> At the moment is not possible to configure the number of shards if an index does not
already exists on the Elasticsearch cluster. It could be a great improvement to handle the
index.number_of_shards (passed in the configuration object). E.g.:
> {code:java}
> Map<String, String> config = Maps.newHashMap();
> config.put("bulk.flush.max.actions", "1");
> config.put("cluster.name", "my-cluster-name");
> config.put("index.number_of_shards", "1");
> {code}

This message was sent by Atlassian JIRA

View raw message