flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-4491) Handle index.number_of_shards in the ES connector
Date Tue, 22 Nov 2016 15:38:58 GMT

    [ https://issues.apache.org/jira/browse/FLINK-4491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15687057#comment-15687057
] 

ASF GitHub Bot commented on FLINK-4491:
---------------------------------------

Github user fpompermaier commented on the issue:

    https://github.com/apache/flink/pull/2790
  
    From my point of view ElasticSearchHelper is very useful if you plan to use the Flink
Elasticsearch Sink in a real production use case, because it reduce a lot the complexity of
interfacing with ES.
    We didn't use ES java client before and it took about 2 days to write that helper...I
think it's not that obvious to rewrite that logic from the scratch.
    Ina production environment you really needs to easy customize indexing templates and mappings
IMHO


> Handle index.number_of_shards in the ES connector
> -------------------------------------------------
>
>                 Key: FLINK-4491
>                 URL: https://issues.apache.org/jira/browse/FLINK-4491
>             Project: Flink
>          Issue Type: Improvement
>          Components: Streaming Connectors
>    Affects Versions: 1.1.0
>            Reporter: Flavio Pompermaier
>            Priority: Minor
>              Labels: elasticsearch, streaming
>
> At the moment is not possible to configure the number of shards if an index does not
already exists on the Elasticsearch cluster. It could be a great improvement to handle the
index.number_of_shards (passed in the configuration object). E.g.:
> {code:java}
> Map<String, String> config = Maps.newHashMap();
> config.put("bulk.flush.max.actions", "1");
> config.put("cluster.name", "my-cluster-name");
> config.put("index.number_of_shards", "1");
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message