flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] twalthr commented on a change in pull request #6391: [FLINK-9885] [FLINK-8101] Finalize Elasticsearch 6.x
Date Mon, 30 Jul 2018 08:24:16 GMT
twalthr commented on a change in pull request #6391: [FLINK-9885] [FLINK-8101] Finalize Elasticsearch
6.x
URL: https://github.com/apache/flink/pull/6391#discussion_r206042028
 
 

 ##########
 File path: docs/dev/connectors/elasticsearch.md
 ##########
 @@ -151,22 +151,47 @@ List<HttpHost> httpHost = new ArrayList<>();
 httpHosts.add(new HttpHost("127.0.0.1", 9200, "http"));
 httpHosts.add(new HttpHost("10.2.3.1", 9200, "http"));
 
-input.addSink(new ElasticsearchSink<>(httpHosts, new ElasticsearchSinkFunction<String>()
{
-    public IndexRequest createIndexRequest(String element) {
-        Map<String, String> json = new HashMap<>();
-        json.put("data", element);
-    
-        return Requests.indexRequest()
-                .index("my-index")
-                .type("my-type")
-                .source(json);
-    }
-    
-    @Override
-    public void process(String element, RuntimeContext ctx, RequestIndexer indexer) {
-        indexer.add(createIndexRequest(element));
+Map<String, String> bulkRequestsConfig = new HashMap<>();
+// This instructs the sink to emit after every element, otherwise they would be buffered
+bulkRequestsConfig.put("bulk.flush.max.actions", "1");
 
 Review comment:
   We could also think about adding this to the builder now. This is Flink configuration and
we don't need a string map anymore, right?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

Mime
View raw message