beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Etienne Chauchot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (BEAM-3026) Improve retrying in ElasticSearch client
Date Fri, 06 Oct 2017 14:58:00 GMT

    [ https://issues.apache.org/jira/browse/BEAM-3026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16194697#comment-16194697
] 

Etienne Chauchot commented on BEAM-3026:
----------------------------------------

The Elasticsearch Rest client that is used in {{ElasticsearchIO}} already does retries with
growing waiting times. See this part of the documentation of the features of the rest client:
 

"failed connection penalization (whether a failed node is retried depends on how many consecutive
times it failed; the more failed attempts the longer the client will wait before trying that
same node again)"

https://www.elastic.co/guide/en/elasticsearch/client/java-rest/5.4/_features.html



> Improve retrying in ElasticSearch client
> ----------------------------------------
>
>                 Key: BEAM-3026
>                 URL: https://issues.apache.org/jira/browse/BEAM-3026
>             Project: Beam
>          Issue Type: Improvement
>          Components: sdk-java-extensions
>            Reporter: Tim Robertson
>            Assignee: Jean-Baptiste Onofré
>
> Currently an overloaded ES server will result in clients failing fast.
> I suggest implementing backoff pauses.  Perhaps something like this:
> {code}
>     ElasticsearchIO.ConnectionConfiguration conn = ElasticsearchIO.ConnectionConfiguration
>       .create(new String[]{"http://...:9200"}, "test", "test")
>       .retryWithWaitStrategy(WaitStrategies.exponentialBackoff(1000, TimeUnit.MILLISECONDS)
>       .retryWithStopStrategy(StopStrategies.stopAfterAttempt(10)
>     );
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message