beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Colin Phipps (JIRA)" <>
Subject [jira] [Created] (BEAM-2556) Client-side throttling for Datastore connector
Date Thu, 06 Jul 2017 09:44:00 GMT
Colin Phipps created BEAM-2556:

             Summary: Client-side throttling for Datastore connector
                 Key: BEAM-2556
             Project: Beam
          Issue Type: Improvement
          Components: sdk-java-gcp
            Reporter: Colin Phipps
            Assignee: Colin Phipps
            Priority: Minor

The Datastore connector currently has exponential backoff on errors, which is good. But it
does not do any other throttling of its write load in response to errors; once a request succeeds,
it resumes writing as quickly as it can.

Write loads will be more stable and more likely to compete if the client throttles itself
in the event that it receives high rates of errors from the Datastore service; specifically
is a technique that Google has had success with on other services.

We (Datastore) have a patch in progress to add this behaviour to the connector.

This message was sent by Atlassian JIRA

View raw message