beam-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (Jira)" <j...@apache.org>
Subject [jira] [Work logged] (BEAM-11983) Java Datastore - Implement IO Request Count metrics
Date Tue, 03 Aug 2021 17:10:00 GMT

     [ https://issues.apache.org/jira/browse/BEAM-11983?focusedWorklogId=633058&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-633058
]

ASF GitHub Bot logged work on BEAM-11983:
-----------------------------------------

                Author: ASF GitHub Bot
            Created on: 03/Aug/21 17:09
            Start Date: 03/Aug/21 17:09
    Worklog Time Spent: 10m 
      Work Description: MiguelAnzoWizeline commented on a change in pull request #15183:
URL: https://github.com/apache/beam/pull/15183#discussion_r681948955



##########
File path: sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/datastore/DatastoreV1.java
##########
@@ -901,12 +907,28 @@ private RunQueryResponse runQueryWithRetries(RunQueryRequest request)
throws Exc
         Sleeper sleeper = Sleeper.DEFAULT;
         BackOff backoff = RUNQUERY_BACKOFF.backoff();
         while (true) {
+          HashMap<String, String> baseLabels = new HashMap<>();
+          baseLabels.put(MonitoringInfoConstants.Labels.PTRANSFORM, "TODO");
+          baseLabels.put(MonitoringInfoConstants.Labels.SERVICE, "Datastore");
+          baseLabels.put(MonitoringInfoConstants.Labels.METHOD, "runQueryWithRetries");
+          baseLabels.put(MonitoringInfoConstants.Labels.RESOURCE, "TODO");
+          baseLabels.put(
+              MonitoringInfoConstants.Labels.BIGQUERY_PROJECT_ID, request.getProjectId());
+          ServiceCallMetric serviceCallMetric =
+              new ServiceCallMetric(MonitoringInfoConstants.Urns.API_REQUEST_COUNT, baseLabels);
           try {
             RunQueryResponse response = datastore.runQuery(request);
+            serviceCallMetric.call("ok");
             rpcSuccesses.inc();
             return response;
           } catch (DatastoreException exception) {
             rpcErrors.inc();
+            GoogleJsonError.ErrorInfo errorInfo = getErrorInfo(exception);
+            if (errorInfo == null) {
+              serviceCallMetric.call(ServiceCallMetric.CANONICAL_STATUS_UNKNOWN);
+              throw exception;
+            }
+            serviceCallMetric.call(errorInfo.getReason());

Review comment:
       @ajamato Hi, one question, this is the last comment left, here the idea would just
be to run the existing integration tests locally (https://github.com/apache/beam/blob/master/sdks/java/extensions/sql/src/test/java/org/apache/beam/sdk/extensions/sql/meta/provider/datastore/DataStoreReadWriteIT.java)
with an error introduced to see if the code with the metric of the exception is called?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Issue Time Tracking
-------------------

    Worklog Id:     (was: 633058)
    Time Spent: 4h 40m  (was: 4.5h)

> Java Datastore - Implement IO Request Count metrics
> ---------------------------------------------------
>
>                 Key: BEAM-11983
>                 URL: https://issues.apache.org/jira/browse/BEAM-11983
>             Project: Beam
>          Issue Type: Test
>          Components: io-java-gcp
>            Reporter: Alex Amato
>            Assignee: Miguel Anzo
>            Priority: P3
>          Time Spent: 4h 40m
>  Remaining Estimate: 0h
>
> Reference PRs (See BigQuery IO example) and detailed explanation of what's needed to
instrument this IO with Request Count metrics is found in this handoff doc:
> [https://docs.google.com/document/d/1lrz2wE5Dl4zlUfPAenjXIQyleZvqevqoxhyE85aj4sc/edit]
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message