beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Work logged] (BEAM-4061) Chaining SpannerIO#write() transforms
Date Tue, 01 May 2018 19:10:00 GMT

     [ https://issues.apache.org/jira/browse/BEAM-4061?focusedWorklogId=97194&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-97194
]

ASF GitHub Bot logged work on BEAM-4061:
----------------------------------------

                Author: ASF GitHub Bot
            Created on: 01/May/18 19:09
            Start Date: 01/May/18 19:09
    Worklog Time Spent: 10m 
      Work Description: mairbek commented on a change in pull request #4264: [BEAM-4061] Introduced
SpannerWriteResult
URL: https://github.com/apache/beam/pull/4264#discussion_r185306589
 
 

 ##########
 File path: sdks/java/io/google-cloud-platform/src/test/java/org/apache/beam/sdk/io/gcp/spanner/SpannerWriteIT.java
 ##########
 @@ -128,19 +140,72 @@ public void testWrite() throws Exception {
                 .withInstanceId(options.getInstanceId())
                 .withDatabaseId(databaseName));
 
-    p.run();
-    DatabaseClient databaseClient =
-        spanner.getDatabaseClient(
-            DatabaseId.of(
-                project, options.getInstanceId(), databaseName));
+    PipelineResult result = p.run();
+    result.waitUntilFinish();
+    assertThat(result.getState(), is(PipelineResult.State.DONE));
+    assertThat(countNumberOfRecords(), equalTo((long) numRecords));
 
 Review comment:
   Isn't it better for the test to compute that without going through beam? `testSequentialWrite`
already tests `Wait.on`.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Issue Time Tracking
-------------------

    Worklog Id:     (was: 97194)
    Time Spent: 1.5h  (was: 1h 20m)

> Chaining SpannerIO#write() transforms
> -------------------------------------
>
>                 Key: BEAM-4061
>                 URL: https://issues.apache.org/jira/browse/BEAM-4061
>             Project: Beam
>          Issue Type: Bug
>          Components: io-java-gcp
>            Reporter: Mairbek Khadikov
>            Assignee: Mairbek Khadikov
>            Priority: Major
>             Fix For: 2.5.0
>
>          Time Spent: 1.5h
>  Remaining Estimate: 0h
>
> It should be possible to chain several Cloud Spanner writes. In practice, we can leverage
Wait.on transform by returning a result object from SpannerIO#write.
> One particular example, when this feature is useful is full database import. When data
in parent tables should be injected before interleaved tables. See more about table hierarchies
in Spanner here https://cloud.google.com/spanner/docs/schema-and-data-model#creating_a_hierarchy_of_interleaved_tables



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message