beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Reuven Lax (JIRA)" <j...@apache.org>
Subject [jira] [Created] (BEAM-2700) BigQueryIO should support using file load jobs when using unbounded collections
Date Sun, 30 Jul 2017 18:18:00 GMT
Reuven Lax created BEAM-2700:
--------------------------------

             Summary: BigQueryIO should support using file load jobs when using unbounded
collections
                 Key: BEAM-2700
                 URL: https://issues.apache.org/jira/browse/BEAM-2700
             Project: Beam
          Issue Type: Bug
          Components: sdk-java-gcp
    Affects Versions: 2.2.0
            Reporter: Reuven Lax
            Assignee: Reuven Lax


Currently the method used for inserting into BigQuery is based on the input PCollection. Bounded
input using file load jobs, unbounded input uses streaming inserts. However while streaming
inserts have far lower latency, then cost quite a bit more and they provide weaker consistency
guarantees. Users should be able to choose which method to use, irrespective of the input
PCollection.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message