beam-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Luke Cwik <lc...@google.com>
Subject Re: Can SpannerIO read data from different GCP project?
Date Mon, 29 Jun 2020 16:17:51 GMT
The intent is that you grant permissions to the account that is running the
Dataflow job to the resources you want it to access in project B before you
start the pipeline. This allows for much finer grain access control and the
ability to revoke permissions without having to disable an entire account.

I would take a look at the general IAM and security documentation within
GCP[1] or open up a support case with GCP requesting guidance.

1: https://cloud.google.com/iam

On Sun, Jun 28, 2020 at 8:56 AM Austin Bennett <whatwouldaustindo@gmail.com>
wrote:

> I havent tried yet, but looks like the connection string asks for the
> project to be specified.  Based on that (and cross project working for
> other circumstances), I would imagine it will work, but...?  Give it a try!
>
> One tricky place might be ensuring proper permissions, in both projects
> (and without being too open).
>
> On Sat, Jun 27, 2020, 5:46 AM Sheng Yang <liff666@163.com> wrote:
>
>> Hi,
>>
>> I am working on Beam using Dataflow engine. Recently I am working on
>> reading spanner data from different project. Say I run my Beam dataflow job
>> in GCP project A, but the Spanner is in GCP project B. I searched all the
>> documents, but can't find any documentation about SpannerIO reading data
>> with the custom credential key files. Right now I am considering JdbcIO
>> because it accepts custom credential as parameters and spanner also have
>> jdbc api[1].
>> Do I have something wrong in my description? Or am I considering the
>> correct approach?
>>
>> String url = "jdbc:cloudspanner:/projects/my_project_id/"
>>
>>            + "instances/my_instance_id/"
>>            + "databases/my_database_name"
>>            + "?credentials=/home/cloudspanner-keys/my-key.json"
>>            + ";autocommit=false";try (Connection connection = DriverManager.getConnection(url))
{
>>   try(ResultSet rs = connection.createStatement()
>>            .executeQuery("SELECT SingerId, AlbumId, MarketingBudget FROM Albums"))
{
>>     while(rs.next()) {
>>       Long singerId = rs.getLong(1);
>>     }
>>   }
>> }
>>
>>
>> [1]: https://github.com/googleapis/java-spanner-jdbc
>>
>> Thanks,
>> Sheng
>>
>>
>>
>>
>

Mime
View raw message