beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Stephen Reichling (JIRA)" <>
Subject [jira] [Created] (BEAM-1068) Service Account Credentials File Specified via Pipeline Option Ignored
Date Thu, 01 Dec 2016 19:21:58 GMT
Stephen Reichling created BEAM-1068:

             Summary: Service Account Credentials File Specified via Pipeline Option Ignored
                 Key: BEAM-1068
             Project: Beam
          Issue Type: Bug
          Components: sdk-py
            Reporter: Stephen Reichling
            Assignee: Frances Perry
            Priority: Minor

When writing a pipeline that authenticates with Google Dataflow APIs using a service account,
specifying the path to that service account's credentials file in the {{PipelineOptions}}
object passed in to the pipeline does not work, it only works when passed as a command-line

For example, if I write code like so:
pipelineOptions = options.PipelineOptions()
gcOptions = pipelineOptions.view_as(options.GoogleCloudOptions)
gcOptions.service_account_name = 'My Service Account Name'
gcOptions.service_account_key_file = '/some/path/keyfile.p12'

pipeline = beam.Pipeline(options=pipelineOptions)
# ... add stages to the pipeline
and execute it like so:
{{python ./}}
...the service account I specify will not be used.

Only if I were to execute the code like so:
{{python ./ --service_account_name 'My Service Account Name' --service_account_key_file
...does it actually use the service account.

The problem appears to be rooted in `` which reconstructs the {{PipelineOptions}} object
directly from {{sys.argv}} rather than using the instance passed in to the pipeline:

This message was sent by Atlassian JIRA

View raw message