beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Udi Meiri (JIRA)" <j...@apache.org>
Subject [jira] [Assigned] (BEAM-2101) Reading public GCS files requires authentication
Date Wed, 13 Dec 2017 18:57:00 GMT

     [ https://issues.apache.org/jira/browse/BEAM-2101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Udi Meiri reassigned BEAM-2101:
-------------------------------

    Assignee: Udi Meiri

> Reading public GCS files requires authentication
> ------------------------------------------------
>
>                 Key: BEAM-2101
>                 URL: https://issues.apache.org/jira/browse/BEAM-2101
>             Project: Beam
>          Issue Type: Bug
>          Components: sdk-py-core
>            Reporter: Ahmet Altay
>            Assignee: Udi Meiri
>
> cc: [~sb2nov]
> [~tibor.kiss@gmail.com] reported this on hackathon, while running wordcount. Even though
plain gsutil works fine on the public file, wordcount fails. It is possible that some other
authentication is required as part of the validation.
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
>   File "apache_beam/io/textio.py", line 390, in __init__
>     skip_header_lines=skip_header_lines)
>   File "apache_beam/io/textio.py", line 89, in __init__
>     validate=validate)
>   File "apache_beam/io/filebasedsource.py", line 103, in __init__
>     self._validate()
>   File "apache_beam/utils/value_provider.py", line 101, in _f
>     return fnc(self, *args, **kwargs)
>   File "apache_beam/io/filebasedsource.py", line 163, in _validate
>     match_result = FileSystems.match([pattern], limits=[1])[0]
>   File "apache_beam/io/filesystems.py", line 81, in match
>     return filesystem.match(patterns, limits)
>   File "apache_beam/io/gcp/gcsfilesystem.py", line 100, in match
>     raise BeamIOError("Match operation failed", exceptions)
> apache_beam.io.filesystem.BeamIOError: Match operation failed with exceptions {'gs://apache-beam-samples/shakespeare/hamlet.txt':
ApplicationDefaultCredentialsError('The Application Default Credentials are not available.
They are available if running in Google Compute Engine. Otherwise, the environment variable
GOOGLE_APPLICATION_CREDENTIALS must be defined pointing to a file defining the credentials.
See https://developers.google.com/accounts/docs/application-default-credentials for more information.',)}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message