beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mark Liu (JIRA)" <>
Subject [jira] [Created] (BEAM-2633) Missing --sdk_location in Quickstart-Python - run with DataflowRunner
Date Tue, 18 Jul 2017 19:02:00 GMT
Mark Liu created BEAM-2633:

             Summary: Missing --sdk_location in Quickstart-Python - run with
                 Key: BEAM-2633
             Project: Beam
          Issue Type: Bug
          Components: website
            Reporter: Mark Liu
            Assignee: Ahmet Altay

Developing guide in [Quickstart-Python - Execute a pipeline locally - DataflowRunner|]
is missing sdk artifacts portion. 

Need to add one step before pipeline execution to build python artifacts (or download from
Apache release source). Also need to update execution command with "--sdk_location=<your-python-artifact-path>".

Otherwise, developers is getting following errors:
  Could not find a version that satisfies the requirement apache-beam==2.1.0 (from versions:
0.6.0, 2.0.0)
No matching distribution found for apache-beam==2.1.0
Traceback (most recent call last):
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/",
line 162, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/",
line 72, in _run_code
    exec code in run_globals
  File "/Users/markliu/Downloads/apache-beam-2.1.0/apache_beam/examples/", line
126, in <module>
  File "/Users/markliu/Downloads/apache-beam-2.1.0/apache_beam/examples/", line
105, in run
    result =
  File "apache_beam/", line 328, in run
  File "apache_beam/runners/dataflow/", line 283, in run
    self.dataflow_client.create_job(self.job), self)
  File "apache_beam/utils/", line 168, in wrapper
    return fun(*args, **kwargs)
  File "apache_beam/runners/dataflow/internal/", line 423, in create_job
  File "apache_beam/runners/dataflow/internal/", line 446, in create_job_description
    job.options, file_copy=self._gcs_file_copy)
  File "apache_beam/runners/dataflow/internal/", line 399, in stage_job_resources
    _stage_beam_sdk_tarball(sdk_remote_location, staged_path, temp_dir)
  File "apache_beam/runners/dataflow/internal/", line 484, in _stage_beam_sdk_tarball
    _dependency_file_copy(_download_pypi_sdk_package(temp_dir), staged_path)
  File "apache_beam/runners/dataflow/internal/", line 580, in _download_pypi_sdk_package
  File "apache_beam/utils/", line 44, in check_call
    return subprocess.check_call(*args, **kwargs)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/",
line 540, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/Users/markliu/tmp/tmp-env/bin/python', '-m', 'pip',
'install', '--download', '/var/folders/vh/2rbdqyz53m905z_t_yhnvrt000bwz2/T/tmpBf92vB', 'apache-beam==2.1.0',
'--no-binary', ':all:', '--no-deps']' returned non-zero exit status 1

This message was sent by Atlassian JIRA

View raw message