Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id C7485200B33 for ; Wed, 15 Jun 2016 01:19:29 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id C44DF160A65; Tue, 14 Jun 2016 23:19:29 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 14DF4160A06 for ; Wed, 15 Jun 2016 01:19:28 +0200 (CEST) Received: (qmail 97460 invoked by uid 500); 14 Jun 2016 23:19:28 -0000 Mailing-List: contact commits-help@beam.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@beam.incubator.apache.org Delivered-To: mailing list commits@beam.incubator.apache.org Received: (qmail 97437 invoked by uid 99); 14 Jun 2016 23:19:28 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 14 Jun 2016 23:19:28 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id D938E186D8B for ; Tue, 14 Jun 2016 23:12:50 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -4.646 X-Spam-Level: X-Spam-Status: No, score=-4.646 tagged_above=-999 required=6.31 tests=[KAM_ASCII_DIVIDERS=0.8, KAM_LAZY_DOMAIN_SECURITY=1, RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RP_MATCHES_RCVD=-1.426] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id k_oys8kL_bQo for ; Tue, 14 Jun 2016 23:12:47 +0000 (UTC) Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with SMTP id 3C4E760E66 for ; Tue, 14 Jun 2016 23:12:40 +0000 (UTC) Received: (qmail 70124 invoked by uid 99); 14 Jun 2016 23:12:38 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 14 Jun 2016 23:12:38 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id 37FD4E0991; Tue, 14 Jun 2016 23:12:38 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: davor@apache.org To: commits@beam.incubator.apache.org Date: Tue, 14 Jun 2016 23:13:25 -0000 Message-Id: <103c5999e6284bebbfc574489272ebb9@git.apache.org> In-Reply-To: <95df9c9428334e3980c0c77c4ddc9382@git.apache.org> References: <95df9c9428334e3980c0c77c4ddc9382@git.apache.org> X-Mailer: ASF-Git Admin Mailer Subject: [50/50] [abbrv] incubator-beam git commit: This closes #461 archived-at: Tue, 14 Jun 2016 23:19:30 -0000 This closes #461 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/cd0f5098 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/cd0f5098 Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam/diff/cd0f5098 Branch: refs/heads/python-sdk Commit: cd0f50980c37cd46861b8081d98ebc0fa69d7ccd Parents: d53e96a 778c0d0 Author: Davor Bonaci Authored: Tue Jun 14 16:11:55 2016 -0700 Committer: Davor Bonaci Committed: Tue Jun 14 16:11:55 2016 -0700 ---------------------------------------------------------------------- sdks/python/.gitignore | 10 + sdks/python/.travis.yml | 7 + sdks/python/CONTRIBUTING.md | 26 + sdks/python/LICENSE | 202 ++ sdks/python/README.md | 480 +++ sdks/python/apache_beam/__init__.py | 81 + sdks/python/apache_beam/coders/__init__.py | 19 + sdks/python/apache_beam/coders/coder_impl.pxd | 112 + sdks/python/apache_beam/coders/coder_impl.py | 319 ++ sdks/python/apache_beam/coders/coders.py | 514 +++ sdks/python/apache_beam/coders/coders_test.py | 63 + .../apache_beam/coders/coders_test_common.py | 183 ++ .../apache_beam/coders/fast_coders_test.py | 37 + sdks/python/apache_beam/coders/observable.py | 36 + .../apache_beam/coders/observable_test.py | 57 + .../apache_beam/coders/slow_coders_test.py | 39 + sdks/python/apache_beam/coders/slow_stream.py | 139 + sdks/python/apache_beam/coders/stream.pxd | 61 + sdks/python/apache_beam/coders/stream.pyx | 204 ++ sdks/python/apache_beam/coders/stream_test.py | 171 + sdks/python/apache_beam/coders/typecoders.py | 157 + .../apache_beam/coders/typecoders_test.py | 117 + sdks/python/apache_beam/dataflow_test.py | 408 +++ sdks/python/apache_beam/error.py | 42 + sdks/python/apache_beam/examples/__init__.py | 0 .../examples/complete/autocomplete.py | 82 + .../examples/complete/autocomplete_test.py | 81 + .../examples/complete/estimate_pi.py | 112 + .../examples/complete/estimate_pi_test.py | 49 + .../complete/juliaset/juliaset/__init__.py | 0 .../complete/juliaset/juliaset/juliaset.py | 122 + .../complete/juliaset/juliaset/juliaset_test.py | 86 + .../examples/complete/juliaset/juliaset_main.py | 58 + .../examples/complete/juliaset/setup.py | 115 + .../apache_beam/examples/complete/tfidf.py | 199 ++ .../apache_beam/examples/complete/tfidf_test.py | 91 + .../examples/complete/top_wikipedia_sessions.py | 173 + .../complete/top_wikipedia_sessions_test.py | 61 + .../examples/cookbook/bigquery_schema.py | 130 + .../examples/cookbook/bigquery_side_input.py | 117 + .../cookbook/bigquery_side_input_test.py | 62 + .../examples/cookbook/bigquery_tornadoes.py | 99 + .../cookbook/bigquery_tornadoes_test.py | 44 + .../apache_beam/examples/cookbook/bigshuffle.py | 87 + .../examples/cookbook/bigshuffle_test.py | 64 + .../apache_beam/examples/cookbook/coders.py | 95 + .../examples/cookbook/coders_test.py | 59 + .../examples/cookbook/combiners_test.py | 76 + .../examples/cookbook/custom_ptransform.py | 135 + .../examples/cookbook/custom_ptransform_test.py | 67 + .../apache_beam/examples/cookbook/filters.py | 107 + .../examples/cookbook/filters_test.py | 68 + .../examples/cookbook/group_with_coder.py | 114 + .../examples/cookbook/group_with_coder_test.py | 90 + .../examples/cookbook/mergecontacts.py | 129 + .../examples/cookbook/mergecontacts_test.py | 124 + .../examples/cookbook/multiple_output_pardo.py | 174 + .../cookbook/multiple_output_pardo_test.py | 72 + .../apache_beam/examples/snippets/snippets.py | 875 +++++ .../examples/snippets/snippets_test.py | 563 ++++ .../apache_beam/examples/streaming_wordcap.py | 64 + .../apache_beam/examples/streaming_wordcount.py | 74 + sdks/python/apache_beam/examples/wordcount.py | 102 + .../apache_beam/examples/wordcount_debugging.py | 157 + .../examples/wordcount_debugging_test.py | 59 + .../apache_beam/examples/wordcount_minimal.py | 114 + .../examples/wordcount_minimal_test.py | 59 + .../apache_beam/examples/wordcount_test.py | 58 + sdks/python/apache_beam/internal/__init__.py | 0 sdks/python/apache_beam/internal/apiclient.py | 938 ++++++ .../apache_beam/internal/apiclient_test.py | 113 + sdks/python/apache_beam/internal/auth.py | 164 + .../apache_beam/internal/clients/__init__.py | 0 .../internal/clients/bigquery/__init__.py | 10 + .../clients/bigquery/bigquery_v2_client.py | 642 ++++ .../clients/bigquery/bigquery_v2_messages.py | 1893 +++++++++++ .../internal/clients/dataflow/__init__.py | 10 + .../clients/dataflow/dataflow_v1b3_client.py | 316 ++ .../clients/dataflow/dataflow_v1b3_messages.py | 3056 ++++++++++++++++++ .../internal/clients/storage/__init__.py | 10 + .../clients/storage/storage_v1_client.py | 1021 ++++++ .../clients/storage/storage_v1_messages.py | 1903 +++++++++++ sdks/python/apache_beam/internal/json_value.py | 130 + .../apache_beam/internal/json_value_test.py | 66 + sdks/python/apache_beam/internal/module_test.py | 62 + sdks/python/apache_beam/internal/pickler.py | 208 ++ .../python/apache_beam/internal/pickler_test.py | 81 + sdks/python/apache_beam/internal/util.py | 93 + sdks/python/apache_beam/internal/util_test.py | 61 + .../python/apache_beam/internal/windmill_pb2.py | 2278 +++++++++++++ .../internal/windmill_service_pb2.py | 164 + sdks/python/apache_beam/io/__init__.py | 28 + sdks/python/apache_beam/io/bigquery.py | 829 +++++ sdks/python/apache_beam/io/bigquery_test.py | 453 +++ sdks/python/apache_beam/io/fileio.py | 750 +++++ sdks/python/apache_beam/io/fileio_test.py | 525 +++ sdks/python/apache_beam/io/gcsio.py | 605 ++++ sdks/python/apache_beam/io/gcsio_test.py | 506 +++ sdks/python/apache_beam/io/iobase.py | 1076 ++++++ sdks/python/apache_beam/io/pubsub.py | 76 + sdks/python/apache_beam/io/range_trackers.py | 273 ++ .../apache_beam/io/range_trackers_test.py | 321 ++ sdks/python/apache_beam/io/sources_test.py | 68 + sdks/python/apache_beam/pipeline.py | 438 +++ sdks/python/apache_beam/pipeline_test.py | 348 ++ sdks/python/apache_beam/pvalue.py | 462 +++ sdks/python/apache_beam/pvalue_test.py | 66 + sdks/python/apache_beam/python_sdk_releases.py | 53 + sdks/python/apache_beam/runners/__init__.py | 27 + sdks/python/apache_beam/runners/common.pxd | 28 + sdks/python/apache_beam/runners/common.py | 181 ++ .../apache_beam/runners/dataflow_runner.py | 642 ++++ .../python/apache_beam/runners/direct_runner.py | 329 ++ sdks/python/apache_beam/runners/runner.py | 308 ++ sdks/python/apache_beam/runners/runner_test.py | 69 + sdks/python/apache_beam/transforms/__init__.py | 26 + .../python/apache_beam/transforms/aggregator.py | 108 + .../apache_beam/transforms/aggregator_test.py | 76 + sdks/python/apache_beam/transforms/combiners.py | 526 +++ .../apache_beam/transforms/combiners_test.py | 228 ++ sdks/python/apache_beam/transforms/core.py | 1295 ++++++++ .../apache_beam/transforms/cy_combiners.pxd | 92 + .../apache_beam/transforms/cy_combiners.py | 253 ++ .../python/apache_beam/transforms/ptransform.py | 706 ++++ .../apache_beam/transforms/ptransform_test.py | 1817 +++++++++++ .../python/apache_beam/transforms/sideinputs.py | 148 + sdks/python/apache_beam/transforms/timeutil.py | 313 ++ .../apache_beam/transforms/timeutil_test.py | 168 + sdks/python/apache_beam/transforms/trigger.py | 961 ++++++ .../apache_beam/transforms/trigger_test.py | 569 ++++ .../transforms/trigger_transcripts.yaml | 207 ++ sdks/python/apache_beam/transforms/util.py | 230 ++ sdks/python/apache_beam/transforms/window.py | 386 +++ .../apache_beam/transforms/window_test.py | 204 ++ .../transforms/write_ptransform_test.py | 127 + sdks/python/apache_beam/typehints/__init__.py | 22 + sdks/python/apache_beam/typehints/decorators.py | 533 +++ sdks/python/apache_beam/typehints/opcodes.py | 331 ++ .../apache_beam/typehints/trivial_inference.py | 415 +++ .../typehints/trivial_inference_test.py | 148 + sdks/python/apache_beam/typehints/typecheck.py | 164 + .../typehints/typed_pipeline_test.py | 251 ++ sdks/python/apache_beam/typehints/typehints.py | 1057 ++++++ .../apache_beam/typehints/typehints_test.py | 1056 ++++++ sdks/python/apache_beam/utils/__init__.py | 22 + sdks/python/apache_beam/utils/counters.pxd | 27 + sdks/python/apache_beam/utils/counters.py | 180 ++ sdks/python/apache_beam/utils/dependency.py | 442 +++ .../python/apache_beam/utils/dependency_test.py | 397 +++ sdks/python/apache_beam/utils/names.py | 78 + sdks/python/apache_beam/utils/options.py | 489 +++ sdks/python/apache_beam/utils/path.py | 47 + sdks/python/apache_beam/utils/path_test.py | 70 + .../apache_beam/utils/pipeline_options_test.py | 107 + .../utils/pipeline_options_validator.py | 169 + .../utils/pipeline_options_validator_test.py | 237 ++ sdks/python/apache_beam/utils/processes.py | 52 + sdks/python/apache_beam/utils/processes_test.py | 106 + sdks/python/apache_beam/utils/profiler.py | 69 + sdks/python/apache_beam/utils/retry.py | 197 ++ sdks/python/apache_beam/utils/retry_test.py | 168 + sdks/python/apache_beam/version.py | 20 + sdks/python/setup.cfg | 21 + sdks/python/setup.py | 95 + sdks/python/tox.ini | 7 + 165 files changed, 45848 insertions(+) ----------------------------------------------------------------------