beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From k...@apache.org
Subject [23/50] [abbrv] beam git commit: This closes #3172: Sync gearpump-runner with master
Date Mon, 07 Aug 2017 23:53:48 GMT
This closes #3172: Sync gearpump-runner with master

  Update gearpump-runner against master changes
  Use stable naming strategy for ByteBuddy invokers
  Translate a Pipeline in SdkComponents
  [TRIVIAL] runners-core: delete placeholder
  Fixes an accidentally found bug in SimpleDoFnRunner
  Removes OldDoFn and its kin from runners-core
  Bump Dataflow containers to 0512
  Improve Pruning performed by the DirectRunnerApiSurfaceTest
  Adding support for subnetwork in Python Pipelineoptions
  Use built-in cmp python function in comparing datastore paths
  ApexRunner SDF support
  Fix documentation for the shard_template_name
  [BEAM-2299] Run maven install on Windows machine for build/test coverage on Windows
  Remove "Dataflow" from apache_beam __init__.py file
  Moving the data file for trigger tests to testing/data
  Fix GcsResourceIdTest in postcommits
  readAvros should't have proto Message upper bound
  Reduce Log Level of PubsubUnboundedSource
  [BEAM-2290] Fix issue where timestamps weren't set when using CompressedSource
  [BEAM-2279] Fix archetype breakages
  internal comments
  Fix shading of guava testlib
  Rename FileSystems.setDefaultConfigInWorkers
  [BEAM-2277] HadoopFileSystem: normalize implementation
  Mark FileSystem and related as Experimental
  [BEAM-2277] Add ResourceIdTester and test existing ResourceId implementations
  Remove '/' entirely from determining FileSystem scheme
  [BEAM-2279] Add HDFS support to Spark runner profiles in archetypes and examples
  [BEAM-2277] Fix URI_SCHEME_PATTERN in FileSystems
  BigtableIO should use AutoValue for read and write
  [BEAM-2153] Move connection management in JmsIO.write() to setup/teardown methods
  Mark More values methods Internal
  Rename filesink to filebasedsink
  Enable SerializableCoder to Serialize with Generic Types
  Remove unused test data
  Fix due to GBKO name change.
  Don't deploy jdk1.8-tests module
  Remove some internal details from the public API.
  Move assert_that, equal_to, is_empty to apache_beam.testing.util
  [BEAM-1345] Clearly delineate public api in apache_beam/typehints.
  [BEAM-1345] Mark apache_beam/internal as internal.
  [BEAM-1345] Annotate public members of pvalue.
  Add internal comments to metrics
  [BEAM-1340] Add __all__ tags to modules in package apache_beam/transforms
  [BEAM-2256] Add the last previous range filter
  Use a consistent calculation for GC Time
  fix lint error in fake_datastore.py
  Add __all__ tags to modules in package apache_beam/testing
  [BEAM-1340] Adds __all__ tags to classes in package apache_beam/io.
  [BEAM-1345] Clearly delineate public api in apache_beam/coders.
  [BEAM-1345] Clearly delineate public api in runners package.
  [BEAM-1345] Mark Pipeline as public.
  [BEAM-1345] Clearly delineate public API in apache_beam/options
  Mark internal modules in python datastoreio
  [BEAM-2260] Improve construction-time errors for Text and AvroIO
  [BEAM-2179] Archetype generate-sources.sh cleanup the existing sources before rsync
  [BEAM-1345] Mark windowed value as experimental
  Add internal usage only comments to util/
  Remove protobuf and http-client dependency from runners/google-cloud-dataflow
  minor typo fix in comment
  Add support for local execution to PubsubIO using the google cloud emulator
  [BEAM-2150] Relax regex to support wildcard globbing for GCS
  bump time of precommits
  [BEAM-2244] Move details of Metrics to Runners Core
  Correct javadoc for mobile gaming examples
  Update SDK Coders to return the Empty List from getCoderArguments
  Skip generating empty jars for parent poms
  Fix a typo in TestDataflowRunnerTest
  Re-enable UsesTimersInParDo tests in Dataflow runner
  TestDataflowRunner: throw AssertionError only when assertion known failed
  Allow any throwable in PAssert to constitute adequate failure
  [BEAM-2242] Ensure that jars are shaded correctly by running the jar plugin before the shade plugin
  [BEAM-2240] Always augment exception with step name.
  Adds dependency on findbugs to examples/java
  Splits WriteBundles into windowed/unwindowed versions
  Simpler code for setting shard numbers on results in FileBasedSink
  Implement dynamic-sharding for windowed file outputs, and add an integration test.
  Renames FileBasedSink inner classes
  [BEAM-2250] remove experimental and internal things from pydoc
  [BEAM-2249] Correctly handle partial reads in AvroSource
  Use text output for first two mobile gaming examples
  Remove verifyDeterministic from StructuredCoder
  Update Coder Documentation
  Improve DirectRunner Javadoc
  [BEAM-2211] Delete deprecated NoopPathValidator
  Remove Timer.cancel() from user-facing API
  Remove Readme files.
  Renames some python classes and functions that were unnecessarily public.
  Mark PipelineVisitor and AppliedPTransform as internal.
  Mark PValue and PValueBase Internal
  [BEAM-2236] Move test utilities out of python core
  Include 'sun.reflect' in GcpCoreApiSurfaceTest
  Fix checkstyle error
  Shade dependencies in sdks/core
  Remove trailing whitespace
  Add per-runner profile to Java 8 examples
  Register TestSparkPipelineOptions only in src/test to avoid hard hamcrest dep
  Update Apache Beam Python version to 2.1.0.dev
  Shade JSR305 in the DirectRunner
  Remove hadoop io readme
  ...


Project: http://git-wip-us.apache.org/repos/asf/beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/99221e73
Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/99221e73
Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/99221e73

Branch: refs/heads/master
Commit: 99221e73991002b8ed95cd5649fadee1d14bba4d
Parents: 58546ac 6c06967
Author: Kenneth Knowles <klk@google.com>
Authored: Thu May 18 13:07:02 2017 -0700
Committer: Kenneth Knowles <klk@google.com>
Committed: Thu May 18 13:07:02 2017 -0700

----------------------------------------------------------------------
 .gitignore                                      |   1 +
 .../jenkins/common_job_properties.groovy        |  29 +-
 .../job_beam_PerformanceTests_Dataflow.groovy   |   3 +
 ...job_beam_PostCommit_Java_MavenInstall.groovy |   8 +-
 ..._PostCommit_Java_MavenInstall_Windows.groovy |  45 ++
 ...tCommit_Java_ValidatesRunner_Dataflow.groovy |   2 +-
 .../job_beam_PreCommit_Java_MavenInstall.groovy |   5 +-
 .../job_beam_Release_NightlySnapshot.groovy     |   2 +-
 README.md                                       |   4 +-
 examples/java/README.md                         |  64 +-
 examples/java/pom.xml                           |  53 +-
 .../apache/beam/examples/WindowedWordCount.java |   9 +-
 .../beam/examples/common/ExampleUtils.java      |   6 +-
 .../examples/common/WriteOneFilePerWindow.java  |  72 +-
 .../org/apache/beam/examples/complete/README.md |   2 +-
 .../apache/beam/examples/complete/TfIdf.java    |   2 +-
 .../org/apache/beam/examples/cookbook/README.md |   2 +-
 .../beam/examples/WindowedWordCountIT.java      |  26 +-
 .../beam/examples/complete/TfIdfTest.java       |   2 +-
 examples/java8/pom.xml                          | 124 ++-
 .../beam/examples/complete/game/GameStats.java  |  15 +-
 .../examples/complete/game/HourlyTeamScore.java |  58 +-
 .../examples/complete/game/LeaderBoard.java     |  62 +-
 .../beam/examples/complete/game/README.md       | 131 ---
 .../beam/examples/complete/game/UserScore.java  |  74 +-
 .../complete/game/utils/WriteToBigQuery.java    |  32 +-
 .../complete/game/utils/WriteToText.java        | 184 +++++
 .../game/utils/WriteWindowedToBigQuery.java     |   9 +-
 examples/pom.xml                                |   2 +-
 pom.xml                                         |  71 +-
 runners/apex/README.md                          |  76 --
 runners/apex/pom.xml                            |  10 +-
 .../beam/runners/apex/ApexPipelineOptions.java  |   5 +
 .../apache/beam/runners/apex/ApexRunner.java    |  51 +-
 .../beam/runners/apex/ApexRunnerRegistrar.java  |   3 +-
 .../beam/runners/apex/ApexRunnerResult.java     |   2 -
 .../beam/runners/apex/ApexYarnLauncher.java     |   2 -
 .../beam/runners/apex/TestApexRunner.java       |  10 +-
 .../translation/ApexPipelineTranslator.java     |  23 +
 .../apex/translation/ParDoTranslator.java       |  68 +-
 .../translation/ReadUnboundedTranslator.java    |   1 -
 .../apex/translation/TransformTranslator.java   |   2 -
 .../apex/translation/TranslationContext.java    |  97 ++-
 .../operators/ApexGroupByKeyOperator.java       |  10 +-
 .../operators/ApexParDoOperator.java            | 131 ++-
 .../operators/ApexProcessFnOperator.java        |   8 +-
 .../ApexReadUnboundedInputOperator.java         |   2 -
 .../operators/ApexTimerInternals.java           |  23 +-
 .../translation/utils/ApexStateInternals.java   |  24 +-
 .../apex/translation/utils/ApexStreamTuple.java |  16 +-
 .../utils/CoderAdapterStreamCodec.java          |   2 -
 .../utils/SerializablePipelineOptions.java      |  19 +-
 .../translation/utils/StateInternalsProxy.java  |  11 +-
 .../utils/ValueAndCoderKryoSerializable.java    |   2 -
 .../apex/translation/utils/ValuesSource.java    |   2 -
 .../beam/runners/apex/ApexRunnerTest.java       |  49 +-
 .../beam/runners/apex/ApexYarnLauncherTest.java |   2 -
 .../apex/examples/UnboundedTextSource.java      |   2 -
 .../runners/apex/examples/WordCountTest.java    |   2 -
 .../translation/ApexGroupByKeyOperatorTest.java |   4 +-
 .../FlattenPCollectionTranslatorTest.java       |  13 +-
 .../apex/translation/ParDoTranslatorTest.java   |   2 +-
 .../operators/ApexTimerInternalsTest.java       |  10 +-
 .../utils/ApexStateInternalsTest.java           |  12 +-
 .../translation/utils/CollectionSource.java     |   2 -
 .../translation/utils/PipelineOptionsTest.java  |  98 ++-
 .../test/resources/beam-runners-apex.properties |   2 +-
 runners/core-construction-java/pom.xml          |   2 +-
 .../DeduplicatedFlattenFactory.java             |   2 +-
 .../EmptyFlattenAsCreateFactory.java            |   2 +-
 .../core/construction/ForwardingPTransform.java |   6 +-
 .../runners/core/construction/PCollections.java |   2 +-
 .../core/construction/PTransformMatchers.java   |   2 +-
 .../construction/PTransformReplacements.java    |   2 +-
 .../runners/core/construction/PTransforms.java  |  19 +-
 .../core/construction/PrimitiveCreate.java      |   4 +-
 .../core/construction/SdkComponents.java        |  54 +-
 .../runners/core/construction/Triggers.java     |   4 +-
 .../UnboundedReadFromBoundedSource.java         |  16 +-
 .../UnsupportedOverrideFactory.java             |   2 +-
 .../core/construction/WindowingStrategies.java  |   4 +-
 .../runners/core/metrics/MetricFiltering.java   | 102 +++
 .../beam/runners/core/metrics/MetricKey.java    |  43 +
 .../beam/runners/core/metrics/package-info.java |  22 +
 .../runners/core/construction/CodersTest.java   |  18 +-
 .../EmptyFlattenAsCreateFactoryTest.java        |   2 +-
 .../core/construction/PCollectionsTest.java     |  17 +-
 .../construction/PTransformMatchersTest.java    |  34 +-
 .../PTransformReplacementsTest.java             |   2 +-
 .../core/construction/PTransformsTest.java      |   2 +-
 .../construction/ReplacementOutputsTest.java    |   2 +-
 .../core/construction/SdkComponentsTest.java    | 106 ++-
 .../SingleInputOutputOverrideFactoryTest.java   |   2 +-
 .../construction/WindowingStrategiesTest.java   |   4 +-
 .../core/metrics/MetricFilteringTest.java       | 148 ++++
 runners/core-java/pom.xml                       |   2 +-
 .../beam/runners/core/AssignWindowsDoFn.java    |  78 --
 .../beam/runners/core/BaseExecutionContext.java |  13 +-
 .../apache/beam/runners/core/DoFnAdapters.java  | 310 -------
 .../apache/beam/runners/core/DoFnRunner.java    |   2 +-
 .../apache/beam/runners/core/DoFnRunners.java   |  52 +-
 .../core/ElementAndRestrictionCoder.java        |  29 +-
 .../beam/runners/core/ExecutionContext.java     |  10 +-
 .../runners/core/GlobalCombineFnRunner.java     |   1 -
 .../runners/core/GlobalCombineFnRunners.java    |  41 +-
 .../GroupAlsoByWindowViaOutputBufferDoFn.java   | 113 ---
 .../core/GroupAlsoByWindowViaWindowSetDoFn.java |  94 ---
 .../GroupAlsoByWindowViaWindowSetNewDoFn.java   |   9 +-
 .../core/GroupAlsoByWindowsAggregators.java     |  26 +
 .../runners/core/GroupAlsoByWindowsDoFn.java    |  39 -
 .../core/GroupByKeyViaGroupByKeyOnly.java       |   2 +-
 .../runners/core/InMemoryStateInternals.java    |  22 +-
 .../runners/core/InMemoryTimerInternals.java    |   2 +-
 .../beam/runners/core/KeyedWorkItemCoder.java   |  22 +-
 .../core/LateDataDroppingDoFnRunner.java        |   9 +-
 .../apache/beam/runners/core/LateDataUtils.java |  37 +-
 .../runners/core/MergingActiveWindowSet.java    |   2 +-
 .../beam/runners/core/MergingStateAccessor.java |   2 +-
 .../apache/beam/runners/core/NonEmptyPanes.java |   8 +-
 .../beam/runners/core/NullSideInputReader.java  |  61 ++
 .../org/apache/beam/runners/core/OldDoFn.java   | 335 --------
 ...eBoundedSplittableProcessElementInvoker.java |  18 +-
 .../beam/runners/core/PaneInfoTracker.java      |   4 +-
 .../beam/runners/core/ProcessFnRunner.java      |   3 +-
 .../core/PushbackSideInputDoFnRunner.java       |   2 +-
 .../core/ReadyCheckingSideInputReader.java      |  34 +
 .../org/apache/beam/runners/core/ReduceFn.java  |   6 +-
 .../runners/core/ReduceFnContextFactory.java    |  13 +-
 .../beam/runners/core/ReduceFnRunner.java       |  50 +-
 .../beam/runners/core/SideInputHandler.java     |  15 +-
 .../beam/runners/core/SideInputReader.java      |  47 ++
 .../beam/runners/core/SimpleDoFnRunner.java     | 223 ++++-
 .../beam/runners/core/SimpleOldDoFnRunner.java  | 500 ------------
 .../core/SimplePushbackSideInputDoFnRunner.java |   3 +-
 .../beam/runners/core/SplittableParDo.java      |  50 +-
 .../apache/beam/runners/core/StateAccessor.java |   2 +-
 .../beam/runners/core/StateInternals.java       |   4 +-
 .../apache/beam/runners/core/StateMerging.java  |  14 +-
 .../apache/beam/runners/core/StateTable.java    |   4 +-
 .../org/apache/beam/runners/core/StateTag.java  |  16 +-
 .../org/apache/beam/runners/core/StateTags.java |  20 +-
 .../beam/runners/core/StatefulDoFnRunner.java   |  14 +-
 .../beam/runners/core/SystemReduceFn.java       |   8 +-
 .../core/TestInMemoryStateInternals.java        |   4 +-
 .../beam/runners/core/TimerInternals.java       |  30 +-
 .../core/UnsupportedSideInputReader.java        |   1 -
 .../apache/beam/runners/core/WatermarkHold.java |  15 +-
 .../core/WindowingInternalsAdapters.java        |  75 --
 .../beam/runners/core/metrics/CounterCell.java  |  86 ++
 .../beam/runners/core/metrics/DirtyState.java   |  99 +++
 .../runners/core/metrics/DistributionCell.java  |  80 ++
 .../runners/core/metrics/DistributionData.java  |  62 ++
 .../beam/runners/core/metrics/GaugeCell.java    |  78 ++
 .../beam/runners/core/metrics/GaugeData.java    |  83 ++
 .../beam/runners/core/metrics/MetricCell.java   |  42 +
 .../runners/core/metrics/MetricUpdates.java     |  79 ++
 .../core/metrics/MetricsContainerImpl.java      | 188 +++++
 .../core/metrics/MetricsContainerStepMap.java   | 495 ++++++++++++
 .../beam/runners/core/metrics/MetricsMap.java   |  88 ++
 .../beam/runners/core/metrics/package-info.java |  22 +
 .../AfterDelayFromFirstElementStateMachine.java |   6 +-
 .../core/triggers/AfterPaneStateMachine.java    |   2 +-
 .../AfterProcessingTimeStateMachine.java        |   2 +-
 ...rSynchronizedProcessingTimeStateMachine.java |   2 +-
 .../triggers/AfterWatermarkStateMachine.java    |   2 +-
 .../triggers/DefaultTriggerStateMachine.java    |   2 +-
 .../triggers/ReshuffleTriggerStateMachine.java  |   2 +-
 .../core/triggers/TriggerStateMachine.java      |   2 +-
 .../TriggerStateMachineContextFactory.java      |   6 +-
 .../triggers/TriggerStateMachineRunner.java     |   4 +-
 runners/core-java/src/test/java/.placeholder    |   0
 ...roupAlsoByWindowViaOutputBufferDoFnTest.java | 109 ---
 .../core/GroupAlsoByWindowsProperties.java      | 744 -----------------
 .../core/InMemoryStateInternalsTest.java        |  16 +-
 .../core/InMemoryTimerInternalsTest.java        |   2 +-
 .../runners/core/KeyedWorkItemCoderTest.java    |   2 +-
 .../core/LateDataDroppingDoFnRunnerTest.java    |  11 +-
 .../beam/runners/core/LateDataUtilsTest.java    |  90 +++
 .../apache/beam/runners/core/NoOpOldDoFn.java   |  65 --
 .../apache/beam/runners/core/OldDoFnTest.java   |  51 --
 ...ndedSplittableProcessElementInvokerTest.java |   1 -
 .../beam/runners/core/ReduceFnRunnerTest.java   |  41 +-
 .../beam/runners/core/ReduceFnTester.java       |   8 +-
 .../beam/runners/core/SideInputHandlerTest.java |   2 +-
 .../beam/runners/core/SimpleDoFnRunnerTest.java |  17 +-
 .../runners/core/SimpleOldDoFnRunnerTest.java   |  86 --
 .../SimplePushbackSideInputDoFnRunnerTest.java  |   3 +-
 .../beam/runners/core/SplittableParDoTest.java  |   7 +-
 .../runners/core/StatefulDoFnRunnerTest.java    |  19 +-
 .../beam/runners/core/TimerInternalsTest.java   |   2 +-
 .../runners/core/metrics/CounterCellTest.java   |  56 ++
 .../runners/core/metrics/DirtyStateTest.java    |  56 ++
 .../core/metrics/DistributionCellTest.java      |  54 ++
 .../runners/core/metrics/GaugeCellTest.java     |  51 ++
 .../core/metrics/MetricUpdateMatchers.java      |  82 ++
 .../core/metrics/MetricsContainerImplTest.java  | 130 +++
 .../metrics/MetricsContainerStepMapTest.java    | 272 +++++++
 .../runners/core/metrics/MetricsMapTest.java    | 103 +++
 .../triggers/TriggerStateMachineTester.java     |   6 +-
 .../core/triggers/TriggerStateMachinesTest.java |   2 +-
 runners/direct-java/pom.xml                     | 100 ++-
 .../direct/AbstractModelEnforcement.java        |   1 -
 .../direct/BoundedReadEvaluatorFactory.java     |  17 +-
 .../beam/runners/direct/BundleFactory.java      |   3 +-
 .../org/apache/beam/runners/direct/Clock.java   |   2 +-
 .../runners/direct/CloningBundleFactory.java    |   2 -
 .../beam/runners/direct/CommittedBundle.java    |  82 ++
 .../beam/runners/direct/CommittedResult.java    |   3 +-
 .../beam/runners/direct/CompletionCallback.java |   9 +-
 .../CopyOnAccessInMemoryStateInternals.java     |  20 +-
 ...ectGBKIntoKeyedWorkItemsOverrideFactory.java |   2 +-
 .../apache/beam/runners/direct/DirectGraph.java |  14 +-
 .../beam/runners/direct/DirectGraphVisitor.java |   2 +-
 .../beam/runners/direct/DirectGroupByKey.java   |   2 +-
 .../direct/DirectGroupByKeyOverrideFactory.java |   2 +-
 .../beam/runners/direct/DirectMetrics.java      |  25 +-
 .../beam/runners/direct/DirectOptions.java      |   2 +-
 .../beam/runners/direct/DirectRegistrar.java    |  12 +-
 .../beam/runners/direct/DirectRunner.java       | 158 +---
 .../runners/direct/DirectTimerInternals.java    |   2 +-
 .../runners/direct/DisplayDataValidator.java    |   6 +-
 .../runners/direct/DoFnLifecycleManager.java    |   2 +-
 .../beam/runners/direct/EmptyInputProvider.java |   3 +-
 .../runners/direct/EmptyTransformEvaluator.java |  50 --
 .../beam/runners/direct/EvaluationContext.java  |  14 +-
 .../runners/direct/ExecutorServiceFactory.java  |   2 +-
 .../direct/ExecutorServiceParallelExecutor.java |  42 +-
 .../runners/direct/FlattenEvaluatorFactory.java |   4 +-
 .../GroupAlsoByWindowEvaluatorFactory.java      |  18 +-
 .../direct/GroupByKeyOnlyEvaluatorFactory.java  |   4 +-
 .../ImmutabilityCheckingBundleFactory.java      |   2 -
 .../direct/ImmutabilityEnforcementFactory.java  |   3 +-
 .../direct/ImmutableListBundleFactory.java      |   2 -
 .../beam/runners/direct/ModelEnforcement.java   |  12 +-
 .../runners/direct/ModelEnforcementFactory.java |   5 +-
 .../beam/runners/direct/NanosOffsetClock.java   |   2 +-
 .../runners/direct/PCollectionViewWindow.java   |  67 ++
 .../runners/direct/PCollectionViewWriter.java   |  34 +
 .../beam/runners/direct/ParDoEvaluator.java     |   7 +-
 .../runners/direct/ParDoEvaluatorFactory.java   |   3 +-
 .../direct/ParDoMultiOverrideFactory.java       |   7 +-
 .../direct/PassthroughTransformEvaluator.java   |   3 +-
 .../beam/runners/direct/PipelineExecutor.java   |   3 +-
 .../beam/runners/direct/RootInputProvider.java  |   3 +-
 .../runners/direct/RootProviderRegistry.java    |   3 +-
 .../beam/runners/direct/SideInputContainer.java |   7 +-
 ...littableProcessElementsEvaluatorFactory.java |  18 +-
 .../direct/StatefulParDoEvaluatorFactory.java   |   7 +-
 .../apache/beam/runners/direct/StepAndKey.java  |   2 +-
 .../runners/direct/StepTransformResult.java     |   7 +-
 .../direct/TestStreamEvaluatorFactory.java      |   6 +-
 .../beam/runners/direct/TransformEvaluator.java |   3 +-
 .../direct/TransformEvaluatorFactory.java       |   8 +-
 .../direct/TransformEvaluatorRegistry.java      |   3 +-
 .../beam/runners/direct/TransformExecutor.java  |  25 +-
 .../beam/runners/direct/TransformResult.java    |   7 +-
 .../direct/UnboundedReadEvaluatorFactory.java   |   4 +-
 .../beam/runners/direct/UncommittedBundle.java  |  57 ++
 .../runners/direct/ViewEvaluatorFactory.java    |   5 +-
 .../runners/direct/ViewOverrideFactory.java     |   2 +-
 .../direct/WatermarkCallbackExecutor.java       |   4 +-
 .../beam/runners/direct/WatermarkManager.java   |   7 +-
 .../runners/direct/WindowEvaluatorFactory.java  |   4 +-
 .../direct/WriteWithShardingFactory.java        |   2 +-
 .../direct/BoundedReadEvaluatorFactoryTest.java |   4 +-
 .../direct/CloningBundleFactoryTest.java        |  32 +-
 .../runners/direct/CommittedResultTest.java     |  16 +-
 .../CopyOnAccessInMemoryStateInternalsTest.java |  16 +-
 .../runners/direct/DirectGraphVisitorTest.java  |   2 +-
 .../beam/runners/direct/DirectGraphs.java       |   2 +-
 .../DirectGroupByKeyOverrideFactoryTest.java    |   2 +-
 .../beam/runners/direct/DirectMetricsTest.java  |  15 +-
 .../direct/DirectRunnerApiSurfaceTest.java      |  68 ++
 .../beam/runners/direct/DirectRunnerTest.java   |  11 +-
 .../direct/DirectTimerInternalsTest.java        |   2 +-
 ...leManagerRemovingTransformEvaluatorTest.java |   2 +-
 .../runners/direct/EvaluationContextTest.java   |  13 +-
 .../direct/FlattenEvaluatorFactoryTest.java     |   4 +-
 .../direct/GroupByKeyEvaluatorFactoryTest.java  |   2 -
 .../GroupByKeyOnlyEvaluatorFactoryTest.java     |   2 -
 .../ImmutabilityCheckingBundleFactoryTest.java  |   2 -
 .../ImmutabilityEnforcementFactoryTest.java     |   3 +-
 .../direct/ImmutableListBundleFactoryTest.java  |   2 -
 .../direct/KeyedPValueTrackingVisitorTest.java  |   2 +-
 .../apache/beam/runners/direct/MockClock.java   |   2 +-
 .../beam/runners/direct/ParDoEvaluatorTest.java |   5 +-
 .../runners/direct/SideInputContainerTest.java  |   8 +-
 .../StatefulParDoEvaluatorFactoryTest.java      |  14 +-
 .../runners/direct/StepTransformResultTest.java |   3 +-
 .../direct/TestStreamEvaluatorFactoryTest.java  |   3 +-
 .../runners/direct/TransformExecutorTest.java   |   8 +-
 .../UnboundedReadEvaluatorFactoryTest.java      |  13 +-
 .../direct/ViewEvaluatorFactoryTest.java        |   6 +-
 .../runners/direct/ViewOverrideFactoryTest.java |   6 +-
 .../direct/WatermarkCallbackExecutorTest.java   |   4 +-
 .../runners/direct/WatermarkManagerTest.java    |   6 +-
 .../direct/WindowEvaluatorFactoryTest.java      |   2 -
 .../direct/WriteWithShardingFactoryTest.java    |  61 +-
 runners/flink/pom.xml                           |   7 +-
 .../flink/FlinkBatchTransformTranslators.java   |   4 +-
 .../flink/FlinkBatchTranslationContext.java     |   4 +-
 .../flink/FlinkDetachedRunnerResult.java        |   1 -
 .../apache/beam/runners/flink/FlinkRunner.java  |   5 +-
 .../runners/flink/FlinkRunnerRegistrar.java     |   2 +-
 .../beam/runners/flink/FlinkRunnerResult.java   |   8 +-
 .../flink/FlinkStreamingPipelineTranslator.java |   2 +-
 .../FlinkStreamingTransformTranslators.java     |  84 +-
 .../flink/FlinkStreamingTranslationContext.java |   2 +-
 .../flink/FlinkStreamingViewOverrides.java      |   2 +-
 .../beam/runners/flink/TestFlinkRunner.java     |   2 +-
 .../metrics/DoFnRunnerWithMetricsUpdate.java    |  17 +-
 .../flink/metrics/FlinkMetricContainer.java     | 276 ++-----
 .../flink/metrics/FlinkMetricResults.java       | 146 ----
 .../flink/metrics/MetricsAccumulator.java       |  60 ++
 .../flink/metrics/ReaderInvocationUtil.java     |  12 +-
 .../functions/AbstractFlinkCombineRunner.java   |   4 +-
 .../functions/FlinkDoFnFunction.java            |   3 +-
 .../FlinkMergingNonShuffleReduceFunction.java   |   2 +-
 .../functions/FlinkPartialReduceFunction.java   |   2 +-
 .../functions/FlinkReduceFunction.java          |   2 +-
 .../functions/FlinkSideInputReader.java         |   4 +-
 .../functions/FlinkStatefulDoFnFunction.java    |   3 +-
 .../functions/HashingFlinkCombineRunner.java    |   4 +-
 .../functions/SortingFlinkCombineRunner.java    |   4 +-
 .../translation/types/CoderTypeSerializer.java  |   4 +-
 .../types/EncodedValueSerializer.java           |   2 -
 .../utils/SerializedPipelineOptions.java        |  20 +-
 .../translation/wrappers/SourceInputFormat.java |   8 +-
 .../wrappers/streaming/DoFnOperator.java        |  58 +-
 .../streaming/SingletonKeyedWorkItemCoder.java  |  20 +-
 .../streaming/SplittableDoFnOperator.java       |  30 +-
 .../wrappers/streaming/WindowDoFnOperator.java  |   2 +-
 .../streaming/io/BoundedSourceWrapper.java      |   8 +-
 .../wrappers/streaming/io/DedupingOperator.java | 187 +++++
 .../streaming/io/UnboundedSourceWrapper.java    |  24 +-
 .../state/FlinkBroadcastStateInternals.java     |  20 +-
 .../state/FlinkKeyGroupStateInternals.java      |  29 +-
 .../state/FlinkSplitStateInternals.java         |  20 +-
 .../streaming/state/FlinkStateInternals.java    |  22 +-
 .../beam/runners/flink/PipelineOptionsTest.java |  89 +-
 .../flink/streaming/DedupingOperatorTest.java   | 131 +++
 .../flink/streaming/DoFnOperatorTest.java       |  16 +-
 .../FlinkBroadcastStateInternalsTest.java       |  10 +-
 .../FlinkKeyGroupStateInternalsTest.java        |   4 +-
 .../streaming/FlinkSplitStateInternalsTest.java |   4 +-
 .../streaming/FlinkStateInternalsTest.java      |  12 +-
 .../streaming/UnboundedSourceWrapperTest.java   |  29 +-
 runners/gearpump/pom.xml                        |   2 +-
 .../gearpump/GearpumpPipelineTranslator.java    |   4 +-
 .../beam/runners/gearpump/GearpumpRunner.java   |   2 +-
 .../gearpump/GearpumpRunnerRegistrar.java       |   2 +-
 .../runners/gearpump/TestGearpumpRunner.java    |   2 +-
 .../ParDoSingleOutputTranslator.java            |   2 +-
 .../translators/TranslationContext.java         |   2 +-
 .../translators/WindowAssignTranslator.java     |   2 +-
 .../translators/functions/DoFnFunction.java     |   2 +-
 .../translators/utils/DoFnRunnerFactory.java    |   4 +-
 runners/google-cloud-dataflow-java/pom.xml      |  76 +-
 .../beam/runners/dataflow/AssignWindows.java    |   2 +-
 .../dataflow/BatchStatefulParDoOverrides.java   |   4 +-
 .../runners/dataflow/BatchViewOverrides.java    |  32 +-
 .../beam/runners/dataflow/DataflowMetrics.java  |   4 +-
 .../runners/dataflow/DataflowPipelineJob.java   |  20 +-
 .../dataflow/DataflowPipelineRegistrar.java     |   4 +-
 .../dataflow/DataflowPipelineTranslator.java    |  40 +-
 .../beam/runners/dataflow/DataflowRunner.java   | 110 +--
 .../dataflow/PrimitiveParDoSingleFactory.java   |   2 +-
 .../beam/runners/dataflow/ReadTranslator.java   |  22 +-
 .../dataflow/ReshuffleOverrideFactory.java      |   8 +-
 .../dataflow/StreamingViewOverrides.java        |   2 +-
 .../dataflow/TestDataflowPipelineOptions.java   |  28 +
 .../runners/dataflow/TestDataflowRunner.java    | 388 +++++++++
 .../runners/dataflow/TransformTranslator.java   |   2 +-
 .../dataflow/internal/CustomSources.java        |  18 +-
 .../runners/dataflow/internal/IsmFormat.java    |  99 ++-
 .../options/DataflowPipelineOptions.java        |  18 +-
 .../testing/TestDataflowPipelineOptions.java    |  28 -
 .../dataflow/testing/TestDataflowRunner.java    | 326 --------
 .../runners/dataflow/testing/package-info.java  |  24 -
 .../util/AvroCoderCloudObjectTranslator.java    |   2 -
 .../beam/runners/dataflow/util/CloudObject.java |   1 -
 .../runners/dataflow/util/CloudObjectKinds.java |   2 -
 .../dataflow/util/CloudObjectTranslator.java    |   2 -
 .../dataflow/util/CloudObjectTranslators.java   |  23 +-
 .../runners/dataflow/util/CloudObjects.java     |   1 -
 .../CoderCloudObjectTranslatorRegistrar.java    |   1 -
 .../dataflow/util/DataflowTemplateJob.java      |   1 -
 ...aultCoderCloudObjectTranslatorRegistrar.java |   2 -
 .../beam/runners/dataflow/util/DoFnInfo.java    |   2 +-
 .../beam/runners/dataflow/util/GcsStager.java   |  26 +-
 .../beam/runners/dataflow/util/PackageUtil.java |  41 +-
 .../runners/dataflow/util/PropertyNames.java    | 112 +++
 .../runners/dataflow/util/RandomAccessData.java |  26 +-
 .../SerializableCoderCloudObjectTranslator.java |   2 -
 .../beam/runners/dataflow/util/Serializer.java  | 262 ------
 .../beam/runners/dataflow/util/Structs.java     | 372 +++++++++
 .../BatchStatefulParDoOverridesTest.java        |   6 +-
 .../runners/dataflow/DataflowMetricsTest.java   |   7 +-
 .../dataflow/DataflowPipelineJobTest.java       |  18 +-
 .../dataflow/DataflowPipelineRegistrarTest.java |   2 +-
 .../DataflowPipelineTranslatorTest.java         | 133 ++-
 .../runners/dataflow/DataflowRunnerTest.java    |  90 +--
 .../PrimitiveParDoSingleFactoryTest.java        |   2 +-
 .../dataflow/TestDataflowRunnerTest.java        | 634 +++++++++++++++
 .../options/DataflowPipelineOptionsTest.java    |  21 +-
 .../options/DataflowProfilingOptionsTest.java   |   4 +-
 .../DataflowWorkerLoggingOptionsTest.java       |   4 +-
 .../testing/TestDataflowRunnerTest.java         | 661 ---------------
 .../transforms/DataflowGroupByKeyTest.java      |   4 +-
 .../dataflow/transforms/DataflowViewTest.java   |   4 +-
 .../runners/dataflow/util/CloudObjectsTest.java |   9 +-
 .../runners/dataflow/util/PackageUtilTest.java  | 145 ++--
 .../dataflow/util/RandomAccessDataTest.java     |   6 +-
 .../beam/runners/dataflow/util/StructsTest.java | 206 +++++
 runners/pom.xml                                 |   2 +-
 runners/spark/README.md                         | 124 ---
 runners/spark/pom.xml                           |  54 +-
 .../beam/runners/spark/SparkPipelineResult.java |   8 +-
 .../apache/beam/runners/spark/SparkRunner.java  |   8 +-
 .../beam/runners/spark/SparkRunnerDebugger.java |   7 +-
 .../runners/spark/SparkRunnerRegistrar.java     |   2 +-
 .../beam/runners/spark/TestSparkRunner.java     |  83 +-
 .../spark/aggregators/NamedAggregators.java     |   6 +-
 .../beam/runners/spark/coders/CoderHelpers.java |   1 -
 .../spark/coders/StatelessJavaSerializer.java   |   1 -
 .../beam/runners/spark/io/CreateStream.java     |   2 +-
 .../beam/runners/spark/io/MicrobatchSource.java |   2 +-
 .../apache/beam/runners/spark/io/SourceRDD.java | 177 ++--
 .../runners/spark/io/SparkUnboundedSource.java  |  20 +-
 .../runners/spark/metrics/AggregatorMetric.java |   1 -
 .../spark/metrics/MetricsAccumulator.java       |  20 +-
 .../spark/metrics/MetricsAccumulatorParam.java  |  20 +-
 .../runners/spark/metrics/SparkBeamMetric.java  |  11 +-
 .../spark/metrics/SparkBeamMetricSource.java    |   3 +-
 .../spark/metrics/SparkMetricResults.java       | 172 ----
 .../spark/metrics/SparkMetricsContainer.java    | 174 ----
 .../SparkGroupAlsoByWindowViaWindowSet.java     |  17 +-
 .../spark/stateful/SparkStateInternals.java     |  20 +-
 .../spark/stateful/SparkTimerInternals.java     |   2 +-
 .../spark/stateful/StateSpecFunctions.java      |   9 +-
 .../translation/DoFnRunnerWithMetrics.java      |  11 +-
 .../spark/translation/EvaluationContext.java    |  18 +-
 .../translation/GroupCombineFunctions.java      |   5 +-
 .../spark/translation/MultiDoFnFunction.java    |   8 +-
 .../translation/SparkAbstractCombineFn.java     |   4 +-
 .../spark/translation/SparkAssignWindowFn.java  |   4 +-
 .../spark/translation/SparkGlobalCombineFn.java |   2 +-
 ...SparkGroupAlsoByWindowViaOutputBufferFn.java |   6 +-
 .../spark/translation/SparkKeyedCombineFn.java  |   2 +-
 .../spark/translation/SparkRuntimeContext.java  |  26 +-
 .../translation/StorageLevelPTransform.java     |   2 +-
 .../spark/translation/TransformTranslator.java  |   8 +-
 .../spark/translation/TranslationUtils.java     |   3 +-
 .../SparkRunnerStreamingContextFactory.java     |   2 +-
 .../streaming/StreamingTransformTranslator.java |   8 +-
 .../spark/util/GlobalWatermarkHolder.java       |   1 -
 .../util/SinglePrimitiveOutputPTransform.java   |   2 +-
 .../spark/util/SparkSideInputReader.java        |   4 +-
 .../apache/beam/runners/spark/CacheTest.java    |  14 +-
 .../beam/runners/spark/ForceStreamingTest.java  |  18 +-
 .../apache/beam/runners/spark/PipelineRule.java | 109 ---
 .../runners/spark/ProvidedSparkContextTest.java |  10 +-
 .../runners/spark/SparkRunnerDebuggerTest.java  |  15 +-
 .../beam/runners/spark/StreamingTest.java       |  23 +
 .../TestSparkPipelineOptionsRegistrar.java      |  36 +
 .../metrics/sink/SparkMetricsSinkTest.java      |  12 +-
 .../beam/runners/spark/io/AvroPipelineTest.java |  10 +-
 .../beam/runners/spark/io/NumShardsTest.java    |   6 +-
 .../spark/io/ReaderToIteratorAdapterTest.java   | 145 ++++
 .../translation/SparkRuntimeContextTest.java    | 122 +++
 .../spark/translation/StorageLevelTest.java     |  31 +-
 .../translation/streaming/CreateStreamTest.java |  53 +-
 .../ResumeFromCheckpointStreamingTest.java      |  64 +-
 .../streaming/StreamingSourceMetricsTest.java   |  16 +-
 .../streaming/TrackStreamingSourcesTest.java    |   4 +-
 sdks/common/fn-api/pom.xml                      |   2 +-
 sdks/common/pom.xml                             |   2 +-
 sdks/common/runner-api/pom.xml                  |   5 +-
 sdks/java/build-tools/pom.xml                   |   2 +-
 .../src/main/resources/beam/findbugs-filter.xml |  12 +-
 sdks/java/core/pom.xml                          |  93 ++-
 .../main/java/org/apache/beam/sdk/Pipeline.java | 101 ++-
 .../org/apache/beam/sdk/PipelineResult.java     |  11 +-
 .../org/apache/beam/sdk/PipelineRunner.java     |  61 ++
 .../beam/sdk/annotations/Experimental.java      |  35 +-
 .../apache/beam/sdk/annotations/Internal.java   |   6 +-
 .../org/apache/beam/sdk/coders/AtomicCoder.java |  85 ++
 .../org/apache/beam/sdk/coders/AvroCoder.java   |  70 +-
 .../apache/beam/sdk/coders/BigDecimalCoder.java |  25 +-
 .../beam/sdk/coders/BigEndianIntegerCoder.java  |  10 +-
 .../beam/sdk/coders/BigEndianLongCoder.java     |  12 +-
 .../apache/beam/sdk/coders/BigIntegerCoder.java |  21 +-
 .../org/apache/beam/sdk/coders/BitSetCoder.java |  15 +-
 .../apache/beam/sdk/coders/ByteArrayCoder.java  |  41 +-
 .../org/apache/beam/sdk/coders/ByteCoder.java   |  10 +-
 .../sdk/coders/CannotProvideCoderException.java |   2 +-
 .../java/org/apache/beam/sdk/coders/Coder.java  | 178 +++-
 .../apache/beam/sdk/coders/CoderFactories.java  | 290 -------
 .../apache/beam/sdk/coders/CoderFactory.java    |  44 -
 .../apache/beam/sdk/coders/CoderProvider.java   |  19 +-
 .../beam/sdk/coders/CoderProviderRegistrar.java |  42 +
 .../apache/beam/sdk/coders/CoderProviders.java  | 240 +++---
 .../apache/beam/sdk/coders/CoderRegistrar.java  |  45 --
 .../apache/beam/sdk/coders/CoderRegistry.java   | 618 +++++---------
 .../apache/beam/sdk/coders/CollectionCoder.java |  22 -
 .../org/apache/beam/sdk/coders/CustomCoder.java |  26 +-
 .../apache/beam/sdk/coders/DefaultCoder.java    | 119 ++-
 .../apache/beam/sdk/coders/DelegateCoder.java   |  11 +
 .../org/apache/beam/sdk/coders/DoubleCoder.java |  10 +-
 .../apache/beam/sdk/coders/DurationCoder.java   |  18 +-
 .../apache/beam/sdk/coders/InstantCoder.java    |  18 +-
 .../apache/beam/sdk/coders/IterableCoder.java   |  22 -
 .../beam/sdk/coders/IterableLikeCoder.java      |  35 +-
 .../org/apache/beam/sdk/coders/KvCoder.java     |  51 +-
 .../beam/sdk/coders/LengthPrefixCoder.java      |  28 +-
 .../org/apache/beam/sdk/coders/ListCoder.java   |  24 +-
 .../org/apache/beam/sdk/coders/MapCoder.java    |  47 +-
 .../apache/beam/sdk/coders/NullableCoder.java   |  31 +-
 .../beam/sdk/coders/SerializableCoder.java      |  66 +-
 .../org/apache/beam/sdk/coders/SetCoder.java    |  25 -
 .../beam/sdk/coders/StringDelegateCoder.java    |  12 +-
 .../apache/beam/sdk/coders/StringUtf8Coder.java |  29 +-
 .../beam/sdk/coders/StructuralByteArray.java    |   4 +-
 .../apache/beam/sdk/coders/StructuredCoder.java |  74 +-
 .../beam/sdk/coders/TextualIntegerCoder.java    |  17 +-
 .../org/apache/beam/sdk/coders/VarIntCoder.java |  10 +-
 .../apache/beam/sdk/coders/VarLongCoder.java    |  15 +-
 .../org/apache/beam/sdk/coders/VoidCoder.java   |  12 +-
 .../apache/beam/sdk/coders/package-info.java    |   2 +-
 .../java/org/apache/beam/sdk/io/AvroIO.java     | 219 +++--
 .../java/org/apache/beam/sdk/io/AvroSink.java   |  34 +-
 .../java/org/apache/beam/sdk/io/AvroSource.java |  23 +-
 .../sdk/io/BoundedReadFromUnboundedSource.java  |   4 +-
 .../apache/beam/sdk/io/CompressedSource.java    |   6 +
 .../beam/sdk/io/DefaultFilenamePolicy.java      | 169 ++++
 .../org/apache/beam/sdk/io/FileBasedSink.java   | 803 ++++++++++---------
 .../org/apache/beam/sdk/io/FileBasedSource.java |  39 +-
 .../java/org/apache/beam/sdk/io/FileSystem.java |   3 +
 .../apache/beam/sdk/io/FileSystemRegistrar.java |   3 +
 .../org/apache/beam/sdk/io/FileSystems.java     |  65 +-
 .../beam/sdk/io/LocalFileSystemRegistrar.java   |   3 +
 .../org/apache/beam/sdk/io/LocalResources.java  |   3 +
 .../main/java/org/apache/beam/sdk/io/Read.java  |   8 +-
 .../java/org/apache/beam/sdk/io/TFRecordIO.java | 153 ++--
 .../java/org/apache/beam/sdk/io/TextIO.java     | 255 +++---
 .../java/org/apache/beam/sdk/io/TextSink.java   |  32 +-
 .../org/apache/beam/sdk/io/UnboundedSource.java |  23 +-
 .../java/org/apache/beam/sdk/io/WriteFiles.java | 285 ++++---
 .../org/apache/beam/sdk/io/fs/ResourceId.java   |   3 +
 .../apache/beam/sdk/metrics/CounterCell.java    |  67 --
 .../org/apache/beam/sdk/metrics/DirtyState.java |  98 ---
 .../beam/sdk/metrics/DistributionCell.java      |  64 --
 .../beam/sdk/metrics/DistributionData.java      |  60 --
 .../beam/sdk/metrics/DistributionResult.java    |   3 +
 .../org/apache/beam/sdk/metrics/GaugeCell.java  |  54 --
 .../org/apache/beam/sdk/metrics/GaugeData.java  |  81 --
 .../apache/beam/sdk/metrics/GaugeResult.java    |   3 +
 .../org/apache/beam/sdk/metrics/Metric.java     |   7 +-
 .../org/apache/beam/sdk/metrics/MetricCell.java |  41 -
 .../beam/sdk/metrics/MetricFiltering.java       |  99 ---
 .../org/apache/beam/sdk/metrics/MetricKey.java  |  41 -
 .../apache/beam/sdk/metrics/MetricUpdates.java  |  78 --
 .../org/apache/beam/sdk/metrics/Metrics.java    |   6 +-
 .../beam/sdk/metrics/MetricsContainer.java      | 129 +--
 .../beam/sdk/metrics/MetricsEnvironment.java    |  14 +-
 .../org/apache/beam/sdk/metrics/MetricsMap.java |  86 --
 .../apache/beam/sdk/metrics/SinkMetrics.java    |   4 +
 .../apache/beam/sdk/metrics/SourceMetrics.java  |   3 +
 .../DefaultPipelineOptionsRegistrar.java        |   3 +-
 .../beam/sdk/options/PipelineOptionSpec.java    |   4 +-
 .../beam/sdk/options/PipelineOptions.java       |   5 +-
 .../sdk/options/PipelineOptionsFactory.java     |   2 +-
 .../apache/beam/sdk/options/ValueProvider.java  |  15 +-
 .../beam/sdk/options/ValueProviderUtils.java    |  60 --
 .../apache/beam/sdk/options/ValueProviders.java |  61 ++
 .../java/org/apache/beam/sdk/package-info.java  |   2 +-
 .../beam/sdk/runners/AppliedPTransform.java     |  73 ++
 .../beam/sdk/runners/PTransformMatcher.java     |   7 +-
 .../beam/sdk/runners/PTransformOverride.java    |   6 +-
 .../sdk/runners/PTransformOverrideFactory.java  |   8 +-
 .../apache/beam/sdk/runners/PipelineRunner.java |  65 --
 .../sdk/runners/PipelineRunnerRegistrar.java    |   5 +
 .../beam/sdk/runners/TransformHierarchy.java    |  15 +-
 .../apache/beam/sdk/runners/package-info.java   |   4 +-
 .../org/apache/beam/sdk/state/BagState.java     |  36 +
 .../apache/beam/sdk/state/CombiningState.java   |  55 ++
 .../apache/beam/sdk/state/GroupingState.java    |  44 +
 .../org/apache/beam/sdk/state/MapState.java     |  73 ++
 .../apache/beam/sdk/state/ReadableState.java    |  51 ++
 .../apache/beam/sdk/state/ReadableStates.java   |  44 +
 .../org/apache/beam/sdk/state/SetState.java     |  47 ++
 .../java/org/apache/beam/sdk/state/State.java   |  36 +
 .../org/apache/beam/sdk/state/StateBinder.java  |  70 ++
 .../org/apache/beam/sdk/state/StateContext.java |  46 ++
 .../apache/beam/sdk/state/StateContexts.java    |  51 ++
 .../org/apache/beam/sdk/state/StateSpec.java    |  63 ++
 .../org/apache/beam/sdk/state/StateSpecs.java   | 683 ++++++++++++++++
 .../org/apache/beam/sdk/state/TimeDomain.java   |  49 ++
 .../java/org/apache/beam/sdk/state/Timer.java   |  70 ++
 .../org/apache/beam/sdk/state/TimerSpec.java    |  30 +
 .../org/apache/beam/sdk/state/TimerSpecs.java   |  39 +
 .../java/org/apache/beam/sdk/state/Timers.java  |  58 ++
 .../org/apache/beam/sdk/state/ValueState.java   |  35 +
 .../beam/sdk/state/WatermarkHoldState.java      |  41 +
 .../org/apache/beam/sdk/state/package-info.java |  22 +
 .../beam/sdk/testing/CoderProperties.java       |   6 +-
 .../apache/beam/sdk/testing/CrashingRunner.java |   2 +-
 .../beam/sdk/testing/FileChecksumMatcher.java   |   2 +-
 .../beam/sdk/testing/MatcherDeserializer.java   |   4 +-
 .../beam/sdk/testing/MatcherSerializer.java     |   4 +-
 .../apache/beam/sdk/testing/NeedsRunner.java    |   2 +-
 .../org/apache/beam/sdk/testing/PAssert.java    |   4 +-
 .../apache/beam/sdk/testing/TestPipeline.java   |  77 +-
 .../beam/sdk/testing/TestPipelineOptions.java   |   2 -
 .../org/apache/beam/sdk/testing/TestStream.java |   4 +-
 .../apache/beam/sdk/testing/UsesMapState.java   |   2 +-
 .../apache/beam/sdk/testing/UsesSetState.java   |   2 +-
 .../apache/beam/sdk/transforms/Aggregator.java  |  34 -
 .../beam/sdk/transforms/AppliedPTransform.java  |  73 --
 .../sdk/transforms/ApproximateQuantiles.java    |  90 ++-
 .../org/apache/beam/sdk/transforms/Combine.java |  44 +-
 .../beam/sdk/transforms/CombineFnBase.java      |  26 +-
 .../apache/beam/sdk/transforms/CombineFns.java  |  25 +-
 .../beam/sdk/transforms/CombineWithContext.java |   3 +-
 .../org/apache/beam/sdk/transforms/Count.java   |  12 +-
 .../org/apache/beam/sdk/transforms/Create.java  |  76 +-
 .../org/apache/beam/sdk/transforms/DoFn.java    |  91 ++-
 .../apache/beam/sdk/transforms/DoFnTester.java  |  66 +-
 .../org/apache/beam/sdk/transforms/Flatten.java |   2 +-
 .../apache/beam/sdk/transforms/GroupByKey.java  |   2 +-
 .../beam/sdk/transforms/GroupIntoBatches.java   |  71 +-
 .../org/apache/beam/sdk/transforms/Latest.java  |  15 +-
 .../beam/sdk/transforms/Materialization.java    |   8 +-
 .../beam/sdk/transforms/Materializations.java   |  11 +-
 .../org/apache/beam/sdk/transforms/Mean.java    |  19 +-
 .../apache/beam/sdk/transforms/PTransform.java  |  10 +-
 .../org/apache/beam/sdk/transforms/ParDo.java   |  20 +-
 .../org/apache/beam/sdk/transforms/Regex.java   |   1 -
 .../beam/sdk/transforms/ReifyTimestamps.java    |  79 ++
 .../apache/beam/sdk/transforms/Reshuffle.java   |  97 +++
 .../org/apache/beam/sdk/transforms/Top.java     |  46 +-
 .../org/apache/beam/sdk/transforms/View.java    |  40 +-
 .../org/apache/beam/sdk/transforms/ViewFn.java  |   6 +-
 .../apache/beam/sdk/transforms/WithKeys.java    |   4 +-
 .../beam/sdk/transforms/join/CoGbkResult.java   |  20 +-
 .../beam/sdk/transforms/join/UnionCoder.java    |  29 +-
 .../reflect/ByteBuddyDoFnInvokerFactory.java    |  31 +-
 .../reflect/ByteBuddyOnTimerInvokerFactory.java |  20 +-
 .../sdk/transforms/reflect/DoFnInvoker.java     |  23 +-
 .../sdk/transforms/reflect/DoFnSignature.java   |  50 +-
 .../sdk/transforms/reflect/DoFnSignatures.java  |  66 +-
 .../reflect/StableInvokerNamingStrategy.java    |  54 ++
 .../sdk/transforms/reflect/package-info.java    |   5 +-
 .../beam/sdk/transforms/windowing/AfterAll.java |   8 +-
 .../sdk/transforms/windowing/AfterEach.java     |   2 +-
 .../sdk/transforms/windowing/AfterFirst.java    |   5 +-
 .../sdk/transforms/windowing/AfterPane.java     |   5 +-
 .../windowing/AfterProcessingTime.java          |   6 +-
 .../transforms/windowing/AfterWatermark.java    |   6 +-
 .../transforms/windowing/DefaultTrigger.java    |  10 +-
 .../sdk/transforms/windowing/GlobalWindow.java  |  10 +-
 .../sdk/transforms/windowing/GlobalWindows.java |   5 +-
 .../transforms/windowing/IntervalWindow.java    |  19 +-
 .../transforms/windowing/InvalidWindows.java    |   2 +-
 .../MergeOverlappingIntervalWindows.java        |   6 +-
 .../beam/sdk/transforms/windowing/Never.java    |   4 +-
 .../windowing/NonMergingWindowFn.java           |   2 +-
 .../transforms/windowing/OrFinallyTrigger.java  |   7 +-
 .../beam/sdk/transforms/windowing/PaneInfo.java |  18 +-
 .../sdk/transforms/windowing/Repeatedly.java    |   6 +-
 .../transforms/windowing/ReshuffleTrigger.java  |  58 ++
 .../beam/sdk/transforms/windowing/Sessions.java |  13 +-
 .../transforms/windowing/TimestampCombiner.java |   5 +
 .../windowing/TimestampTransform.java           |  41 +-
 .../beam/sdk/transforms/windowing/Trigger.java  |  18 +-
 .../beam/sdk/transforms/windowing/Window.java   |  16 +-
 .../beam/sdk/transforms/windowing/WindowFn.java |  10 +-
 .../transforms/windowing/WindowMappingFn.java   |  12 +-
 .../org/apache/beam/sdk/util/ApiSurface.java    |  10 +-
 .../apache/beam/sdk/util/AppliedCombineFn.java  |   1 +
 .../java/org/apache/beam/sdk/util/BackOff.java  |  81 ++
 .../org/apache/beam/sdk/util/BackOffUtils.java  |  57 ++
 .../org/apache/beam/sdk/util/BitSetCoder.java   |  21 +-
 .../org/apache/beam/sdk/util/ClassPath.java     |   1 +
 .../apache/beam/sdk/util/CloudKnownType.java    | 143 ----
 .../org/apache/beam/sdk/util/CloudObject.java   | 187 -----
 .../org/apache/beam/sdk/util/CoderUtils.java    | 127 +--
 .../beam/sdk/util/CombineContextFactory.java    |  27 +-
 .../org/apache/beam/sdk/util/CombineFnUtil.java |   2 +-
 .../beam/sdk/util/ExplicitShardedFile.java      |   3 -
 .../beam/sdk/util/FileIOChannelFactory.java     | 223 -----
 .../sdk/util/FileIOChannelFactoryRegistrar.java |  38 -
 .../org/apache/beam/sdk/util/FluentBackoff.java |   1 -
 .../apache/beam/sdk/util/IOChannelFactory.java  | 127 ---
 .../sdk/util/IOChannelFactoryRegistrar.java     |  48 --
 .../apache/beam/sdk/util/IOChannelUtils.java    | 280 -------
 .../apache/beam/sdk/util/IdentityWindowFn.java  |   1 +
 .../apache/beam/sdk/util/NoopPathValidator.java |  49 --
 .../beam/sdk/util/NullSideInputReader.java      |  61 --
 .../beam/sdk/util/NumberedShardedFile.java      |   7 +-
 .../beam/sdk/util/PCollectionViewWindow.java    |  67 --
 .../apache/beam/sdk/util/PCollectionViews.java  | 496 ------------
 .../org/apache/beam/sdk/util/PathValidator.java |  48 --
 .../org/apache/beam/sdk/util/PropertyNames.java | 112 ---
 .../sdk/util/ReadyCheckingSideInputReader.java  |  34 -
 .../apache/beam/sdk/util/ReifyTimestamps.java   |  76 --
 .../org/apache/beam/sdk/util/ReleaseInfo.java   |  70 +-
 .../org/apache/beam/sdk/util/Reshuffle.java     |  88 --
 .../apache/beam/sdk/util/ReshuffleTrigger.java  |  52 --
 .../org/apache/beam/sdk/util/Serializer.java    | 147 ----
 .../org/apache/beam/sdk/util/ShardedFile.java   |   2 -
 .../apache/beam/sdk/util/SideInputReader.java   |  47 --
 .../java/org/apache/beam/sdk/util/Sleeper.java  |  48 ++
 .../java/org/apache/beam/sdk/util/Structs.java  | 371 ---------
 .../beam/sdk/util/SystemDoFnInternal.java       |   3 -
 .../org/apache/beam/sdk/util/TimeDomain.java    |  43 -
 .../java/org/apache/beam/sdk/util/Timer.java    |  67 --
 .../org/apache/beam/sdk/util/TimerSpec.java     |  30 -
 .../org/apache/beam/sdk/util/TimerSpecs.java    |  41 -
 .../java/org/apache/beam/sdk/util/Timers.java   |  58 --
 .../sdk/util/UploadIdResponseInterceptor.java   |  60 --
 .../apache/beam/sdk/util/ValueWithRecordId.java | 134 ----
 .../java/org/apache/beam/sdk/util/Values.java   |  88 --
 .../org/apache/beam/sdk/util/WindowedValue.java |  93 +--
 .../apache/beam/sdk/util/WindowingStrategy.java | 283 -------
 .../apache/beam/sdk/util/state/BagState.java    |  28 -
 .../beam/sdk/util/state/CombiningState.java     |  53 --
 .../beam/sdk/util/state/GroupingState.java      |  42 -
 .../apache/beam/sdk/util/state/MapState.java    |  79 --
 .../beam/sdk/util/state/ReadableState.java      |  51 --
 .../beam/sdk/util/state/ReadableStates.java     |  45 --
 .../apache/beam/sdk/util/state/SetState.java    |  45 --
 .../org/apache/beam/sdk/util/state/State.java   |  32 -
 .../apache/beam/sdk/util/state/StateBinder.java |  66 --
 .../beam/sdk/util/state/StateContext.java       |  43 -
 .../beam/sdk/util/state/StateContexts.java      |  51 --
 .../apache/beam/sdk/util/state/StateSpec.java   |  53 --
 .../apache/beam/sdk/util/state/StateSpecs.java  | 629 ---------------
 .../apache/beam/sdk/util/state/ValueState.java  |  37 -
 .../beam/sdk/util/state/WatermarkHoldState.java |  42 -
 .../beam/sdk/util/state/package-info.java       |  22 -
 .../java/org/apache/beam/sdk/values/PBegin.java |  26 +-
 .../org/apache/beam/sdk/values/PCollection.java | 162 +++-
 .../apache/beam/sdk/values/PCollectionList.java |  14 +-
 .../beam/sdk/values/PCollectionTuple.java       |  24 +-
 .../apache/beam/sdk/values/PCollectionView.java |  33 +-
 .../beam/sdk/values/PCollectionViews.java       | 495 ++++++++++++
 .../java/org/apache/beam/sdk/values/PDone.java  |  12 +-
 .../org/apache/beam/sdk/values/POutput.java     |  25 +-
 .../java/org/apache/beam/sdk/values/PValue.java |  12 +-
 .../org/apache/beam/sdk/values/PValueBase.java  |  61 +-
 .../apache/beam/sdk/values/TaggedPValue.java    |   6 +-
 .../beam/sdk/values/TimestampedValue.java       |  30 +-
 .../org/apache/beam/sdk/values/TupleTag.java    |  22 -
 .../org/apache/beam/sdk/values/TypedPValue.java | 208 -----
 .../beam/sdk/values/ValueInSingleWindow.java    |  29 +-
 .../beam/sdk/values/ValueWithRecordId.java      | 149 ++++
 .../beam/sdk/values/WindowingStrategy.java      | 283 +++++++
 .../org/apache/beam/SdkCoreApiSurfaceTest.java  |   1 -
 .../java/org/apache/beam/sdk/PipelineTest.java  |   3 +-
 .../apache/beam/sdk/coders/AvroCoderTest.java   |  11 +-
 .../beam/sdk/coders/BigDecimalCoderTest.java    |   5 +-
 .../beam/sdk/coders/BigIntegerCoderTest.java    |   5 +-
 .../beam/sdk/coders/CoderFactoriesTest.java     | 100 ---
 .../beam/sdk/coders/CoderProvidersTest.java     |  82 +-
 .../beam/sdk/coders/CoderRegistryTest.java      | 191 ++---
 .../apache/beam/sdk/coders/CustomCoderTest.java |   4 +-
 .../beam/sdk/coders/DefaultCoderTest.java       |  65 +-
 .../beam/sdk/coders/DelegateCoderTest.java      |  25 -
 .../beam/sdk/coders/IterableCoderTest.java      |  17 -
 .../beam/sdk/coders/LengthPrefixCoderTest.java  |  24 +-
 .../apache/beam/sdk/coders/ListCoderTest.java   |  17 -
 .../apache/beam/sdk/coders/MapCoderTest.java    |  20 -
 .../beam/sdk/coders/NullableCoderTest.java      |  31 +-
 .../beam/sdk/coders/SerializableCoderTest.java  |  47 +-
 .../beam/sdk/coders/StructuredCoderTest.java    |  12 +-
 .../java/org/apache/beam/sdk/io/AvroIOTest.java | 101 ++-
 .../beam/sdk/io/CompressedSourceTest.java       |  25 +-
 .../beam/sdk/io/DefaultFilenamePolicyTest.java  |  55 ++
 .../sdk/io/DrunkWritableByteChannelFactory.java |   1 -
 .../apache/beam/sdk/io/FileBasedSinkTest.java   | 460 +++++------
 .../apache/beam/sdk/io/LocalResourceIdTest.java |   6 +
 .../java/org/apache/beam/sdk/io/SimpleSink.java |  31 +-
 .../org/apache/beam/sdk/io/TFRecordIOTest.java  |   2 -
 .../java/org/apache/beam/sdk/io/TextIOTest.java |  57 +-
 .../org/apache/beam/sdk/io/WriteFilesTest.java  |  41 +-
 .../apache/beam/sdk/io/fs/ResourceIdTester.java | 150 ++++
 .../beam/sdk/metrics/CounterCellTest.java       |  55 --
 .../apache/beam/sdk/metrics/DirtyStateTest.java |  56 --
 .../beam/sdk/metrics/DistributionCellTest.java  |  53 --
 .../apache/beam/sdk/metrics/GaugeCellTest.java  |  48 --
 .../beam/sdk/metrics/MetricFilteringTest.java   | 145 ----
 .../apache/beam/sdk/metrics/MetricMatchers.java | 242 ------
 .../beam/sdk/metrics/MetricResultsMatchers.java | 190 +++++
 .../beam/sdk/metrics/MetricsContainerTest.java  | 129 ---
 .../sdk/metrics/MetricsEnvironmentTest.java     |  23 +-
 .../apache/beam/sdk/metrics/MetricsMapTest.java | 103 ---
 .../apache/beam/sdk/metrics/MetricsTest.java    |  47 +-
 .../sdk/options/PipelineOptionsFactoryTest.java |   4 +-
 .../beam/sdk/options/PipelineOptionsTest.java   |   1 -
 .../sdk/options/ProxyInvocationHandlerTest.java |   4 +-
 .../beam/sdk/options/ValueProviderTest.java     |  23 +-
 .../sdk/options/ValueProviderUtilsTest.java     |  78 --
 .../beam/sdk/options/ValueProvidersTest.java    |  79 ++
 .../beam/sdk/runners/PipelineRunnerTest.java    |   1 +
 .../sdk/runners/TransformHierarchyTest.java     |   5 +-
 .../beam/sdk/runners/TransformTreeTest.java     |   2 +-
 .../beam/sdk/testing/CoderPropertiesTest.java   |  73 +-
 .../beam/sdk/testing/CrashingRunnerTest.java    |   2 +-
 .../beam/sdk/testing/ExpectedLogsTest.java      |   2 +-
 .../sdk/testing/FastNanoClockAndSleeper.java    |  47 --
 .../testing/FastNanoClockAndSleeperTest.java    |  47 --
 .../sdk/testing/FileChecksumMatcherTest.java    |  12 +-
 .../apache/beam/sdk/testing/PAssertTest.java    |  14 +-
 .../sdk/testing/PCollectionViewTesting.java     |   2 +-
 .../sdk/testing/SerializableMatchersTest.java   |   9 +-
 .../beam/sdk/testing/SystemNanoTimeSleeper.java |   2 +-
 .../beam/sdk/testing/TestPipelineTest.java      |   6 +-
 .../beam/sdk/testing/WindowSupplierTest.java    |  10 +-
 .../sdk/transforms/ApproximateUniqueTest.java   |   2 +-
 .../beam/sdk/transforms/CombineFnsTest.java     |  30 +-
 .../apache/beam/sdk/transforms/CombineTest.java |  39 +-
 .../apache/beam/sdk/transforms/CreateTest.java  |  25 +-
 .../beam/sdk/transforms/DoFnTesterTest.java     |  14 +-
 .../sdk/transforms/FlatMapElementsTest.java     |   4 +-
 .../beam/sdk/transforms/GroupByKeyTest.java     |  15 +-
 .../beam/sdk/transforms/LatestFnTest.java       |   2 +-
 .../beam/sdk/transforms/MapElementsTest.java    |   4 +-
 .../beam/sdk/transforms/ParDoLifecycleTest.java |  16 +-
 .../apache/beam/sdk/transforms/ParDoTest.java   | 258 +++---
 .../apache/beam/sdk/transforms/RegexTest.java   |   2 -
 .../sdk/transforms/ReifyTimestampsTest.java     | 140 ++++
 .../beam/sdk/transforms/ReshuffleTest.java      | 293 +++++++
 .../beam/sdk/transforms/SplittableDoFnTest.java |   4 +-
 .../apache/beam/sdk/transforms/ViewTest.java    |  17 +-
 .../display/DisplayDataEvaluator.java           |   2 +-
 .../transforms/reflect/DoFnInvokersTest.java    |  73 +-
 .../DoFnSignaturesSplittableDoFnTest.java       |   4 +-
 .../transforms/reflect/DoFnSignaturesTest.java  |  35 +-
 .../transforms/reflect/OnTimerInvokersTest.java |  38 +-
 .../testhelper/DoFnInvokersTestHelper.java      |   6 +-
 .../transforms/windowing/GlobalWindowTest.java  |   2 +-
 .../windowing/ReshuffleTriggerTest.java         |  42 +
 .../sdk/transforms/windowing/WindowTest.java    |   4 +-
 ...BufferedElementCountingOutputStreamTest.java |   5 +-
 .../apache/beam/sdk/util/CoderUtilsTest.java    |   8 +-
 .../apache/beam/sdk/util/CombineFnUtilTest.java |   2 +-
 .../util/FileIOChannelFactoryRegistrarTest.java |  44 -
 .../beam/sdk/util/FileIOChannelFactoryTest.java | 242 ------
 .../apache/beam/sdk/util/FluentBackoffTest.java |   1 -
 .../beam/sdk/util/IOChannelUtilsTest.java       | 118 ---
 .../beam/sdk/util/NumberedShardedFileTest.java  |  14 +-
 .../beam/sdk/util/ReifyTimestampsTest.java      | 109 ---
 .../org/apache/beam/sdk/util/ReshuffleTest.java | 270 -------
 .../beam/sdk/util/ReshuffleTriggerTest.java     |  44 -
 .../beam/sdk/util/SerializableUtilsTest.java    |   8 +-
 .../apache/beam/sdk/util/SerializerTest.java    | 162 ----
 .../org/apache/beam/sdk/util/StructsTest.java   | 206 -----
 .../util/UploadIdResponseInterceptorTest.java   |  98 ---
 .../beam/sdk/util/ValueWithRecordIdTest.java    |  34 -
 .../beam/sdk/values/PCollectionTupleTest.java   |   1 -
 .../apache/beam/sdk/values/TypedPValueTest.java |  28 +-
 .../beam/sdk/values/ValueWithRecordIdTest.java  |  34 +
 .../google-cloud-platform-core/pom.xml          |   8 +-
 .../sdk/extensions/gcp/options/GcpOptions.java  |   9 +-
 .../sdk/extensions/gcp/options/GcsOptions.java  |   4 +-
 .../gcp/storage/GcsCreateOptions.java           |  56 ++
 .../extensions/gcp/storage/GcsFileSystem.java   |  20 +-
 .../gcp/storage/GcsFileSystemRegistrar.java     |   5 +-
 .../gcp/storage/GcsPathValidator.java           | 105 +++
 .../gcp/storage/NoopPathValidator.java          |  53 ++
 .../extensions/gcp/storage/PathValidator.java   |  59 ++
 .../apache/beam/sdk/util/BackOffAdapter.java    |  43 +
 .../beam/sdk/util/GcsIOChannelFactory.java      | 111 ---
 .../sdk/util/GcsIOChannelFactoryRegistrar.java  |  38 -
 .../apache/beam/sdk/util/GcsPathValidator.java  |  98 ---
 .../java/org/apache/beam/sdk/util/GcsUtil.java  |  71 +-
 .../sdk/util/UploadIdResponseInterceptor.java   |  60 ++
 .../extensions/gcp/GcpCoreApiSurfaceTest.java   |   3 +-
 .../extensions/gcp/options/GcpOptionsTest.java  |   4 +-
 .../gcp/options/GoogleApiDebugOptionsTest.java  |   8 +-
 .../gcp/storage/GcsFileSystemTest.java          |  30 +-
 .../gcp/storage/GcsPathValidatorTest.java       | 107 +++
 .../gcp/storage/GcsResourceIdTest.java          |   9 +
 .../beam/sdk/util/FastNanoClockAndSleeper.java  |  47 ++
 .../sdk/util/FastNanoClockAndSleeperTest.java   |  47 ++
 .../util/GcsIOChannelFactoryRegistrarTest.java  |  44 -
 .../beam/sdk/util/GcsIOChannelFactoryTest.java  |  43 -
 .../beam/sdk/util/GcsPathValidatorTest.java     | 106 ---
 .../org/apache/beam/sdk/util/GcsUtilTest.java   |  80 +-
 .../util/UploadIdResponseInterceptorTest.java   |  98 +++
 sdks/java/extensions/jackson/pom.xml            |   2 +-
 sdks/java/extensions/join-library/README.md     |  42 -
 sdks/java/extensions/join-library/pom.xml       |   2 +-
 sdks/java/extensions/pom.xml                    |   2 +-
 sdks/java/extensions/protobuf/pom.xml           |   2 +-
 .../extensions/protobuf/ByteStringCoder.java    |  24 +-
 .../sdk/extensions/protobuf/ProtoCoder.java     |  85 +-
 .../ProtobufCoderProviderRegistrar.java         |  41 +
 .../protobuf/ProtobufCoderRegistrar.java        |  39 -
 .../protobuf/ByteStringCoderTest.java           |  10 +-
 .../sdk/extensions/protobuf/ProtoCoderTest.java |   7 +-
 sdks/java/extensions/sorter/README.md           |  42 -
 sdks/java/extensions/sorter/pom.xml             |   2 +-
 .../extensions/sorter/ExternalSorterTest.java   |   1 -
 sdks/java/harness/pom.xml                       |  12 +-
 .../org/apache/beam/fn/harness/FnHarness.java   |   8 +-
 .../harness/control/ProcessBundleHandler.java   |   4 +-
 .../BeamFnDataBufferingOutboundObserver.java    |   3 +-
 .../harness/data/BeamFnDataInboundObserver.java |   3 +-
 .../beam/runners/core/BeamFnDataReadRunner.java |  19 +-
 .../runners/core/BeamFnDataWriteRunner.java     |  20 +-
 .../control/ProcessBundleHandlerTest.java       |  55 +-
 ...BeamFnDataBufferingOutboundObserverTest.java |   3 +-
 .../data/BeamFnDataInboundObserverTest.java     |   3 +-
 .../runners/core/BeamFnDataReadRunnerTest.java  |  14 +-
 .../runners/core/BeamFnDataWriteRunnerTest.java |  14 +-
 sdks/java/io/common/pom.xml                     |   2 +-
 .../apache/beam/sdk/io/common/HashingFn.java    |   2 -
 sdks/java/io/elasticsearch/pom.xml              |   2 +-
 .../sdk/io/elasticsearch/ElasticsearchIO.java   |  12 +-
 sdks/java/io/google-cloud-platform/pom.xml      |  24 +-
 .../beam/sdk/io/gcp/bigquery/BatchLoads.java    | 253 ++++--
 .../BigQueryCoderProviderRegistrar.java         |  40 +
 .../io/gcp/bigquery/BigQueryCoderRegistrar.java |  39 -
 .../sdk/io/gcp/bigquery/BigQueryHelpers.java    | 141 ++--
 .../beam/sdk/io/gcp/bigquery/BigQueryIO.java    | 562 +++++++------
 .../io/gcp/bigquery/BigQueryQuerySource.java    |  44 +-
 .../io/gcp/bigquery/BigQueryServicesImpl.java   | 101 +--
 .../sdk/io/gcp/bigquery/BigQuerySourceBase.java |  46 +-
 .../gcp/bigquery/BigQueryTableRowIterator.java  |   7 +-
 .../io/gcp/bigquery/BigQueryTableSource.java    |  17 +-
 .../sdk/io/gcp/bigquery/CalculateSchemas.java   |  78 ++
 .../beam/sdk/io/gcp/bigquery/CreateTables.java  |  66 +-
 .../io/gcp/bigquery/DynamicDestinations.java    | 178 ++++
 .../bigquery/DynamicDestinationsHelpers.java    | 193 +++++
 .../beam/sdk/io/gcp/bigquery/PrepareWrite.java  |  36 +-
 .../sdk/io/gcp/bigquery/ShardedKeyCoder.java    |  22 +-
 .../sdk/io/gcp/bigquery/StreamingInserts.java   |  59 +-
 .../sdk/io/gcp/bigquery/StreamingWriteFn.java   |   4 +-
 .../io/gcp/bigquery/StreamingWriteTables.java   |  10 +-
 .../sdk/io/gcp/bigquery/TableDestination.java   |  14 +-
 .../io/gcp/bigquery/TableDestinationCoder.java  |  27 +-
 .../sdk/io/gcp/bigquery/TableRowInfoCoder.java  |  19 +-
 .../sdk/io/gcp/bigquery/TableRowJsonCoder.java  |  19 +-
 .../sdk/io/gcp/bigquery/TableRowWriter.java     |  59 +-
 .../sdk/io/gcp/bigquery/TagWithUniqueIds.java   |   2 +-
 .../io/gcp/bigquery/WriteBundlesToFiles.java    | 190 +++--
 .../bigquery/WriteGroupedRecordsToFiles.java    |  68 ++
 .../sdk/io/gcp/bigquery/WritePartition.java     |  86 +-
 .../beam/sdk/io/gcp/bigquery/WriteRename.java   |  10 +-
 .../beam/sdk/io/gcp/bigquery/WriteResult.java   |  23 +-
 .../beam/sdk/io/gcp/bigquery/WriteTables.java   | 160 ++--
 .../beam/sdk/io/gcp/bigtable/BigtableIO.java    | 245 +++---
 .../io/gcp/bigtable/BigtableServiceImpl.java    |   9 -
 .../beam/sdk/io/gcp/datastore/DatastoreV1.java  |  19 +-
 .../pubsub/PubsubCoderProviderRegistrar.java    |  37 +
 .../apache/beam/sdk/io/gcp/pubsub/PubsubIO.java |  78 +-
 .../beam/sdk/io/gcp/pubsub/PubsubMessage.java   |  61 ++
 .../pubsub/PubsubMessagePayloadOnlyCoder.java   |  32 +-
 .../PubsubMessageWithAttributesCoder.java       |  40 +-
 .../sdk/io/gcp/pubsub/PubsubUnboundedSink.java  |  40 +-
 .../io/gcp/pubsub/PubsubUnboundedSource.java    |  51 +-
 .../BigQueryCoderProviderRegistrarTest.java     |  40 +
 .../bigquery/BigQueryCoderRegistrarTest.java    |  40 -
 .../sdk/io/gcp/bigquery/BigQueryIOTest.java     | 561 ++++++++-----
 .../gcp/bigquery/BigQueryServicesImplTest.java  |  24 +-
 .../io/gcp/bigquery/FakeBigQueryServices.java   |   2 -
 .../sdk/io/gcp/bigquery/FakeJobService.java     |  55 +-
 .../sdk/io/gcp/bigquery/TableContainer.java     |   1 -
 .../beam/sdk/io/gcp/datastore/V1TestUtil.java   |   6 +-
 .../io/gcp/pubsub/PubsubUnboundedSinkTest.java  |  41 +-
 .../gcp/pubsub/PubsubUnboundedSourceTest.java   |   4 +-
 .../sdk/io/gcp/testing/BigqueryMatcher.java     |   4 +-
 .../sdk/io/gcp/testing/BigqueryMatcherTest.java |   7 +-
 sdks/java/io/hadoop-common/pom.xml              |   9 +-
 .../beam/sdk/io/hadoop/WritableCoder.java       |  78 +-
 .../beam/sdk/io/hadoop/WritableCoderTest.java   |  10 +
 sdks/java/io/hadoop-file-system/README.md       |  43 -
 sdks/java/io/hadoop-file-system/pom.xml         |  45 +-
 .../apache/beam/sdk/io/hdfs/HDFSFileSink.java   | 478 -----------
 .../apache/beam/sdk/io/hdfs/HDFSFileSource.java | 625 ---------------
 .../beam/sdk/io/hdfs/HadoopFileSystem.java      |  32 +-
 .../sdk/io/hdfs/HadoopFileSystemOptions.java    |  76 +-
 .../sdk/io/hdfs/HadoopFileSystemRegistrar.java  |   3 +
 .../beam/sdk/io/hdfs/HadoopResourceId.java      |  16 +-
 .../java/org/apache/beam/sdk/io/hdfs/Sink.java  | 195 -----
 .../org/apache/beam/sdk/io/hdfs/UGIHelper.java  |  38 -
 .../java/org/apache/beam/sdk/io/hdfs/Write.java | 585 --------------
 .../apache/beam/sdk/io/hdfs/package-info.java   |   3 +-
 .../beam/sdk/io/hdfs/HDFSFileSinkTest.java      | 172 ----
 .../beam/sdk/io/hdfs/HDFSFileSourceTest.java    | 231 ------
 .../sdk/io/hdfs/HadoopFileSystemModuleTest.java |   2 +-
 .../io/hdfs/HadoopFileSystemOptionsTest.java    | 125 +++
 .../beam/sdk/io/hdfs/HadoopFileSystemTest.java  |   5 +-
 .../beam/sdk/io/hdfs/HadoopResourceIdTest.java  |  71 ++
 sdks/java/io/hadoop/README.md                   | 167 ----
 sdks/java/io/hadoop/input-format/pom.xml        |   2 +-
 .../ConfigurableEmployeeInputFormat.java        |   1 -
 .../hadoop/inputformat/EmployeeInputFormat.java |   1 -
 .../ReuseObjectsEmployeeInputFormat.java        |   1 -
 .../hadoop/inputformat/TestEmployeeDataSet.java |   2 -
 sdks/java/io/hadoop/jdk1.8-tests/pom.xml        |  30 +-
 .../inputformat/HIFIOWithElasticTest.java       |   1 -
 .../HIFIOWithEmbeddedCassandraTest.java         |   2 -
 .../integration/tests/HIFIOCassandraIT.java     |   2 -
 .../integration/tests/HIFIOElasticIT.java       |   1 -
 sdks/java/io/hadoop/pom.xml                     |   2 +-
 sdks/java/io/hbase/pom.xml                      |  27 +-
 .../org/apache/beam/sdk/io/hbase/HBaseIO.java   |   7 +-
 .../beam/sdk/io/hbase/HBaseMutationCoder.java   |  10 +-
 .../beam/sdk/io/hbase/HBaseResultCoder.java     |   8 +-
 .../apache/beam/sdk/io/hbase/HBaseIOTest.java   |   4 +-
 sdks/java/io/jdbc/pom.xml                       |   2 +-
 .../org/apache/beam/sdk/io/jdbc/JdbcIO.java     |  13 +-
 sdks/java/io/jdbc/src/test/README.md            |  32 -
 .../org/apache/beam/sdk/io/jdbc/JdbcIOIT.java   |   1 -
 sdks/java/io/jms/pom.xml                        |   2 +-
 .../java/org/apache/beam/sdk/io/jms/JmsIO.java  |  17 +-
 .../org/apache/beam/sdk/io/jms/JmsRecord.java   |   1 -
 sdks/java/io/kafka/pom.xml                      |   2 +-
 .../beam/sdk/io/kafka/KafkaCheckpointMark.java  |   2 -
 .../org/apache/beam/sdk/io/kafka/KafkaIO.java   | 348 +++-----
 .../apache/beam/sdk/io/kafka/KafkaRecord.java   |   1 -
 .../beam/sdk/io/kafka/KafkaRecordCoder.java     |  19 +-
 .../CoderBasedKafkaDeserializer.java            |  71 --
 .../CoderBasedKafkaSerializer.java              |  73 --
 .../serialization/InstantDeserializer.java      |   1 -
 .../kafka/serialization/InstantSerializer.java  |   1 -
 .../apache/beam/sdk/io/kafka/KafkaIOTest.java   | 166 +---
 sdks/java/io/kinesis/pom.xml                    |   2 +-
 .../beam/sdk/io/kinesis/KinesisRecordCoder.java |  42 +-
 sdks/java/io/mongodb/pom.xml                    |   2 +-
 .../beam/sdk/io/mongodb/MongoDbGridFSIO.java    |   7 +-
 .../apache/beam/sdk/io/mongodb/MongoDbIO.java   |  57 +-
 .../sdk/io/mongodb/MongoDBGridFSIOTest.java     |   3 -
 .../beam/sdk/io/mongodb/MongoDbIOTest.java      |  20 +-
 sdks/java/io/mqtt/pom.xml                       |   2 +-
 .../org/apache/beam/sdk/io/mqtt/MqttIO.java     |   3 -
 sdks/java/io/pom.xml                            |   2 +-
 sdks/java/io/xml/pom.xml                        |   2 +-
 .../org/apache/beam/sdk/io/xml/JAXBCoder.java   |  63 +-
 .../java/org/apache/beam/sdk/io/xml/XmlIO.java  |  19 +-
 .../org/apache/beam/sdk/io/xml/XmlSink.java     |  24 +-
 .../apache/beam/sdk/io/xml/JAXBCoderTest.java   |  25 +-
 .../org/apache/beam/sdk/io/xml/XmlSinkTest.java |  52 +-
 sdks/java/java8tests/pom.xml                    |   2 +-
 .../beam/sdk/transforms/DistinctJava8Test.java  |   8 -
 .../beam/sdk/transforms/FilterJava8Test.java    |   2 +-
 .../beam/sdk/transforms/PartitionJava8Test.java |   2 +-
 .../beam/sdk/transforms/WithKeysJava8Test.java  |   9 -
 sdks/java/javadoc/README.md                     |  23 -
 sdks/java/javadoc/ant.xml                       |   5 +-
 sdks/java/javadoc/pom.xml                       |   2 +-
 .../examples-java8/generate-sources.sh          |  12 +-
 .../maven-archetypes/examples-java8/pom.xml     | 108 +--
 .../main/resources/archetype-resources/pom.xml  |  23 +-
 .../examples/generate-sources.sh                |   8 +-
 sdks/java/maven-archetypes/examples/pom.xml     |  67 +-
 .../main/resources/archetype-resources/pom.xml  |  16 +-
 sdks/java/maven-archetypes/pom.xml              |  22 +-
 sdks/java/maven-archetypes/starter/pom.xml      |  19 +-
 .../resources/projects/basic/reference/pom.xml  |   2 +-
 sdks/java/pom.xml                               |   2 +-
 sdks/pom.xml                                    |  20 +-
 sdks/python/README.md                           | 298 -------
 sdks/python/apache_beam/__init__.py             |   2 +-
 sdks/python/apache_beam/coders/coder_impl.py    |  59 +-
 sdks/python/apache_beam/coders/coders.py        |  32 +-
 sdks/python/apache_beam/coders/coders_test.py   |  11 +-
 .../apache_beam/coders/coders_test_common.py    |   2 +-
 sdks/python/apache_beam/coders/observable.py    |   9 +-
 sdks/python/apache_beam/coders/slow_stream.py   |  21 +-
 .../apache_beam/coders/standard_coders_test.py  |   4 +-
 sdks/python/apache_beam/coders/stream.pyx       |   5 +
 sdks/python/apache_beam/coders/typecoders.py    |   7 +-
 .../examples/complete/autocomplete.py           |   4 +-
 .../examples/complete/autocomplete_test.py      |   6 +-
 .../examples/complete/estimate_pi.py            |   4 +-
 .../examples/complete/estimate_pi_test.py       |   6 +-
 .../examples/complete/game/README.md            |  69 --
 .../examples/complete/game/hourly_team_score.py |   6 +-
 .../complete/game/hourly_team_score_test.py     |   6 +-
 .../examples/complete/game/user_score.py        |   4 +-
 .../examples/complete/game/user_score_test.py   |   6 +-
 .../apache_beam/examples/complete/tfidf.py      |   4 +-
 .../apache_beam/examples/complete/tfidf_test.py |   6 +-
 .../examples/complete/top_wikipedia_sessions.py |   4 +-
 .../complete/top_wikipedia_sessions_test.py     |   6 +-
 .../examples/cookbook/bigquery_side_input.py    |   4 +-
 .../cookbook/bigquery_side_input_test.py        |   6 +-
 .../cookbook/bigquery_tornadoes_it_test.py      |   4 +-
 .../cookbook/bigquery_tornadoes_test.py         |   8 +-
 .../apache_beam/examples/cookbook/coders.py     |   4 +-
 .../examples/cookbook/coders_test.py            |   6 +-
 .../examples/cookbook/combiners_test.py         |   8 +-
 .../examples/cookbook/custom_ptransform.py      |   2 +-
 .../examples/cookbook/custom_ptransform_test.py |   6 +-
 .../examples/cookbook/datastore_wordcount.py    |  19 +-
 .../examples/cookbook/filters_test.py           |  14 +-
 .../examples/cookbook/group_with_coder.py       |   4 +-
 .../examples/cookbook/mergecontacts.py          |  18 +-
 .../examples/cookbook/multiple_output_pardo.py  |   4 +-
 .../apache_beam/examples/snippets/snippets.py   |  53 +-
 .../examples/snippets/snippets_test.py          |  49 +-
 sdks/python/apache_beam/examples/wordcount.py   |  42 +-
 .../apache_beam/examples/wordcount_debugging.py |  10 +-
 .../apache_beam/examples/wordcount_it_test.py   |   6 +-
 .../apache_beam/examples/wordcount_minimal.py   |   4 +-
 sdks/python/apache_beam/internal/__init__.py    |   2 +
 .../python/apache_beam/internal/gcp/__init__.py |   2 +
 sdks/python/apache_beam/internal/gcp/auth.py    |  50 +-
 .../apache_beam/internal/gcp/json_value.py      |  14 +-
 .../apache_beam/internal/gcp/json_value_test.py |   4 +-
 sdks/python/apache_beam/internal/pickler.py     |  28 +-
 sdks/python/apache_beam/internal/util.py        |  21 +-
 sdks/python/apache_beam/io/__init__.py          |   2 +-
 sdks/python/apache_beam/io/avroio.py            |  17 +-
 sdks/python/apache_beam/io/avroio_test.py       |   6 +-
 sdks/python/apache_beam/io/concat_source.py     |  12 +-
 .../python/apache_beam/io/concat_source_test.py |   6 +-
 sdks/python/apache_beam/io/filebasedsink.py     | 299 +++++++
 .../python/apache_beam/io/filebasedsink_test.py | 303 +++++++
 sdks/python/apache_beam/io/filebasedsource.py   |   8 +-
 .../apache_beam/io/filebasedsource_test.py      |  10 +-
 sdks/python/apache_beam/io/fileio.py            | 297 -------
 sdks/python/apache_beam/io/fileio_test.py       | 304 -------
 sdks/python/apache_beam/io/filesystem.py        |  19 +
 sdks/python/apache_beam/io/filesystems.py       |  38 +-
 sdks/python/apache_beam/io/filesystems_test.py  |  15 +-
 sdks/python/apache_beam/io/filesystems_util.py  |  36 -
 sdks/python/apache_beam/io/gcp/bigquery.py      |  10 +-
 sdks/python/apache_beam/io/gcp/bigquery_test.py |   2 +-
 .../io/gcp/datastore/v1/fake_datastore.py       |  14 +-
 .../apache_beam/io/gcp/datastore/v1/helper.py   |  35 +-
 .../io/gcp/datastore/v1/helper_test.py          |  26 +-
 .../io/gcp/datastore/v1/query_splitter_test.py  |  37 +-
 sdks/python/apache_beam/io/gcp/gcsfilesystem.py |   8 +
 .../apache_beam/io/gcp/gcsfilesystem_test.py    |   5 +
 sdks/python/apache_beam/io/gcp/gcsio.py         |   9 +-
 sdks/python/apache_beam/io/gcp/pubsub.py        |   2 +
 .../io/gcp/tests/bigquery_matcher.py            |   5 +-
 .../io/gcp/tests/bigquery_matcher_test.py       |   2 +-
 sdks/python/apache_beam/io/iobase.py            |  23 +-
 sdks/python/apache_beam/io/localfilesystem.py   |   7 +
 .../apache_beam/io/localfilesystem_test.py      |   4 +
 sdks/python/apache_beam/io/range_trackers.py    |   7 +-
 sdks/python/apache_beam/io/source_test_utils.py |   8 +
 sdks/python/apache_beam/io/sources_test.py      |   6 +-
 sdks/python/apache_beam/io/textio.py            |  19 +-
 sdks/python/apache_beam/io/textio_test.py       |   7 +-
 sdks/python/apache_beam/io/tfrecordio.py        |  19 +-
 sdks/python/apache_beam/io/tfrecordio_test.py   |  26 +-
 sdks/python/apache_beam/metrics/__init__.py     |   1 +
 sdks/python/apache_beam/metrics/cells.py        |  50 +-
 sdks/python/apache_beam/metrics/execution.py    |   7 +-
 sdks/python/apache_beam/metrics/metric.py       |   4 +
 sdks/python/apache_beam/metrics/metricbase.py   |   2 +
 sdks/python/apache_beam/options/__init__.py     |  16 +
 .../apache_beam/options/pipeline_options.py     | 650 +++++++++++++++
 .../options/pipeline_options_test.py            | 240 ++++++
 .../options/pipeline_options_validator.py       | 201 +++++
 .../options/pipeline_options_validator_test.py  | 343 ++++++++
 .../apache_beam/options/value_provider.py       | 111 +++
 .../apache_beam/options/value_provider_test.py  | 145 ++++
 sdks/python/apache_beam/pipeline.py             |  33 +-
 sdks/python/apache_beam/pipeline_test.py        |   8 +-
 sdks/python/apache_beam/pvalue.py               |  18 +-
 sdks/python/apache_beam/pvalue_test.py          |   2 +-
 sdks/python/apache_beam/runners/api/__init__.py |   4 +-
 .../apache_beam/runners/api/beam_fn_api_pb2.py  |   2 +
 .../runners/api/beam_fn_api_pb2_grpc.py         |   2 +
 .../runners/api/beam_runner_api_pb2.py          |   2 +
 sdks/python/apache_beam/runners/common.pxd      |   6 +-
 sdks/python/apache_beam/runners/common.py       |  99 ++-
 sdks/python/apache_beam/runners/common_test.py  |  58 ++
 .../apache_beam/runners/dataflow/__init__.py    |   9 +
 .../runners/dataflow/dataflow_metrics.py        | 114 ++-
 .../runners/dataflow/dataflow_metrics_test.py   | 197 +++--
 .../runners/dataflow/dataflow_runner.py         |  11 +-
 .../runners/dataflow/dataflow_runner_test.py    |   9 +-
 .../runners/dataflow/internal/apiclient.py      |  17 +-
 .../runners/dataflow/internal/apiclient_test.py |  23 +-
 .../runners/dataflow/internal/dependency.py     |  22 +-
 .../dataflow/internal/dependency_test.py        |   6 +-
 .../runners/dataflow/internal/names.py          |   8 +-
 .../runners/dataflow/native_io/iobase.py        |   2 +
 .../runners/dataflow/template_runner_test.py    |   2 +-
 .../runners/dataflow/test_dataflow_runner.py    |   5 +-
 .../apache_beam/runners/direct/__init__.py      |   6 +-
 .../runners/direct/bundle_factory.py            |  16 +-
 sdks/python/apache_beam/runners/direct/clock.py |   5 +-
 .../consumer_tracking_pipeline_visitor.py       |   4 +-
 .../apache_beam/runners/direct/direct_runner.py |   7 +-
 .../apache_beam/runners/direct/executor.py      |  67 +-
 .../runners/direct/transform_evaluator.py       |  20 +-
 .../runners/direct/transform_result.py          |   4 +-
 .../runners/direct/watermark_manager.py         |  14 +-
 .../apache_beam/runners/pipeline_context.py     |  10 +-
 .../apache_beam/runners/portability/__init__.py |   2 +
 .../runners/portability/fn_api_runner.py        |   2 +
 .../portability/maptask_executor_runner.py      |   9 +-
 .../portability/maptask_executor_runner_test.py |  35 +-
 sdks/python/apache_beam/runners/runner.py       |  11 +-
 sdks/python/apache_beam/runners/runner_test.py  |   6 +-
 .../apache_beam/runners/worker/__init__.py      |   2 +
 .../apache_beam/runners/worker/data_plane.py    |   2 +
 .../apache_beam/runners/worker/log_handler.py   |   2 +
 .../python/apache_beam/runners/worker/logger.py |   2 +
 .../apache_beam/runners/worker/opcounters.py    |   2 +
 .../runners/worker/operation_specs.py           |   2 +
 .../apache_beam/runners/worker/sdk_worker.py    |   4 +
 .../runners/worker/sdk_worker_main.py           |   2 +
 .../apache_beam/runners/worker/sideinputs.py    |   2 +
 .../runners/worker/statesampler_fake.py         |   2 +
 sdks/python/apache_beam/test_pipeline.py        | 163 ----
 sdks/python/apache_beam/test_pipeline_test.py   | 112 ---
 sdks/python/apache_beam/testing/__init__.py     |  16 +
 .../testing/data/standard_coders.yaml           | 196 +++++
 .../testing/data/trigger_transcripts.yaml       | 224 ++++++
 .../apache_beam/testing/pipeline_verifiers.py   | 154 ++++
 .../testing/pipeline_verifiers_test.py          | 148 ++++
 .../python/apache_beam/testing/test_pipeline.py | 168 ++++
 .../apache_beam/testing/test_pipeline_test.py   | 112 +++
 sdks/python/apache_beam/testing/test_stream.py  | 175 ++++
 .../apache_beam/testing/test_stream_test.py     |  83 ++
 sdks/python/apache_beam/testing/test_utils.py   |  73 ++
 sdks/python/apache_beam/testing/util.py         | 107 +++
 sdks/python/apache_beam/testing/util_test.py    |  50 ++
 sdks/python/apache_beam/tests/__init__.py       |  16 -
 sdks/python/apache_beam/tests/data/README.md    |  20 -
 .../apache_beam/tests/data/privatekey.p12       | Bin 2452 -> 0 bytes
 .../apache_beam/tests/data/standard_coders.yaml | 196 -----
 .../apache_beam/tests/pipeline_verifiers.py     | 146 ----
 .../tests/pipeline_verifiers_test.py            | 148 ----
 sdks/python/apache_beam/tests/test_utils.py     |  69 --
 sdks/python/apache_beam/transforms/__init__.py  |   2 +-
 .../apache_beam/transforms/combiners_test.py    |   4 +-
 sdks/python/apache_beam/transforms/core.py      |  96 ++-
 .../apache_beam/transforms/create_test.py       |   5 +-
 .../apache_beam/transforms/cy_combiners.py      |   5 +-
 sdks/python/apache_beam/transforms/display.py   |   2 +-
 .../apache_beam/transforms/display_test.py      |   2 +-
 .../python/apache_beam/transforms/ptransform.py |  45 +-
 .../apache_beam/transforms/ptransform_test.py   |  22 +-
 .../python/apache_beam/transforms/sideinputs.py |   2 +
 .../apache_beam/transforms/sideinputs_test.py   |   4 +-
 sdks/python/apache_beam/transforms/timeutil.py  |  33 +-
 sdks/python/apache_beam/transforms/trigger.py   | 118 +--
 .../apache_beam/transforms/trigger_test.py      |  50 +-
 .../transforms/trigger_transcripts.yaml         | 224 ------
 sdks/python/apache_beam/transforms/util.py      |  79 --
 sdks/python/apache_beam/transforms/util_test.py |  50 --
 sdks/python/apache_beam/transforms/window.py    |  51 +-
 .../apache_beam/transforms/window_test.py       |  12 +-
 .../transforms/write_ptransform_test.py         |   4 +-
 sdks/python/apache_beam/typehints/decorators.py |  17 +-
 sdks/python/apache_beam/typehints/opcodes.py    |   2 +
 .../apache_beam/typehints/trivial_inference.py  |   2 +
 sdks/python/apache_beam/typehints/typecheck.py  |  17 +-
 .../typehints/typed_pipeline_test.py            |   6 +-
 sdks/python/apache_beam/typehints/typehints.py  |  18 +
 .../apache_beam/typehints/typehints_test.py     |  19 +-
 sdks/python/apache_beam/utils/__init__.py       |   5 +-
 sdks/python/apache_beam/utils/annotations.py    |   4 +-
 sdks/python/apache_beam/utils/counters.py       |   5 +-
 .../apache_beam/utils/pipeline_options.py       | 627 ---------------
 .../apache_beam/utils/pipeline_options_test.py  | 240 ------
 .../utils/pipeline_options_validator.py         | 199 -----
 .../utils/pipeline_options_validator_test.py    | 343 --------
 sdks/python/apache_beam/utils/processes.py      |   6 +-
 sdks/python/apache_beam/utils/profiler.py       |   5 +-
 sdks/python/apache_beam/utils/proto_utils.py    |   2 +
 sdks/python/apache_beam/utils/retry.py          |   2 +
 sdks/python/apache_beam/utils/test_stream.py    | 163 ----
 .../apache_beam/utils/test_stream_test.py       |  83 --
 sdks/python/apache_beam/utils/timestamp.py      |   5 +-
 sdks/python/apache_beam/utils/urns.py           |   2 +
 sdks/python/apache_beam/utils/value_provider.py | 103 ---
 .../apache_beam/utils/value_provider_test.py    | 145 ----
 sdks/python/apache_beam/utils/windowed_value.py |   4 +-
 .../apache_beam/utils/windowed_value_test.py    |   2 +-
 sdks/python/apache_beam/version.py              |   2 +-
 sdks/python/generate_pydoc.sh                   |  15 +-
 sdks/python/pom.xml                             |   2 +-
 sdks/python/setup.py                            |   4 +-
 sdks/python/tox.ini                             |   5 +
 1287 files changed, 28743 insertions(+), 31376 deletions(-)
----------------------------------------------------------------------



Mime
View raw message