Reply-To: dev@beam.apache.org Delivered-To: mailing list commits@beam.apache.org Received: (qmail 58494 invoked by uid 99); 14 Mar 2017 22:33:42 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 14 Mar 2017 22:33:42 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id 78F8EF4B77; Tue, 14 Mar 2017 22:33:41 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: davor@apache.org To: commits@beam.apache.org Date: Tue, 14 Mar 2017 22:34:20 -0000 Message-Id: <1c82f8b222464cd7ae9ce72f8df0567a@git.apache.org> In-Reply-To: <76d8237921994d299dfe8de2dcc6f748@git.apache.org> References: <76d8237921994d299dfe8de2dcc6f748@git.apache.org> X-Mailer: ASF-Git Admin Mailer Subject: [41/52] [abbrv] [partial] beam-site git commit: Regenerate website archived-at: Tue, 14 Mar 2017 22:33:50 -0000 http://git-wip-us.apache.org/repos/asf/beam-site/blob/dc04d667/content/documentation/sdks/javadoc/0.6.0/org/apache/beam/runners/apex/package-frame.html ---------------------------------------------------------------------- diff --git a/content/documentation/sdks/javadoc/0.6.0/org/apache/beam/runners/apex/package-frame.html b/content/documentation/sdks/javadoc/0.6.0/org/apache/beam/runners/apex/package-frame.html new file mode 100644 index 0000000..b9d24a3 --- /dev/null +++ b/content/documentation/sdks/javadoc/0.6.0/org/apache/beam/runners/apex/package-frame.html @@ -0,0 +1,33 @@ + + + + + +org.apache.beam.runners.apex + + + + + +

org.apache.beam.runners.apex

Interfaces

ApexPipelineOptions

Classes

+ + http://git-wip-us.apache.org/repos/asf/beam-site/blob/dc04d667/content/documentation/sdks/javadoc/0.6.0/org/apache/beam/runners/apex/package-summary.html ---------------------------------------------------------------------- diff --git a/content/documentation/sdks/javadoc/0.6.0/org/apache/beam/runners/apex/package-summary.html b/content/documentation/sdks/javadoc/0.6.0/org/apache/beam/runners/apex/package-summary.html new file mode 100644 index 0000000..837d83b --- /dev/null +++ b/content/documentation/sdks/javadoc/0.6.0/org/apache/beam/runners/apex/package-summary.html @@ -0,0 +1,225 @@ + + + + + +org.apache.beam.runners.apex + + + + + + + + +

+ + +

Skip navigation links

+ + + +

Prev Package
Next Package

Frames
No Frames

All Classes

+ +

+ + +

+ + + + + + + + + + + + +
Interface Summary
Interface Description
ApexPipelineOptions +
Options that configure the Apex pipeline.
+
+

Interface Summary
Interface	Description
ApexPipelineOptions	+ Options that configure the Apex pipeline. +

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

Class Summary
Class	Description
ApexRunner	+ A `PipelineRunner` that translates the + pipeline to an Apex DAG and executes it on an Apex cluster. +
ApexRunner.CreateApexPCollectionView<ElemT,ViewT>	+ Creates a primitive `PCollectionView`. +
ApexRunnerRegistrar	+ Contains the `PipelineRunnerRegistrar` and `PipelineOptionsRegistrar` for the + `ApexRunner`. +
ApexRunnerRegistrar.Options	+ Registers the `ApexPipelineOptions`. +
ApexRunnerRegistrar.Runner	+ Registers the `ApexRunner`. +
ApexRunnerResult	+ Result of executing a `Pipeline` with Apex in embedded mode. +
ApexYarnLauncher	+ Proxy to launch the YARN application through the hadoop script to run in the + pre-configured environment (class path, configuration, native libraries etc.). +
ApexYarnLauncher.LaunchParams	+ Launch parameters that will be serialized and passed to the child process. +
ApexYarnLauncher.ProcessWatcher	+ Starts a command and waits for it to complete. +
TestApexRunner	+ Apex `PipelineRunner` for testing. +

+ + + +

Package org.apache.beam.runners.apex Description

Implementation of the Beam runner for Apache Apex.

+ +

+ + +

Skip navigation links

+ + + +

Prev Package
Next Package

Frames
No Frames

All Classes

+ +

+ + +

+ + + http://git-wip-us.apache.org/repos/asf/beam-site/blob/dc04d667/content/documentation/sdks/javadoc/0.6.0/org/apache/beam/runners/apex/package-tree.html ---------------------------------------------------------------------- diff --git a/content/documentation/sdks/javadoc/0.6.0/org/apache/beam/runners/apex/package-tree.html b/content/documentation/sdks/javadoc/0.6.0/org/apache/beam/runners/apex/package-tree.html new file mode 100644 index 0000000..bb05c4f --- /dev/null +++ b/content/documentation/sdks/javadoc/0.6.0/org/apache/beam/runners/apex/package-tree.html @@ -0,0 +1,169 @@ + + + + + +org.apache.beam.runners.apex Class Hierarchy + + + + + + + + +

+ + +

Skip navigation links

+ + + +

Frames
No Frames

All Classes

+ +

+ + +

Class Hierarchy

java.lang.Object +
- org.apache.beam.runners.apex.ApexRunnerRegistrar
- org.apache.beam.runners.apex.ApexRunnerRegistrar.Options (implements org.apache.beam.sdk.options.PipelineOptionsRegistrar)
- org.apache.beam.runners.apex.ApexRunnerRegistrar.Runner (implements org.apache.beam.sdk.runners.PipelineRunnerRegistrar)
- org.apache.beam.runners.apex.ApexRunnerResult (implements org.apache.beam.sdk.PipelineResult)
- org.apache.beam.runners.apex.ApexYarnLauncher
- org.apache.beam.runners.apex.ApexYarnLauncher.LaunchParams (implements java.io.Serializable)
- org.apache.beam.runners.apex.ApexYarnLauncher.ProcessWatcher (implements java.lang.Runnable)
- org.apache.beam.sdk.runners.PipelineRunner<ResultT> +
  - org.apache.beam.runners.apex.ApexRunner
  - org.apache.beam.runners.apex.TestApexRunner
  +
- org.apache.beam.sdk.transforms.PTransform<InputT,OutputT> (implements org.apache.beam.sdk.transforms.display.HasDisplayData, java.io.Serializable) +
  - org.apache.beam.runners.apex.ApexRunner.CreateApexPCollectionView<ElemT,ViewT>
  +
+

Interface Hierarchy

org.apache.beam.sdk.transforms.display.HasDisplayData +
- org.apache.beam.sdk.options.PipelineOptions +
  - org.apache.beam.runners.apex.ApexPipelineOptions (also extends java.io.Serializable)
  +
+
java.io.Serializable +
- org.apache.beam.runners.apex.ApexPipelineOptions (also extends org.apache.beam.sdk.options.PipelineOptions)
+

+ +

+ + +

Skip navigation links

+ + + +

Frames
No Frames

All Classes

+ +

+ + +

+ + + http://git-wip-us.apache.org/repos/asf/beam-site/blob/dc04d667/content/documentation/sdks/javadoc/0.6.0/org/apache/beam/runners/dataflow/BatchStatefulParDoOverrides.BatchStatefulDoFn.html ---------------------------------------------------------------------- diff --git a/content/documentation/sdks/javadoc/0.6.0/org/apache/beam/runners/dataflow/BatchStatefulParDoOverrides.BatchStatefulDoFn.html b/content/documentation/sdks/javadoc/0.6.0/org/apache/beam/runners/dataflow/BatchStatefulParDoOverrides.BatchStatefulDoFn.html new file mode 100644 index 0000000..7d37036 --- /dev/null +++ b/content/documentation/sdks/javadoc/0.6.0/org/apache/beam/runners/dataflow/BatchStatefulParDoOverrides.BatchStatefulDoFn.html @@ -0,0 +1,338 @@ + + + + + +BatchStatefulParDoOverrides.BatchStatefulDoFn + + + + + + + + +

+ + +

Skip navigation links

+ + + +

Prev Class
Next Class

Frames
No Frames

All Classes

+ +

Summary:
Nested |
Field |
Constr |
Method

Detail:
Field |
Constr |
Method

+ + +

+ + + +

java.lang.Object
+
- org.apache.beam.sdk.transforms.DoFn<KV<K,java.lang.Iterable<KV<Instant,WindowedValue<KV<K,V>>>>>,OutputT>
- +
  - org.apache.beam.runners.dataflow.BatchStatefulParDoOverrides.BatchStatefulDoFn<K,V,OutputT>
  +
+

+
+
All Implemented Interfaces:
+
java.io.Serializable, HasDisplayData
+
+
+
Enclosing class:
+
BatchStatefulParDoOverrides
+
+
+
+
```
public static class BatchStatefulParDoOverrides.BatchStatefulDoFn<K,V,OutputT>
+extends DoFn<KV<K,java.lang.Iterable<KV<Instant,WindowedValue<KV<K,V>>>>>,OutputT>
```
+
A key-preserving DoFn that explodes an iterable that has been grouped by key and + window.
+
+
See Also:
+
Serialized Form
+
+

+ +

+ + +
Nested Class Summary
+
- + + +
  Nested classes/interfaces inherited from class org.apache.beam.sdk.transforms.DoFn
  +DoFn.BoundedPerElement, DoFn.Context, DoFn.FinishBundle, DoFn.GetInitialRestriction, DoFn.GetRestrictionCoder, DoFn.NewTracker, DoFn.OnTimer, DoFn.OnTimerContext, DoFn.OutputReceiver<T>, DoFn.ProcessContext, DoFn.ProcessContinuation, DoFn.ProcessElement, DoFn.Setup, DoFn.SplitRestriction, DoFn.StartBundle, DoFn.StateId, DoFn.Teardown, DoFn.TimerId, DoFn.UnboundedPerElement
+

+ +

+ + +
Field Summary
+
- + + +
  Fields inherited from class org.apache.beam.sdk.transforms.DoFn
  +aggregators
+

+ +

+ + +

Method Summary

+ + + + + + + + + + + + + + + + + + +

All Methods Instance Methods Concrete Methods
Modifier and Type	Method and Description
`TypeDescriptor<OutputT>`	`getOutputTypeDescriptor()` + Returns a `TypeDescriptor` capturing what is known statically + about the output type of this `DoFn` instance's + most-derived class. +
`DoFn<KV<K,V>,OutputT>`	`getUnderlyingDoFn()`
`void`	`processElement(DoFn.ProcessContext c, + BoundedWindow window)`

+ + +
Methods inherited from class org.apache.beam.sdk.transforms.DoFn
+createAggregator, createAggregator, getAllowedTimestampSkew, getInputTypeDescriptor, populateDisplayData, prepareForProcessing

+ + +
Methods inherited from class java.lang.Object
+clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

+ +
- + + +
  Method Detail
  + + + +
  - +
    getUnderlyingDoFn
    +
```
public DoFn<KV<K,V>,OutputT> getUnderlyingDoFn()
```
    +
  + + + +
  - +
    processElement
    +
```
@DoFn.ProcessElement
+public void processElement(DoFn.ProcessContext c,
+                                                BoundedWindow window)
```
    +
  + + + +
  - +
    getOutputTypeDescriptor
    +
```
public TypeDescriptor<OutputT> getOutputTypeDescriptor()
```
    +
    Description copied from class: DoFn
    +
    Returns a TypeDescriptor capturing what is known statically + about the output type of this DoFn instance's + most-derived class. + +
    In the normal case of a concrete DoFn subclass with + no generic type parameters of its own (including anonymous inner + classes), this will be a complete non-generic type, which is good + for choosing a default output Coder<O> for the output + PCollection<O>.
    +
    +
    Overrides:
    +
    getOutputTypeDescriptor in class DoFn<KV<K,java.lang.Iterable<KV<Instant,WindowedValue <KV<K,V>>>>>,OutputT>
    +
    +
  +
+

+ + +

Skip navigation links

+ + + +

Prev Class
Next Class

Frames
No Frames

All Classes

+ +

Summary:
Nested |
Field |
Constr |
Method

Detail:
Field |
Constr |
Method

+ + +

+ + + http://git-wip-us.apache.org/repos/asf/beam-site/blob/dc04d667/content/documentation/sdks/javadoc/0.6.0/org/apache/beam/runners/dataflow/BatchStatefulParDoOverrides.html ---------------------------------------------------------------------- diff --git a/content/documentation/sdks/javadoc/0.6.0/org/apache/beam/runners/dataflow/BatchStatefulParDoOverrides.html b/content/documentation/sdks/javadoc/0.6.0/org/apache/beam/runners/dataflow/BatchStatefulParDoOverrides.html new file mode 100644 index 0000000..8fce805 --- /dev/null +++ b/content/documentation/sdks/javadoc/0.6.0/org/apache/beam/runners/dataflow/BatchStatefulParDoOverrides.html @@ -0,0 +1,320 @@ + + + + + +BatchStatefulParDoOverrides + + + + + + + + +

+ + +

Skip navigation links

+ + + +

Prev Class
Next Class

Frames
No Frames

All Classes

+ +

Summary:
Nested |
Field |
Constr |
Method

Detail:
Field |
Constr |
Method

+ + +

+ + + +

java.lang.Object
+
- org.apache.beam.runners.dataflow.BatchStatefulParDoOverrides
+

+
+
+
```
public class BatchStatefulParDoOverrides
+extends java.lang.Object
```
+
PTransformOverrideFactories that expands to correctly implement + stateful ParDo using window-unaware BatchViewOverrides.GroupByKeyAndSortValuesOnly to linearize + processing per key. + +
This implementation relies on implementation details of the Dataflow runner, specifically + standard fusion behavior of ParDo tranforms following a GroupByKey.
+

+ +

+ + +

Nested Class Summary

+ + + + + + + + + + +

Nested Classes
Modifier and Type	Class and Description
`static class`	`BatchStatefulParDoOverrides.BatchStatefulDoFn<K,V,OutputT>` + A key-preserving `DoFn` that explodes an iterable that has been grouped by key and + window. +

+ +

+ + +
Constructor Summary
+ + + + + + + + +
Constructors
Constructor and Description
BatchStatefulParDoOverrides()
+

Constructors
Constructor and Description
`BatchStatefulParDoOverrides()`

+ +

+ + +

Method Summary

+ + + + + + + + + + + + + + +

All Methods Static Methods Concrete Methods
Modifier and Type	Method and Description
`static <K,InputT,OutputT> PTransformOverrideFactory<PCollection<KV<K,InputT>>,PCollectionTuple,ParDo.BoundMulti<KV<K,InputT>,OutputT>>`	`multiOutputOverrideFactory()` + Returns a `PTransformOverrideFactory` that replaces a multi-output + `ParDo` with a composite transform specialized for the `DataflowRunner`. +
`static <K,InputT,OutputT> PTransformOverrideFactory<PCollection<KV<K,InputT>>,PCollection<OutputT>,ParDo.Bound<KV<K,InputT>,OutputT>>`	`singleOutputOverrideFactory()` + Returns a `PTransformOverrideFactory` that replaces a single-output + `ParDo` with a composite transform specialized for the `DataflowRunner`. +

+ + +
Methods inherited from class java.lang.Object
+clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

+ +
- + + +
  Constructor Detail
  + + + +
  - +
    BatchStatefulParDoOverrides
    +
```
public BatchStatefulParDoOverrides()
```
    +
  +
+ +
- + + +
  Method Detail
  + + + +
  - +
    singleOutputOverrideFactory
    +
```
public static <K,InputT,OutputT> PTransformOverrideFactory<PCollection<KV<K,InputT>>,PCollection<OutputT>,ParDo.Bound<KV<K,InputT>,OutputT>> singleOutputOverrideFactory()
```
    +
    Returns a PTransformOverrideFactory that replaces a single-output + ParDo with a composite transform specialized for the DataflowRunner.
    +
  + + + +
  - +
    multiOutputOverrideFactory
    +
```
public static <K,InputT,OutputT> PTransformOverrideFactory<PCollection<KV<K,InputT>>,PCollectionTuple,ParDo.BoundMulti<KV<K,InputT>,OutputT>> multiOutputOverrideFactory()
```
    +
    Returns a PTransformOverrideFactory that replaces a multi-output + ParDo with a composite transform specialized for the DataflowRunner.
    +
  +
+

+ + +

Skip navigation links

+ + + +

Prev Class
Next Class

Frames
No Frames

All Classes

+ +

Summary:
Nested |
Field |
Constr |
Method

Detail:
Field |
Constr |
Method

+ + +

+ + + http://git-wip-us.apache.org/repos/asf/beam-site/blob/dc04d667/content/documentation/sdks/javadoc/0.6.0/org/apache/beam/runners/dataflow/DataflowClient.html ---------------------------------------------------------------------- diff --git a/content/documentation/sdks/javadoc/0.6.0/org/apache/beam/runners/dataflow/DataflowClient.html b/content/documentation/sdks/javadoc/0.6.0/org/apache/beam/runners/dataflow/DataflowClient.html new file mode 100644 index 0000000..4a7f123 --- /dev/null +++ b/content/documentation/sdks/javadoc/0.6.0/org/apache/beam/runners/dataflow/DataflowClient.html @@ -0,0 +1,426 @@ + + + + + +DataflowClient + + + + + + + + +

+ + +

Skip navigation links

+ + + +

Prev Class
Next Class

Frames
No Frames

All Classes

+ +

Summary:
Nested |
Field |
Constr |
Method

Detail:
Field |
Constr |
Method

+ + +

+ + + +

java.lang.Object
+
- org.apache.beam.runners.dataflow.DataflowClient
+

+
+
+
```
public class DataflowClient
+extends java.lang.Object
```
+
Wrapper around the generated Dataflow client to provide common functionality.
+

+ +

+ + +

Method Summary

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

All Methods Static Methods Instance Methods Concrete Methods
Modifier and Type	Method and Description
`static DataflowClient`	`create(DataflowPipelineOptions options)`
`com.google.api.services.dataflow.model.Job`	`createJob(com.google.api.services.dataflow.model.Job job)` + Creates the Dataflow `Job`. +
`com.google.api.services.dataflow.model.Job`	`getJob(java.lang.String jobId)` + Gets the Dataflow `Job` with the given `jobId`. +
`com.google.api.services.dataflow.model.JobMetrics`	`getJobMetrics(java.lang.String jobId)` + Gets the `JobMetrics` with the given `jobId`. +
`com.google.api.services.dataflow.model.LeaseWorkItemResponse`	`leaseWorkItem(java.lang.String jobId, + com.google.api.services.dataflow.model.LeaseWorkItemRequest request)` + Leases the work item for `jobId`. +
`com.google.api.services.dataflow.model.ListJobMessagesResponse`	`listJobMessages(java.lang.String jobId, + java.lang.String pageToken)` + Lists job messages with the given `jobId`. +
`com.google.api.services.dataflow.model.ListJobsResponse`	`listJobs(java.lang.String pageToken)` + Lists Dataflow `Jobs` in the project associated with + the `DataflowPipelineOptions`. +
`com.google.api.services.dataflow.model.ReportWorkItemStatusResponse`	`reportWorkItemStatus(java.lang.String jobId, + com.google.api.services.dataflow.model.ReportWorkItemStatusRequest request)` + Reports the status of the work item for `jobId`. +
`com.google.api.services.dataflow.model.Job`	`updateJob(java.lang.String jobId, + com.google.api.services.dataflow.model.Job content)` + Updates the Dataflow `Job` with the given `jobId`. +

+ + +
Methods inherited from class java.lang.Object
+clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

+ +

+ + +

Method Detail

+ + + +

create

public static DataflowClient create(DataflowPipelineOptions options)

+ + + +

createJob

public com.google.api.services.dataflow.model.Job createJob(@Nonnull
+                                                            com.google.api.services.dataflow.model.Job job)
+                                                     throws java.io.IOException

Creates the Dataflow Job.

Throws:: java.io.IOException

+ + + +

listJobs

public com.google.api.services.dataflow.model.ListJobsResponse listJobs(@Nullable
+                                                                        java.lang.String pageToken)
+                                                                 throws java.io.IOException

Lists Dataflow Jobs in the project associated with + the DataflowPipelineOptions.

Throws:: java.io.IOException

+ + + +

updateJob

public com.google.api.services.dataflow.model.Job updateJob(@Nonnull
+                                                            java.lang.String jobId,
+                                                            @Nonnull
+                                                            com.google.api.services.dataflow.model.Job content)
+                                                     throws java.io.IOException

Updates the Dataflow Job with the given jobId.

Throws:: java.io.IOException

+ + + +

getJob

public com.google.api.services.dataflow.model.Job getJob(@Nonnull
+                                                         java.lang.String jobId)
+                                                  throws java.io.IOException

Gets the Dataflow Job with the given jobId.

Throws:: java.io.IOException

+ + + +

getJobMetrics

public com.google.api.services.dataflow.model.JobMetrics getJobMetrics(@Nonnull
+                                                                       java.lang.String jobId)
+                                                                throws java.io.IOException

Gets the JobMetrics with the given jobId.

Throws:: java.io.IOException

+ + + +

listJobMessages

public com.google.api.services.dataflow.model.ListJobMessagesResponse listJobMessages(@Nonnull
+                                                                                      java.lang.String jobId,
+                                                                                      @Nullable
+                                                                                      java.lang.String pageToken)
+                                                                               throws java.io.IOException

Lists job messages with the given jobId.

Throws:: java.io.IOException

+ + + +

leaseWorkItem

public com.google.api.services.dataflow.model.LeaseWorkItemResponse leaseWorkItem(@Nonnull
+                                                                                  java.lang.String jobId,
+                                                                                  @Nonnull
+                                                                                  com.google.api.services.dataflow.model.LeaseWorkItemRequest request)
+                                                                           throws java.io.IOException

Leases the work item for jobId.

Throws:: java.io.IOException

+ + + +

reportWorkItemStatus

public com.google.api.services.dataflow.model.ReportWorkItemStatusResponse reportWorkItemStatus(@Nonnull
+                                                                                                java.lang.String jobId,
+                                                                                                @Nonnull
+                                                                                                com.google.api.services.dataflow.model.ReportWorkItemStatusRequest request)
+                                                                                         throws java.io.IOException

Reports the status of the work item for jobId.

Throws:: java.io.IOException

+ + +

Skip navigation links

+ + + +

Prev Class
Next Class

Frames
No Frames

All Classes

+ +

Summary:
Nested |
Field |
Constr |
Method

Detail:
Field |
Constr |
Method

+ + +

org.apache.beam.runners.apex

Interfaces

Classes

Package org.apache.beam.runners.apex

Package org.apache.beam.runners.apex Description

Hierarchy For Package org.apache.beam.runners.apex

Class Hierarchy

Interface Hierarchy

Class BatchStatefulParDoOverrides.BatchStatefulDoFn<K,V,OutputT>

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.beam.sdk.transforms.DoFn

Field Summary

Fields inherited from class org.apache.beam.sdk.transforms.DoFn

Method Summary

Methods inherited from class org.apache.beam.sdk.transforms.DoFn

Methods inherited from class java.lang.Object

Method Detail

getUnderlyingDoFn

processElement

getOutputTypeDescriptor

Class BatchStatefulParDoOverrides

Nested Class Summary

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Constructor Detail

BatchStatefulParDoOverrides

Method Detail

singleOutputOverrideFactory

multiOutputOverrideFactory

Class DataflowClient

Method Summary

Methods inherited from class java.lang.Object

Method Detail

create

createJob

listJobs

updateJob

getJob

getJobMetrics

listJobMessages

leaseWorkItem

reportWorkItemStatus