ApexYarnLauncher (Apache Beam SDK for Java, version 0.4.0)

java.lang.Object
+
- org.apache.beam.runners.apex.ApexYarnLauncher
+

+
+
+
```
public class ApexYarnLauncher
+extends Object
```
+
Proxy to launch the YARN application through the hadoop script to run in the + pre-configured environment (class path, configuration, native libraries etc.). + +
The proxy takes the DAG and communicates with the Hadoop services to launch + it on the cluster.
+

+ +

Nested Class Summary

+ + + + + + + + + + + + + + +

Nested Classes
Modifier and Type	Class and Description
`protected static class`	`ApexYarnLauncher.LaunchParams` + Launch parameters that will be serialized and passed to the child process. +
`static class`	`ApexYarnLauncher.ProcessWatcher` + Starts a command and waits for it to complete. +

+ +

+ + +
Constructor Summary
+ + + + + + + + +
Constructors
Constructor and Description
ApexYarnLauncher()
+

Constructors
Constructor and Description
`ApexYarnLauncher()`

+ +

+ + +

Method Summary

+ + + + + + + + + + + + + + + + + + + + + + + + + + +

All Methods Static Methods Instance Methods Concrete Methods
Modifier and Type	Method and Description
`static void`	`createJar(File dir, + File jarFile)` + Create a jar file from the given directory. +
`static List<File>`	`getYarnDeployDependencies()` + From the current classpath, find the jar files that need to be deployed + with the application to run on YARN. +
`protected org.apache.apex.api.Launcher.AppHandle`	`launchApp(ApexYarnLauncher.LaunchParams params)`
`org.apache.apex.api.Launcher.AppHandle`	`launchApp(com.datatorrent.api.StreamingApplication app)`
`static void`	`main(String[] args)` + The main method expects the serialized DAG and will launch the YARN application. +

+ + +
Methods inherited from class java.lang.Object
+clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

+ +
- + + +
  Constructor Detail
  + + + +
  - +
    ApexYarnLauncher
    +
```
public ApexYarnLauncher()
```
    +
  +
+ +
- + + +
  Method Detail
  + + + +
  - +
    launchApp
    +
```
public org.apache.apex.api.Launcher.AppHandle launchApp(com.datatorrent.api.StreamingApplication app)
+                                                 throws IOException
```
    +
    +
    Throws:
    +
    IOException
    +
    +
  + + + +
  - +
    launchApp
    +
```
protected org.apache.apex.api.Launcher.AppHandle launchApp(ApexYarnLauncher.LaunchParams params)
+                                                    throws IOException
```
    +
    +
    Throws:
    +
    IOException
    +
    +
  + + + +
  - +
    getYarnDeployDependencies
    +
```
public static List<File> getYarnDeployDependencies()
+                                            throws IOException
```
    +
    From the current classpath, find the jar files that need to be deployed + with the application to run on YARN. Hadoop dependencies are provided + through the Hadoop installation and the application should not bundle them + to avoid conflicts. This is done by removing the Hadoop compile + dependencies (transitively) by parsing the Maven dependency tree.
    +
    +
    Returns:
    +
    list of jar files to ship
    +
    Throws:
    +
    IOException - when dependency information cannot be read
    +
    +
  + + + +
  - +
    createJar
    +
```
public static void createJar(File dir,
+                             File jarFile)
+                      throws IOException
```
    +
    Create a jar file from the given directory.
    +
    +
    Parameters:
    +
    dir - source directory
    +
    jarFile - jar file name
    +
    Throws:
    +
    IOException - when file cannot be created
    +
    +
  + + + +
  - +
    main
    +
```
public static void main(String[] args)
+                 throws IOException
```
    +
    The main method expects the serialized DAG and will launch the YARN application.
    +
    +
    Parameters:
    +
    args - location of launch parameters
    +
    Throws:
    +
    IOException - when parameters cannot be read
    +
    +
  +
+

java.lang.Object
+
- org.apache.beam.sdk.runners.PipelineRunner<ApexRunnerResult>
- +
  - org.apache.beam.runners.apex.TestApexRunner
  +
+

+
+

public class TestApexRunner
+extends PipelineRunner<ApexRunnerResult>

Apex PipelineRunner for testing.

+ +

+ + +

Method Summary

+ + + + + + + + + + + + + + + + + + +

All Methods Static Methods Instance Methods Concrete Methods
Modifier and Type	Method and Description
`<OutputT extends POutput,InputT extends PInput> OutputT`	`apply(PTransform<InputT,OutputT> transform, + InputT input)` + Applies a transform to the given input, returning the output. +
`static TestApexRunner`	`fromOptions(PipelineOptions options)`
`ApexRunnerResult`	`run(Pipeline pipeline)` + Processes the given Pipeline, returning the results. +

+ + +
Methods inherited from class java.lang.Object
+clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

+ +
- + + +
  Method Detail
  + + + +
  - +
    fromOptions
    +
```
public static TestApexRunner fromOptions(PipelineOptions options)
```
    +
  + + + + + +
  - +
    apply
    +
```
public <OutputT extends POutput,InputT extends PInput> OutputT apply(PTransform<InputT,OutputT> transform,
+                                                                     InputT input)
```
    +
    Description copied from class: PipelineRunner
    +
    Applies a transform to the given input, returning the output. + +
    The default implementation calls PTransform.apply(input), but can be overridden + to customize behavior for a particular runner.
    +
    +
    Overrides:
    +
    apply in class PipelineRunner<ApexRunnerResult>
    +
    +
  + + + +
  - +
    run
    +
```
public ApexRunnerResult run(Pipeline pipeline)
```
    +
    Description copied from class: PipelineRunner
    +
    Processes the given Pipeline, returning the results.
    +
    +
    Specified by:
    +
    run in class PipelineRunner<ApexRunnerResult>
    +
    +
  +
+

Packages that use ApexPipelineOptions
Package	Description
org.apache.beam.runners.apex	+ Implementation of the Beam runner for Apache Apex. +

Packages that use ApexPipelineOptions

Package

Description

org.apache.beam.runners.apex

Implementation of the Beam runner for Apache Apex.

Constructors in org.apache.beam.runners.apex with parameters of type ApexPipelineOptions
Constructor and Description
`ApexRunner(ApexPipelineOptions options)`

Constructors in org.apache.beam.runners.apex with parameters of type ApexPipelineOptions

Constructor and Description

ApexRunner(ApexPipelineOptions options)

Packages that use ApexRunner.CreateApexPCollectionView
Package	Description
org.apache.beam.runners.apex	+ Implementation of the Beam runner for Apache Apex. +

Packages that use ApexRunner.CreateApexPCollectionView

Package

Description

org.apache.beam.runners.apex

Implementation of the Beam runner for Apache Apex.

Methods in org.apache.beam.runners.apex that return ApexRunner.CreateApexPCollectionView
Modifier and Type	Method and Description
`static <ElemT,ViewT> ApexRunner.CreateApexPCollectionView<ElemT,ViewT>`	ApexRunner.CreateApexPCollectionView.`of(PCollectionView<ViewT> view)`

Methods in org.apache.beam.runners.apex that return ApexRunner.CreateApexPCollectionView

Modifier and Type

Method and Description

static <ElemT,ViewT>
ApexRunner.CreateApexPCollectionView<ElemT,ViewT>

ApexRunner.CreateApexPCollectionView.of(PCollectionView<ViewT> view)

Packages that use ApexRunner
Package	Description
org.apache.beam.runners.apex	+ Implementation of the Beam runner for Apache Apex. +

Packages that use ApexRunner

Package

Description

org.apache.beam.runners.apex

Implementation of the Beam runner for Apache Apex.

Methods in org.apache.beam.runners.apex that return ApexRunner
Modifier and Type	Method and Description
`static ApexRunner`	ApexRunner.`fromOptions(PipelineOptions options)`

Methods in org.apache.beam.runners.apex that return ApexRunner

Modifier and Type

Method and Description

static ApexRunner

ApexRunner.fromOptions(PipelineOptions options)

Packages that use ApexRunnerResult
Package	Description
org.apache.beam.runners.apex	+ Implementation of the Beam runner for Apache Apex. +

Packages that use ApexRunnerResult

Package

Description

org.apache.beam.runners.apex

Implementation of the Beam runner for Apache Apex.

Methods in org.apache.beam.runners.apex that return ApexRunnerResult
Modifier and Type	Method and Description
`ApexRunnerResult`	ApexRunner.`run(Pipeline pipeline)`
`ApexRunnerResult`	TestApexRunner.`run(Pipeline pipeline)`

Methods in org.apache.beam.runners.apex that return ApexRunnerResult

Modifier and Type

Method and Description

ApexRunnerResult

ApexRunner.run(Pipeline pipeline)

ApexRunnerResult

TestApexRunner.run(Pipeline pipeline)

Class ApexYarnLauncher

Nested Class Summary

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Constructor Detail

ApexYarnLauncher

Method Detail

launchApp

launchApp

getYarnDeployDependencies

createJar

main

Class TestApexRunner

Method Summary

Methods inherited from class java.lang.Object

Method Detail

fromOptions

apply

run

Uses of Interface
org.apache.beam.runners.apex.ApexPipelineOptions

Uses of ApexPipelineOptions in org.apache.beam.runners.apex

Uses of Class
org.apache.beam.runners.apex.ApexRunner.CreateApexPCollectionView

Uses of ApexRunner.CreateApexPCollectionView in org.apache.beam.runners.apex

Uses of Class
org.apache.beam.runners.apex.ApexRunner

Uses of ApexRunner in org.apache.beam.runners.apex

Uses of Class
org.apache.beam.runners.apex.ApexRunnerRegistrar.Options

Uses of Class
org.apache.beam.runners.apex.ApexRunnerRegistrar.Runner

Uses of Class
org.apache.beam.runners.apex.ApexRunnerRegistrar

Uses of Class
org.apache.beam.runners.apex.ApexRunnerResult

Uses of ApexRunnerResult in org.apache.beam.runners.apex

Class ApexYarnLauncher

Nested Class Summary

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Constructor Detail

ApexYarnLauncher

Method Detail

launchApp

launchApp

getYarnDeployDependencies

createJar

main

Class TestApexRunner

Method Summary

Methods inherited from class java.lang.Object

Method Detail

fromOptions

apply

run

Uses of Interfaceorg.apache.beam.runners.apex.ApexPipelineOptions

Uses of ApexPipelineOptions in org.apache.beam.runners.apex

Uses of Classorg.apache.beam.runners.apex.ApexRunner.CreateApexPCollectionView

Uses of ApexRunner.CreateApexPCollectionView in org.apache.beam.runners.apex

Uses of Classorg.apache.beam.runners.apex.ApexRunner

Uses of ApexRunner in org.apache.beam.runners.apex

Uses of Classorg.apache.beam.runners.apex.ApexRunnerRegistrar.Options

Uses of Classorg.apache.beam.runners.apex.ApexRunnerRegistrar.Runner

Uses of Classorg.apache.beam.runners.apex.ApexRunnerRegistrar

Uses of Classorg.apache.beam.runners.apex.ApexRunnerResult

Uses of ApexRunnerResult in org.apache.beam.runners.apex

Uses of Interface
org.apache.beam.runners.apex.ApexPipelineOptions

Uses of Class
org.apache.beam.runners.apex.ApexRunner.CreateApexPCollectionView

Uses of Class
org.apache.beam.runners.apex.ApexRunner

Uses of Class
org.apache.beam.runners.apex.ApexRunnerRegistrar.Options

Uses of Class
org.apache.beam.runners.apex.ApexRunnerRegistrar.Runner

Uses of Class
org.apache.beam.runners.apex.ApexRunnerRegistrar

Uses of Class
org.apache.beam.runners.apex.ApexRunnerResult