beam-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ivan <email2liy...@gmail.com>
Subject How does param desiredBundleSizeBytes of BoundedSource#split get determined at runtime?
Date Tue, 11 Jul 2017 03:39:51 GMT
Hi, we are trying to build a custom BoundedSource based on gRPC call. in 
class BoundedSource<T> we got method below

public abstract java.util.List<? extendsBoundedSource 
<https://beam.apache.org/documentation/sdks/javadoc/2.0.0/org/apache/beam/sdk/io/BoundedSource.html><T

<https://beam.apache.org/documentation/sdks/javadoc/2.0.0/org/apache/beam/sdk/io/BoundedSource.html>>>
split(long desiredBundleSizeBytes,
                                                                  PipelineOptions 
<https://beam.apache.org/documentation/sdks/javadoc/2.0.0/org/apache/beam/sdk/options/PipelineOptions.html>
 options)
                                                           throws java.lang.Exception

at runtime, how does the parameter desiredBundleSizeBytes get determined, is it different
under different runner? what's the behavior in Flink-Runner? what's the relational ship of
this parameter and the parallelism we specify when in theFlinkPipelieOptions?



Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message