flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Paschek, Robert" <robert.pasc...@tu-berlin.de>
Subject Getting the NumberOfParallelSubtask
Date Mon, 20 Jun 2016 13:56:34 GMT
Hi Mailing list,

using a RichMapPartitionFunction i can access the total number m of this mapper utilized in
my job with
int m = getRuntimeContext().getNumberOfParallelSubtasks();

I think that would be - in general - the total number of CPU Cores used by Apache Flink among
the cluster.

Is there a way to access the number of the following reducer?

In general i would assume that the number of the following reducers depends on the number
of groups generated by the groupBy() transformation. So the number of the reducer r would
be 1 <= r <= m.

My Job:
DataSet<?> output = input
				.mapPartition(new MR_GPMRS_Mapper())
				.groupBy(0)
				.reduceGroup(new MR_GPMRS_Reducer());

Thank you in advance
Robert
Mime
View raw message