flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stephan Ewen <se...@apache.org>
Subject Re: How to execute one operator per Machine that reads a full dataset
Date Fri, 15 Aug 2014 11:31:16 GMT
Hey!

You basically want five different pipelines, on different machines?

I think that is simply a different scheduling the tasks, not sharing
resources. I am working on the scheduler, I can see how to take this
usecase into account...

Stephan



On Fri, Aug 15, 2014 at 1:09 PM, Kay Fleischmann <fleischmann.kay@gmail.com>
wrote:

> Hey guys. Lets say i have a cluster setup with 5 machines.
>
> I am wondering how to distribute an operator (e.g. MapPartition) to all
> machines, exactly one per machine. All of them should read a specific
> source completely.
>
> As i see flink broadcast variable do not work because i nee two different
> sources. Is there an other way?
>
> Kay
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message