flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stephan Ewen <se...@apache.org>
Subject Re: Use specific worker for heavy computations
Date Wed, 12 Oct 2016 14:01:41 GMT
So far, we have not introduced location constraints.
The reason is that this goes a bit against the paradigm of location
transparency, which is necessary for failover, dynamically adjusting
parallelism (which is a feature being worked on), etc.

On Wed, Oct 12, 2016 at 10:35 AM, Robert Metzger <rmetzger@apache.org>
wrote:

> Hi Mariano,
>
> currently, there is nothing available in Flink to execute an operation on a
> specific machine.
>
> Regards,
> Robert
>
>
> On Wed, Sep 28, 2016 at 9:40 PM, Mariano Gonzalez <
> mariano.gonzalez@uptake.com> wrote:
>
> > I need to load a PFA (portable format for analytics) that can be around
> 30
> > GB and later process it with hadrian which is the java implementation for
> > PFA's (https://github.com/opendatagroup/hadrian).
> >
> > I would like to execute this transformation step inside a specific worker
> > of the cluster (since I don't want to load 30 GB on every single worker
> > node). Unfortunately, hadrian cannot be executed in a distributed way.
> >
> > So my question would be if there is a way to do some routing with Flink
> and
> > execute this particular transformation step using always the same worker
> > node?
> >
> > Perhaps my approach is completely wrong, so if anybody has any
> suggestions
> > I would be more than happy to hear them:)
> >
> > Thanks
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message