flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robert Metzger <rmetz...@apache.org>
Subject Re: Use specific worker for heavy computations
Date Wed, 12 Oct 2016 08:35:08 GMT
Hi Mariano,

currently, there is nothing available in Flink to execute an operation on a
specific machine.

Regards,
Robert


On Wed, Sep 28, 2016 at 9:40 PM, Mariano Gonzalez <
mariano.gonzalez@uptake.com> wrote:

> I need to load a PFA (portable format for analytics) that can be around 30
> GB and later process it with hadrian which is the java implementation for
> PFA's (https://github.com/opendatagroup/hadrian).
>
> I would like to execute this transformation step inside a specific worker
> of the cluster (since I don't want to load 30 GB on every single worker
> node). Unfortunately, hadrian cannot be executed in a distributed way.
>
> So my question would be if there is a way to do some routing with Flink and
> execute this particular transformation step using always the same worker
> node?
>
> Perhaps my approach is completely wrong, so if anybody has any suggestions
> I would be more than happy to hear them:)
>
> Thanks
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message