flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ken Krugler <kkrugler_li...@transpac.com>
Subject Re: Reducing parallelism leads to NoResourceAvailableException
Date Thu, 28 Apr 2016 15:26:23 GMT

> On Apr 28, 2016, at 1:32am, Aljoscha Krettek <aljoscha@apache.org> wrote:
> 
> Hi,
> is this a streaming or batch job?

Batch.

> If it is a batch job, are you using either collect() or print() on a DataSet?

Definitely not a print(). Don’t know about collect(), since the job is created via the Cascading-Flink
planner. Fabian would know best.

— Ken

> 
> Cheers,
> Aljoscha
> 
> On Thu, 28 Apr 2016 at 00:52 Ken Krugler <kkrugler_lists@transpac.com <mailto:kkrugler_lists@transpac.com>>
wrote:
> Hi all,
> 
> In trying out different settings for performance, I run into a job failure case that
puzzles me.
> 
> I’d done a run with a parallelism of 20 (-p 20 via CLI), and the job ran successfully,
on a cluster with 40 slots.
> 
> I then tried with -p 15, and it failed with:
> 
> NoResourceAvailableException: Not enough free slots available to run the job. You can
decrease the operator parallelism…
> 
> But the change was to reduce parallelism - why would that now cause this problem?
> 
> Thanks,
> 
> — Ken

--------------------------
Ken Krugler
+1 530-210-6378
http://www.scaleunlimited.com
custom big data solutions & training
Hadoop, Cascading, Cassandra & Solr




Mime
View raw message