flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From shyla deshpande <deshpandesh...@gmail.com>
Subject Re: docker, error NoResourceAvailableException..
Date Wed, 15 Aug 2018 17:53:33 GMT
Thanks Dominik, I will try that.

On Wed, Aug 15, 2018 at 3:10 AM, Dominik Wosiński <wossyn@gmail.com> wrote:

> Hey,
> The problem is that your command does start Job Manager container, but it
> does not start the Task Manager . That is why you have 0 slots. Currently,
> the default *numberOfTaskSlots* is set to the number of CPUs avaialbe on
> the machine.
>
>
> So, You generally can to do 2 things:
>
>
> 1) Start Job Manager and 2 Task Managers. If you have Docker Compose
> available, you can paste this to your *docker-compose.yml* :
>
>
>
> *services:  jobmanager:    image: *${FLINK_DOCKER_IMAGE_NAME:-flink}
>
> *expose:      *- "6123"
>
> *ports:      *- "8081:8081"
>     *command: *jobmanager
>
> *environment:      *- JOB_MANAGER_RPC_ADDRESS=jobmanager
>
>
> *taskmanager:    image: *${FLINK_DOCKER_IMAGE_NAME:-flink}
>
> *expose:      *- "6121"
>       - "6122"
>
> *depends_on:      *- jobmanager
>     *command: *taskmanager
>
> *links:      *- "jobmanager:jobmanager"
>
> *environment:      *- JOB_MANAGER_RPC_ADDRESS=jobmanager
>
>
> *taskmanager1:    image: *${FLINK_DOCKER_IMAGE_NAME:-flink}
>
> *expose:      *- "6190"
>       - "6120"
>
> *depends_on:      *- jobmanager
>     *command: *taskmanager
>
> *links:      *- "jobmanager:jobmanager"
>
> *environment:      *- JOB_MANAGER_RPC_ADDRESS=jobmanager
>
>
>
> This will give you 1 Job Manager and 2 Task Managers with one task slot
> each, so 2 Task slots in general.
>
> 2) You can deploy 1 Job Manager and 1 Task Manager.Then you need to modify
> *flink-conf.yml* and set the following setting :
>
> *taskmanager.numberOfTaskSlots: *2
>
>
> This will give you 2 Task Slots with only 1 Task Manager. But you will
> need to somehow override config in the container, possibly using :
> https://docs.docker.com/storage/volumes/
>
> Regards,
> Dominik.
>
> *Od: *shyla deshpande <deshpandeshyla@gmail.com>
> *Wysłano: *środa, 15 sierpnia 2018 07:23
> *Do: *user <user@flink.apache.org>
> *Temat: *docker, error NoResourceAvailableException..
>
>
>
> Hello all,
>
>
>
> Trying to use docker as a single node flink cluster.
>
>
>
> docker run --name flink_local -p 8081:8081 -t flink local
>
>
>
> I submited a job to the cluster using the Web UI. The job failed. I see
> this error message in the docker logs.
>
>
>
> org.apache.flink.runtime.jobmanager.scheduler.NoResourceAvailableException:
> Could not allocate all requires slots within timeout of 300000 ms. Slots
> required: 2, slots allocated: 0
>
>
>
> The Web UI, shows 0 taskmanagers and 0 task slots on the Flink dashboard.
>
> How do I start the docker with 2 Task slots?
>
>
>
> Appreciate any help.
>
>
>
> Thanks
>
>
>

Mime
View raw message