spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Holden Karau <hol...@pigscanfly.ca>
Subject Re: TaskCommitDenied (Driver denied task commit)
Date Thu, 21 Jan 2016 22:14:30 GMT
Can you post more of your log? How big are the partitions? What is the
action you are performing?

On Thu, Jan 21, 2016 at 2:02 PM, Arun Luthra <arun.luthra@gmail.com> wrote:

> Example warning:
>
> 16/01/21 21:57:57 WARN TaskSetManager: Lost task 2168.0 in stage 1.0 (TID
> 4436, XXXXXXX): TaskCommitDenied (Driver denied task commit) for job: 1,
> partition: 2168, attempt: 4436
>
>
> Is there a solution for this? Increase driver memory? I'm using just 1G
> driver memory but ideally I won't have to increase it.
>
> The RDD being processed has 2262 partitions.
>
> Arun
>



-- 
Cell : 425-233-8271
Twitter: https://twitter.com/holdenkarau

Mime
View raw message