spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Thomas Graves (JIRA)" <>
Subject [jira] [Commented] (SPARK-3375) spark on yarn alpha container allocation issues
Date Wed, 03 Sep 2014 19:11:51 GMT


Thomas Graves commented on SPARK-3375:

It appears there is a race condition in which if you ask for the containers to soon yarn won't
pick them up.  If I remove the call to     allocator.allocateResources() in the registerAM
and let it wait until the report thread asks then it works.  I need to investigate more on
the YARN side.

> spark on yarn alpha container allocation issues
> -----------------------------------------------
>                 Key: SPARK-3375
>                 URL:
>             Project: Spark
>          Issue Type: Bug
>          Components: YARN
>    Affects Versions: 1.2.0
>            Reporter: Thomas Graves
>            Priority: Blocker
> It looks like if yarn doesn't get the containers immediately it stops asking for them
and the yarn application hangs with never getting any executors.  This was introduced by

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message