Mailing-List: contact dev-help@apex.incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@apex.incubator.apache.org
Date: Mon, 21 Mar 2016 23:32:25 +0000 (UTC)
From: "Munagala V. Ramanath (JIRA)" <jira@apache.org>
To: dev@apex.incubator.apache.org
Message-ID: <JIRA.12950993.1458161633000.11291.1458603145583@Atlassian.JIRA>
In-Reply-To: <JIRA.12950993.1458161633000@Atlassian.JIRA>
References: <JIRA.12950993.1458161633000@Atlassian.JIRA>
 <JIRA.12950993.1458161633087@arcas>
Subject: [jira] [Commented] (APEXCORE-392) Stack Overflow when launching
 jobs
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable


    [ https://issues.apache.org/jira/browse/APEXCORE-392?page=3Dcom.atlassi=
an.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=3D15=
205362#comment-15205362 ]=20

Munagala V. Ramanath commented on APEXCORE-392:
-----------------------------------------------

Which Java version is in use here ?


> Stack Overflow when launching jobs
> ----------------------------------
>
>                 Key: APEXCORE-392
>                 URL: https://issues.apache.org/jira/browse/APEXCORE-392
>             Project: Apache Apex Core
>          Issue Type: Bug
>    Affects Versions: 3.2.0, 3.3.0
>            Reporter: Ilya Ganelin
>            Assignee: Chandni Singh
>            Priority: Blocker
>
> I=E2=80=99m running into a very frustrating issue where certain DAG confi=
gurations cause the following error log (attached). When this happens, my a=
pplication even fails to launch. This does not seem to be a YARN issue sinc=
e this occurs even with a relatively small number of partitions/memory.=20
> This issue DOES appear to be related to HDFS input/output operations sinc=
e the specific parameter that appears to affect things is the number of phy=
sical partitions for the HDFS input/output operators.
> I=E2=80=99ve also attached the input and output operators in question:
> https://gist.github.com/ilganeli/7f770374113b40ffa18a
> I can get this to occur predictable by
>   1.  Increasing the partition count on my input operator (reads from HDF=
S) - values above 20 cause this error
>   2.  Increase the partition count on my output operator (writes to HDFS)=
 - values above 20 cause this error
>   3.  Set stream locality from the default to either thread local, node l=
ocal, or container_local on the output operator
> This behavior is very frustrating as it=E2=80=99s preventing me from part=
itioning my HDFS I/O appropriately, thus allowing me to scale to higher thr=
oughputs.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)