Mailing-List: contact mapreduce-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: mapreduce-dev@hadoop.apache.org
Received-SPF: neutral (nike.apache.org: local policy)
Content-Type: text/plain; charset=iso-8859-1
Mime-Version: 1.0 (Apple Message framework v1278)
Subject: Re: Cyclic dependency in JobControl job DAG
From: Vinod Kumar Vavilapalli <vinodkv@hortonworks.com>
In-Reply-To: <CC0DD78A.3C404%evans@yahoo-inc.com>
Date: Mon, 25 Jun 2012 11:13:36 -0700
Content-Transfer-Encoding: quoted-printable
Message-Id: <24CF501E-A635-4956-9B9C-6475F3095AA1@hortonworks.com>
References: <CC0DD78A.3C404%evans@yahoo-inc.com>
To: mapreduce-dev@hadoop.apache.org


+1, can you please open a JIRA ticket (at =
https://issues.apache.org/jira/browse/MAPREDUCE) and attach a patch?

Thanks,
+Vinod

On Jun 25, 2012, at 6:53 AM, Robert Evans wrote:

> I personally think it is useful.  I would say contribute it.
>=20
> (Moved common-dev to bcc, we try not to cross post on these lists)
>=20
> --Bobby Evans
>=20
> On 6/25/12 3:37 AM, "madhu phatak" <phatak.dev@gmail.com> wrote:
>=20
> Hi,
> In current implementation of JobControl, whenever there is a cyclic
> dependency between the jobs it throws a Stack overflow exception .
> For example,
>   ControlledJob job1 =3D new ControlledJob(new Configuration());
> job1.setJobName("job1");
> ControlledJob job2 =3D new ControlledJob(new Configuration());
> job2.setJobName("job2");
> job1.addDependingJob(job2);
> job2.addDependingJob(job1);
>                JobControl jobControl =3D new JobControl("jobcontrol");
> jobControl.addJob(job1);
> jobControl.addJob(job2);
> jobControl.run();
>=20
> throws
>  java.lang.StackOverflowError
> at java.util.ArrayList.get(ArrayList.java:322)
> at
> =
org.apache.hadoop.mapreduce.lib.jobcontrol.ControlledJob.checkState(Contro=
lledJob.java:295)
>=20
> Whenever we write complex application, there is always possibility of
> cyclic dependencies.I have written a method which checks for the =
cyclic
> dependency  upfront and informs it to the user. I want to know from =
you
> guys, do you think is it a useful feature? If yes I can contribute it =
as a
> patch.
>=20
> Regards,
> Madhukara Phatak
> --
> https://github.com/zinnia-phatak-dev/Nectar
>=20