Mailing-List: contact crunch-user-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: crunch-user@incubator.apache.org
Received-SPF: pass (athena.apache.org: domain of jwills@cloudera.com
 designates 209.85.220.182 as permitted sender)
MIME-Version: 1.0
In-Reply-To: <BLU0-SMTP153F327E8AA06314C5F3A23AD000@phx.gbl>
References: <BLU0-SMTP153F327E8AA06314C5F3A23AD000@phx.gbl>
From: Josh Wills <jwills@cloudera.com>
Date: Mon, 4 Feb 2013 19:19:12 -0800
Message-ID: 
 <CAH29n6Mb_cYBFwTfn4CEBXThBuJYLk60CHdurO2tr2ioeUsw0w@mail.gmail.com>
Subject: Re: Visualize DAG of a pipeline
To: "crunch-user@incubator.apache.org" <crunch-user@incubator.apache.org>,
 Gabriel Reid <gabriel.reid@gmail.com>
Content-Type: multipart/alternative; boundary=047d7bacc178a184b704d4f1ad04

--047d7bacc178a184b704d4f1ad04
Content-Type: text/plain; charset=ISO-8859-1

+greid

Gabriel wrote one, IIRC-- I think that a .dot file with the plan for the
job gets embedded in the Configuration object returned from the planner.


On Mon, Feb 4, 2013 at 7:13 PM, Chao Shi <stepinto@live.com> wrote:

> Hi crunch users,
>
> I would like to know if there are any tool to help me understand crunch
> optimized MR stages.
>
> Particularly, I think I need to see the DAG of job stages. I'm writing a
> pipeline consists of several joins. The pipeline produces significant
> more intermediate output than I expect. I want to investigate what's going
> wrong there.
>
> Thanks,
> Chao
>


-- 
Director of Data Science
Cloudera <http://www.cloudera.com>
Twitter: @josh_wills <http://twitter.com/josh_wills>

--047d7bacc178a184b704d4f1ad04
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">+greid<br><br>Gabriel wrote one, IIRC-- I think that a .do=
t file with the plan for the job gets embedded in the Configuration object =
returned from the planner.<div class=3D"gmail_extra"><br><br><div class=3D"=
gmail_quote">

On Mon, Feb 4, 2013 at 7:13 PM, Chao Shi <span dir=3D"ltr">&lt;<a href=3D"m=
ailto:stepinto@live.com" target=3D"_blank">stepinto@live.com</a>&gt;</span>=
 wrote:<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex">Hi crunch users,<div><br></div><div>I would =
like to know if there are any tool to help me understand crunch optimized M=
R stages.</div>


<div><br></div><div>Particularly, I think I need to see the DAG of job stag=
es. I&#39;m writing a pipeline consists of several joins. The pipeline prod=
uces significant more=A0intermediate=A0output than I expect. I want to inve=
stigate what&#39;s going wrong there.</div>


<div><br></div><div>Thanks,</div><div>Chao</div>
</blockquote></div><br><br clear=3D"all"><div><br></div>-- <br><div>Directo=
r of Data Science</div><div><a href=3D"http://www.cloudera.com" target=3D"_=
blank">Cloudera</a></div><div>Twitter: <a href=3D"http://twitter.com/josh_w=
ills" target=3D"_blank">@josh_wills</a></div>


</div></div>

--047d7bacc178a184b704d4f1ad04--