crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Anujojha5 <anujoj...@gmail.com>
Subject Re: Materialize running the pipeline and not giving stage result.
Date Fri, 23 May 2014 21:39:16 GMT
Thanks a lot Micah.

Sent from my iPhone

> On May 22, 2014, at 2:52 PM, Micah Whitacre <mkwhit@gmail.com> wrote:
> 
> I logged CRUNCH-400[1] to track this.
> 
> [1] - https://issues.apache.org/jira/browse/CRUNCH-400
> 
> 
>> On Fri, May 16, 2014 at 6:02 PM, Jinal Shah <jinalshah2007@gmail.com> wrote:
>> 
>> Yes, MRPipeline.
>>> On May 16, 2014 5:29 PM, "Josh Wills" <jwills@cloudera.com> wrote:
>>> 
>>> Which Pipeline type are you running? MRPipeline?
>>> 
>>> 
>>>> On Fri, May 16, 2014 at 12:41 PM, anuj ojha <anujojha5@gmail.com> wrote:
>>>> 
>>>> Hello All,
>>>>        I am doing the following in my pipeline:
>>>> {
>>>> some processing.
>>>> ........
>>>> pcollections.materialize();
>>>> 
>>>> final PipelineResult pipelineResult = pipeline.run();
>>>> 
>>>> 
>>>> final List<StageResult> stageResults =
>> pipelineResult.getStageResults();
>>>> 
>>>> 
>>>> }
>>>> 
>>>> 
>>>> I am trying to access stageresult and I am getting it as null. I can't
>>>> avoid calling materialize as I use the data which I get from
>>> materializing
>>>> pcollection in further processing. Is there another way to achieve
>> what I
>>>> am doing above?
>>> 
>>> 
>>> 
>>> --
>>> Director of Data Science
>>> Cloudera <http://www.cloudera.com>
>>> Twitter: @josh_wills <http://twitter.com/josh_wills>
>> 

Mime
View raw message