incubator-crunch-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Micah Whitacre <mkwhita...@gmail.com>
Subject Re: Viewing intermediate states for debugging
Date Thu, 24 Jan 2013 15:58:36 GMT
running dependency:tree on both projects shows that the version of
Avro is 1.7.0 for running under both profiles.  I wish it was that
easy.  :)

On Thu, Jan 24, 2013 at 9:53 AM, Josh Wills <jwills@cloudera.com> wrote:
>
>
>
> On Thu, Jan 24, 2013 at 6:40 AM, Micah Whitacre <mkwhitacre@gmail.com>
> wrote:
>>
>> Taking a step back and comparing what is being generated for a normal
>> successful test run of "-Dcrunch.platform=2" I do see a p1 and p2
>> directory being created, with the expected materialized output being
>> in the p1 directory.  So I'm still curious about tracking all of the
>> intermediate state but it doesn't look like it is an issue with regard
>> to creating the output in the wrong directory.
>
>
> That's a relief. :)
>
> I think the issue with temp outputs has to do with our use of the
> TemporaryPath libraries for creating, well, temporary paths. We do this so
> we play nicely with CI frameworks, but you might need to disable it for
> investigating intermediate outputs.
>
> Re: the specific error you're seeing, that looks interesting. I wonder if
> it's an Avro version change or some such thing. Will see if I can replicate
> it.
>
>
> --
> Director of Data Science
> Cloudera
> Twitter: @josh_wills

Mime
View raw message