incubator-crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Josh Wills <jwi...@cloudera.com>
Subject Re: pipeline.done with pcollection.materialize in tests, why ?
Date Thu, 26 Jul 2012 13:34:35 GMT
Pipeline.done does some cleanup, like deleting the temp directories that
contain any intermediate output Crunch generated, including the
materialized data. With the changes in CRUNCH-24 to move all of the
temporary directories to use JUnit's TemporaryFolder, I don't think the
done() calls are necessary, but they aren't doing any harm.

On Thu, Jul 26, 2012 at 4:27 AM, Rahul <rsharma@xebia.com> wrote:

> Hi,
>
> I have seen tests where /Pipeline.done()/ was being called in the /@After/
> method  of a junit or at the end of test after all assertions have been
> done.
> All of these tests were using the /materialize() /API to get the
> collection data.
> Now, why is /pipeline.done()/  is being called in such cases ? Does it
> close some open ends, if so then which ones ?
>
> regards
> Rahul
>



-- 
Director of Data Science
Cloudera <http://www.cloudera.com>
Twitter: @josh_wills <http://twitter.com/josh_wills>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message