crunch-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ron Hashimshony <>
Subject Use of getDettachedValue and Unit Tests
Date Fri, 22 May 2015 14:18:10 GMT
We love Crunch for the great Unit-Testing capabilities, which gives us a
good confidence when running the pipeline on the real data.
However, we did find one place in which the Unit-Tests failed us - when we
need to add *getDettachedValue* call on *iterable*, unit-tests did behave
differently and did not reuse the same objects, as happened in the
production pipeline on the whole data.
Is there any way to incorporate this validation in unit-tests?
We have tests that run a set of *DoFn*s setting the input *MemCollection*s and
check the *MemCollection*s out using *MemPipeline*, and other tests for a
single *DoFn*'s *process* using *InMemoryEmitter*.
Ron Hashimshony

View raw message