hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alan Gates <ga...@yahoo-inc.com>
Subject Re: Local and MapReduce Test modes
Date Thu, 24 Apr 2008 12:31:00 GMT

We definitely need to be testing in both modes.  As you note, we need 
some way for a test to indicate it is only appropriate to run it in one 
mode or the other.  We also should define ant targets so that developers 
can run just local or just mapreduce.


P.S. The illustrate command now works in mapreduce mode too.

Mathieu Poumeyrol wrote:
> All,
> During my — not finished yet — struggle with the "using" clause for 
> the order operator, I found out that the tests for the order clause 
> where only run in MapReduce mode, not in local mode. I proposed a 
> patch to fix this test, but Pi rightly suggest that we look for a more 
> global approach to run the tests systematically in both modes. Indeed, 
> I think I have read somewhere that the new (and very exciting) 
> "illustrate" operator is only tested and only work in local mode — for 
> now.
> We could have an abstract test class responsible for PigContext setup 
> and teardown (local mode or MiniCluster based). All (or most of) the 
> unit tests would inherit from it. The ant script could run the test 
> suite twice passing some property to the abstract class to define 
> which PigContext version should be used. Some tests may still want to 
> "skip" one of the two modes, but at least running test in two modes 
> would become the default and natural behavior.
> What do you think ?

View raw message