hawq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Paul Guo <paul...@gmail.com>
Subject Re: sanity-check before running cases in feature-test
Date Wed, 20 Jul 2016 02:23:24 GMT
I totally agree that test case results should be tristate (not just OK and
FAIL).
Unfortually Goggletest does not return a runtime SKIPPED similar results.
So I guess we have to fail for those "skipped" cases.

Note Goggletest could skip running or disable cases, but that is not what
we expect.

In goggliest, case is disabled by explicitly adding DISABLED_ before case
name, e.g.

TEST(FooTest, DISABLED_DoesAbc) { ... }
https://github.com/google/googletest/blob/master/googletest/docs/V1_7_AdvancedGuide.md#temporarily-disabling-tests

or use gtest_filter option in command line to skip some cases.

I introduced a new class TestPrepare in a git code change (Not motivated
for this thread).

HAWQ-925. Set default locale, timezone & datastyle before running sql
command/file

Since sanity checking still have some concerns,
<https://github.com/apache/incubator-hawq/pull/802/files>



2016-07-12 15:58 GMT+08:00 Lili Ma <lilima@apache.org>:

> Agree with @Jiali on the "SKIPPED" solution, but I think we need do some
> emphasis for the skipped test cases to let the user know. The aim is to
> avoid the user who run the test thinks all the tests have run successfully,
> including the skipped tests.
>
> 2016-07-12 14:22 GMT+08:00 Jiali Yao <jyao@pivotal.io>:
>
> > For the test case checking, I think it should report "SKIPPED" instead of
> >  ERROR.
> > The test case should check whether this feature is supported or not. If
> > supported, run the case; otherwise skipped it.
> > Agree on that we should add it in common lib.
> >
> > On the other topic, I think source greenplum_path.sh is must. It is env
> > related.
> >
> > Thanks
> >
> > Jiali
> >
> >
> >
> > On Tue, Jul 12, 2016 at 2:19 PM, Lei Chang <lei_chang@apache.org> wrote:
> >
> > > I think the better way is to let test cases run under some conditions.
> > >
> > > for example, pl/python is optional, if user did not run configure with
> > > pl/python option, the test about pl/python should not run.
> > >
> > > Cheers
> > > Lei
> > >
> > >
> > >
> > > On Tue, Jul 12, 2016 at 2:15 PM, Ivan Weng <iweng@pivotal.io> wrote:
> > >
> > > > Agree with Hong. Test case should check its environment needed. If
> the
> > > > check failed, it should terminate the execution and report the error.
> > > >
> > > > On Tue, Jul 12, 2016 at 2:04 PM, Hong Wu <xunzhangthu@gmail.com>
> > wrote:
> > > >
> > > > > It is user/developer themselves that should take care. Say, if you
> > > write
> > > > a
> > > > > test case which is related to plpython, why don't you configure
> HAWQ
> > > with
> > > > > "--with-python" option? We should write a README for feature-test
> > that
> > > > > guides user to run this tests. For example, tell them sourcing
> > > > > "greenplum.sh" before running tests.
> > > > >
> > > > > Consequently, I think add such sanity-check is a little bit of
> > > > > over-engineering which will bring extra problems and complexities.
> > > > >
> > > > > Best
> > > > > xunzhang
> > > > >
> > > > > 2016-07-12 13:47 GMT+08:00 Paul Guo <paulguo@gmail.com>:
> > > > >
> > > > > > I have >1 times to encounter some feature test failures due
to
> > > reported
> > > > > > missing stuffs.
> > > > > >
> > > > > > e.g.
> > > > > >
> > > > > > 1. I did not have pl/python installed in my hawq build so
> > > > > >    UDF/sql/function_set_returning.sql fails to "create language
> > > > > plpythonu"
> > > > > >    This makes this case fails.
> > > > > >
> > > > > > 2. Sometimes I forgot to source a greenplum.sh, then all cases
> run
> > > > > >     with failures due to missing psql.
> > > > > >
> > > > > > We seem to be able to improve.
> > > > > >
> > > > > > 1) Sanity-check some file existence in common code, e.g.
> > > > > >     psql, gpdiff.pl,
> > > > > >
> > > > > > 2) Some cases could do sanity-check in their own test constructor
> > > > > > functions,
> > > > > >     e.g. if the case uses the extension plpython, the test case
> > > should
> > > > > > check it itself.
> > > > > >
> > > > > > More thoughts?
> > > > > >
> > > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message