hawq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hong Wu <xunzhang...@gmail.com>
Subject Re: feature test framework issue
Date Sat, 09 Jul 2016 15:33:44 GMT
@Paul, @Ruilong:

Check out my pull request for this issue in #781
<https://github.com/apache/incubator-hawq/pull/781>. Further discussion and
comments will be at Github or JIRA(HAWQ-911
<https://issues.apache.org/jira/browse/HAWQ-911>), so I will close this
thread here. Thanks.

Best
xunzhang

2016-07-09 15:38 GMT+08:00 Hong Wu <xunzhangthu@gmail.com>:

> Ok, I see.
>
> I buy this thread. Waiting for my pull request soon.
>
> Best
> xunzhang
>
> 2016-07-09 14:49 GMT+08:00 Paul Guo <paulguo@gmail.com>:
>
>> Not really. Makefile compiles all test cpp files, links them with various
>> libraries e.g. libpq, libgtest, etc in one command line. This does not
>> make
>> sense.  Test cases, test lib code should be independent makefile targets.
>>
>> 2016-07-08 22:15 GMT+08:00 Ruilong Huo <rhuo@pivotal.io>:
>>
>> > From Makefile of feature test, we can see most of the time spent is on
>> > linking google test, libpq, libxml2 with feature test.
>> > Can we pre-link google test, libpq, libxml2 as they are relatively
>> stable?
>> >
>> > PS: we can refined the compile of feature test code as well as shared
>> > library so that they don't get compiled even there is no code change.
>> >
>> > Best regards,
>> > Ruilong Huo
>> >
>> > On Fri, Jul 8, 2016 at 10:34 AM, Gmail <xunzhangthu@gmail.com> wrote:
>> >
>> >> Thanks for pointing this.
>> >> I am not sure, but I could remember the current way is to compile to
>> >> object files and then linking together to a feature-test binary.
>> >>
>> >> Besides, I think the current compile time is like 80% of linking + 20%
>> of
>> >> compiling objects.
>> >>
>> >> Best
>> >> xunzhang
>> >> Send from my iPhone
>> >>
>> >> > 在 2016年7月8日,10:23,Paul Guo <paulguo@gmail.com> 写道:
>> >> >
>> >> > I'm adding some feature tests into hawq code, however I found
>> currently
>> >> the makefile code compiles all of the test cpp files each time,
>> regardless
>> >> of single file change or no file change. This is too time-consuming
>> given
>> >> in the future there will be more and more cases. See more below.
>> >> >
>> >> > g++ -I/usr/include -I/usr/local/include -I/usr/include/libxml2
>> >> -I/data2/github/incubator-hawq/src/test/feature/
>> >> -I/data2/github/incubator-hawq/src/test/feature/lib/
>> >> -I/data2/github/incubator-hawq/src/interfaces/libpq
>> >> -I/data2/github/incubator-hawq/src/interfaces
>> >> -I/data2/github/incubator-hawq/src/include
>> >>
>> -I/data2/github/incubator-hawq/depends/thirdparty/googletest/googletest/include
>> >>
>> -I/data2/github/incubator-hawq/depends/thirdparty/googletest/googlemock/include
>> >> -Wall -O0 -g -std=c++11 test_main.cpp ExternalSource/test_errortbl.cpp
>> >> ExternalSource/test_exttab.cpp ManagementTool/test_hawq_register.cpp
>> >> PreparedStatement/TestPreparedStatement.cpp UDF/TestUDF.cpp
>> >> ao/TestAoSnappy.cpp catalog/test_alter_owner.cpp
>> >> catalog/test_alter_table.cpp lib/command.cpp lib/data_gen.cpp
>> >> lib/file_replace.cpp lib/hawq_config.cpp lib/psql.cpp lib/sql_util.cpp
>> >> lib/string_util.cpp lib/xml_parser.cpp parquet/test_parquet.cpp
>> >> partition/test_partition.cpp query/test_aggregate.cpp
>> query/test_insert.cpp
>> >> query/test_prepare.cpp query/test_sequence.cpp testlib/test_lib.cpp
>> >> type/TestType.cpp -L../../../src/port -L../../../src/port
>> -Wl,--as-needed
>> >>
>> -L/data2/github/incubator-hawq/depends/libhdfs3/build/install/data2/github/hawq-install/lib
>> >>
>> -L/data2/github/incubator-hawq/depends/libyarn/build/install/data2/github/hawq-install/lib
>> >> -Wl,-rpath,'/data2/github/hawq-install/lib',--enable-new-dtags
>> >> -L/usr/local/lib -L/usr/lib
>> >> -L/data2/github/incubator-hawq/src/test/feature/
>> >> -L/data2/github/incubator-hawq/src/test/feature/lib/
>> >> -L/data2/github/incubator-hawq/src/interfaces/libpq
>> >>
>> -L/data2/github/incubator-hawq/depends/thirdparty/googletest/build/googlemock
>> >>
>> -L/data2/github/incubator-hawq/depends/thirdparty/googletest/build/googlemock/gtest
>> >> -lpgport -ljson-c -levent -lyaml -lsnappy -lbz2 -lrt -lssl -lcrypto -lz
>> >> -lreadline -lcrypt -ldl -lm  -L/usr/local/lib -lcurl -lyarn -lkrb5
>> -lgtest
>> >> -lpq -lxml2 -ltest -o feature-test
>> >> >
>> >> >
>> >> > PROG = test_main.cpp $(wildcard */*.cpp)
>> >> > RM = rm -rf
>> >> >
>> >> > .PHONY: all sharelib sharelibclean distclean clean doc
>> >> >
>> >> > all: sharelib
>> >> >     $(MAKE) -C lib all
>> >> >     $(CXX) $(CPPFLAGS) $(CXXFLAGS) $(PROG) $(LDFLAGS) $(LIBS) -o
>> >> feature-test
>> >> >
>> >> > Above is related code in Makefile. since share lib is a .PHONY target
>> >> without dependency, so
>> >> > above code will always be executed.
>> >> >
>> >> > So we could compile each test_$case.cpp into one .o and link link
>> them
>> >> into one.
>> >> > The will make compile faster, besides, take the feature test cases
>> >> under ao/ for example,
>> >> > we could make the relative path starts with $test_root/ao for the ao/
>> >> cases. This
>> >> > make more sense.
>> >> >
>> >> > TEST_F(TestAOSnappy, Create1048576)
>> >> > {
>> >> >     hawq::test::SQLUtility util;
>> >> >     util.execSQLFile("ao/sql/ao_crtb_with_row_snappy_1048576.sql",
>> >> >                      "ao/ans/ao_crtb_with_row_snappy_1048576.ans",
>> >> >                      initFile);
>> >> > }
>> >> >
>> >> > Any idea?
>> >>
>> >
>> >
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message