impala-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alex Behm <alex.b...@cloudera.com>
Subject Re: IMPALA-4624 requires reload of data for local PlannerTest runs
Date Tue, 07 Mar 2017 00:42:35 GMT
The regex is baked into the test validation code: TestUtils.java, look
at FileSizeFilter

On Mon, Mar 6, 2017 at 4:36 PM, Joseph McDonnell <joemcdonnell@cloudera.com>
wrote:

> Looking through the PlannerTest/*.test files, I don't see any regexes for
> the file sizes.
>
> On Mon, Mar 6, 2017 at 4:22 PM, Daniel Hecht <dhecht@cloudera.com> wrote:
>
> > I thought we had replaced all file sizes in the planner tests expected
> > results with regex '/d+' so they wouldn't be file size sensitive. Maybe
> we
> > just missed some places?
> >
> > On Mon, Mar 6, 2017 at 4:09 PM, Joseph McDonnell <
> > joemcdonnell@cloudera.com>
> > wrote:
> >
> > > IMPALA-4624 changes Impala's Parquet writer to fill in the column chunk
> > > metadata's encoding_stats field. This changed the file size for some of
> > the
> > > Parquet files used in our tests, resulting in a diff in
> > > PlannerTest::testPredicatePropagation. Since the log file is now
> updated
> > > to
> > > the new file sizes, any run of PlannerTest with the old tables will hit
> > > this diff. The two tables involved are tpch_parquet.regionkey and
> > > tpch_parquet.nation. Here is a workaround short of a full data reload:
> > >
> > > use tpch_parquet;
> > > insert overwrite table nation select * from nation;
> > > insert overwrite table region select * from region;
> > >
> > > It is possible that other tests that are not run as part of the normal
> > test
> > > suite may have a similar issue.
> > >
> > > Thanks,
> > > Joe
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message