impala-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Joe McDonnell <joemcdonn...@cloudera.com>
Subject Test data directory layout change (IMPALA-6052)
Date Wed, 13 Dec 2017 21:14:39 GMT
I just uploaded a preview of a code change for IMPALA-6052, which changes
the HDFS directory locations for Impala test data:
https://gerrit.cloudera.org/#/c/8260/
Summary of the change below.

This change would require all developers to reload test data, so I wanted
to start a discussion about the timing of this change. In particular, does
a change like this belong in a point release (2.12)? Are there any concerns
about going forward with this change for 2.12?

Thanks,
Joe

Test tables will now be organized into database directories rather than
being at the top level of /test-warehouse. The new format matches the
default placement of a table when LOCATION is not specified.

e.g.
Table: functional.alltypes
Old: /test-warehouse/alltypes
New: /test-warehouse/functional.db/alltypes

Table: functional_parquet.alltypes
Old: /test-warehouse/alltypes_parquet
New: /test-warehouse/functional_parquet.db/alltypes

Before this change, /test-warehouse has 900+ subdirectories. After the
change, it has about 60. This should make it easier to navigate our HDFS
directories.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message