kylin-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Debdutto Chakraborty <debdut...@gmail.com>
Subject Partition Date Issue
Date Tue, 15 May 2018 13:42:26 GMT
Hi,

So, we have a hive table with analytical events data (impressions, clicks,
conversions and such). A typical day produces around 50 to 100 million rows
in this table with around 30 columns.

We were trying to move to Kylin and prepare cubes from the data which is in
this table.

Now the problem is:

   1. This hive table is partitioned on YEAR, MONTH, DAY columns. Which are
   separate columns.
   2. Kylin does not accept such separate columns as "Partition Date
   Column".
   3. Running Hive queries on non partitioned columns is a nightmare.


The only solution to this that I see is that give the user an option during
configuration to specify separate columns like this and then create the
query accordingly.

My only concern is that if this will impact the cube's "Refresh Settings"

Please let me know if this should be done. I'm open to do the development
and open a PR.

Regards,
Debdutto Chakraborty

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message