kylin-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Zhou Kang <zhoukan...@outlook.com>
Subject [DISCUSS] Cost-benefit of HBase scan result compression
Date Fri, 03 Jan 2020 02:32:22 GMT
Hiļ¼Œall

kylin.storage.hbase.endpoint-compress-result is TRUE as default.
In Xiaomi Group, we found compression will cause query time latency up to 30 sec and more.
After we analyze log in HBase, we found compression is useless in most situations.

Detail info you can see in : https://issues.apache.org/jira/browse/KYLIN-4322

And more, in our environment,

1.     Only 0.05% data is bigger than 1M

2.     Almost 70% compression data is larger than source data.

So, should we set this config FALSE as default.
And, kylin.storage.hbase.endpoint-compress-result should be override in cube or project, which
is forbidden in CubeVisitService:visitCube now.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message