kylin-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lu, Kang-Sen" <...@rbbn.com>
Subject question about kylin cube build first step performance
Date Thu, 11 Apr 2019 12:45:18 GMT
I am running kylin 2.5.1.

When I build a cube, the first step tooks 10 minutes. The fact table hour partition is 300MB.
The flat table size is 700MB.

In kylin.log, I saw this log text:

2019-04-10 15:44:49,463 INFO  [Scheduler 766417835 Job 77921bda-8994-12c4-a4ff-11c0f561e8d4-328]
hive.CreateFlatHiveTableStep:38 : INFO  : Map 1: 0(+1)/1       Map 2: 2/2      Map 3: 2/2
2019-04-10 15:44:49,463 INFO  [Scheduler 766417835 Job 77921bda-8994-12c4-a4ff-11c0f561e8d4-328]
hive.CreateFlatHiveTableStep:38 : INFO  : Map 1: 0(+1)/1       Map 2: 2/2      Map 3: 2/2

Does it mean only 3 mapper are used in this first step, i.e. to generate flat table?

How could I reduce the computation time, for both step 1 and step 2? Given that we have a
10-node cluster to use.

Thanks.

Kang-sen


-----------------------------------------------------------------------------------------------------------------------
Notice: This e-mail together with any attachments may contain information of Ribbon Communications
Inc. that
is confidential and/or proprietary for the sole use of the intended recipient.  Any review,
disclosure, reliance or
distribution by others or forwarding without express permission is strictly prohibited.  If
you are not the intended
recipient, please notify the sender immediately and then delete all copies, including any
attachments.
-----------------------------------------------------------------------------------------------------------------------

Mime
View raw message