kylin-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alberto Ramón <a.ramonporto...@gmail.com>
Subject Re: Optimize Cube Build process
Date Wed, 31 Jan 2018 20:03:04 GMT
How many nodes do you have?
how many RAM and CPU do you have per node?

On 31 January 2018 at 05:07, Kumar, Manoj H <manoj.h.kumar@jpmorgan.com>
wrote:

> It has close to 68 mapper & reducers 500.. It keeps running on this. Pls.
> advise.
>
>
>
> Regards,
>
> Manoj
>
>
>
> *From:* Kumar, Manoj H
> *Sent:* Wednesday, January 31, 2018 9:24 AM
> *To:* 'user@kylin.apache.org' <user@kylin.apache.org>
> *Subject:* Optimize Cube Build process
>
>
>
> Hi Folks – I have close to 33 million of fact data to be processed, Data
> is having lot of unique/Distinct values such Loan_unique_code,
> Facility_code,card_id such.. Dimension looks up are made of these.
>
>
>
> Fact table – 33 millions
>
> Looks up tables having to 3 to 4 millions
>
> Cube build type I have chosen – inmem
>
> Engine – Mapreduce
>
>
>
> Cube build step is taking 90 minutes which is seems to be high. What I can
> do in order to minimize build time? What Parameter I should tweak so that
> Build time gets reduced. Thanks.
>
>
>
>
>
> I have Followed the same steps as given below but it doesn’t help in this
> case
>
>
>
> http://kylin.apache.org/docs21/howto/howto_optimize_build.html
>
>
>
>
>
> Regards,
>
> Manoj
>
>
>
> This message is confidential and subject to terms at: http://
> www.jpmorgan.com/emaildisclaimer including on confidentiality, legal
> privilege, viruses and monitoring of electronic messages. If you are not
> the intended recipient, please delete this message and notify the sender
> immediately. Any unauthorized use is strictly prohibited.
>

Mime
View raw message