kylin-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ShaoFeng Shi <shaofeng...@apache.org>
Subject Re: [!!Mass Mail][Probable spam]Re: sometimes need quite a long time when building cube
Date Mon, 07 Oct 2019 00:55:54 GMT
Please check the file size of the intermediate hive table first; The file
size should be even after the "Redistribute" step. If not, please check the
columns that it redistributed by (the first three dimensions by default).

Best regards,

Shaofeng Shi 史少锋
Apache Kylin PMC
Email: shaofengshi@apache.org

Apache Kylin FAQ: https://kylin.apache.org/docs/gettingstarted/faq.html
Join Kylin user mail group: user-subscribe@kylin.apache.org
Join Kylin dev mail group: dev-subscribe@kylin.apache.org




Bryan Liu (CN) <Bryan.Liu@homecredit.cn> 于2019年10月7日周一 上午6:52写道:

> Hi Shaofeng
>    It was in map phase. Thank you
>
> Bryan
>
>
> 在 2019年10月6日,22:03,ShaoFeng Shi <shaofengshi@apache.org> 写道:
>
> Hi Bryan,
>
> What's the phase of the job in the second screenshot? map phase or reduce
> phase?
>
> Best regards,
>
> Shaofeng Shi 史少锋
> Apache Kylin PMC
> Email: shaofengshi@apache.org
>
> Apache Kylin FAQ: https://kylin.apache.org/docs/gettingstarted/faq.html
> Join Kylin user mail group: user-subscribe@kylin.apache.org
> Join Kylin dev mail group: dev-subscribe@kylin.apache.org
>
>
>
>
> Bryan Liu (CN) <Bryan.Liu@homecredit.cn> 于2019年9月26日周四 下午3:37写道:
>
>> Dears,
>>
>>
>>
>>    I am doing some testing with  Kylin now.  My Cube based on one source
>> table with about 60~70M rows of data for one month.
>>
>>    Normally we build cube need about 25mins .
>>
>>    But sometimes which need more than 3hours , usually in busy period.
>> When I am checking the MapReduce Jobs for cube building step 3(Extract Fact
>> Table Distinct Columns) , I found some Jobs just take several Seconds. But
>> some Jobs take quit a long time.
>>
>>
>>
>>   Please refer to screenshot as bellow.
>>
>>    I think Hadoop do not have enough resource is one reason.  Meanwhile,
>> there should have some problem with Cube building step 2.  Seems the data
>> is non-equilibrium.
>>
>>   Could you please give me some advice ? thank you so much .
>>
>> <image002.jpg>
>>
>> <image006.jpg>
>>
>>

Mime
View raw message