kylin-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sonny Heer <sonnyh...@gmail.com>
Subject Re: #3 Step Name: Extract Fact Table Distinct Columns (slow)
Date Wed, 14 Mar 2018 16:54:07 GMT
8 YARN nodes with 11 slots each.  each slot is configured to ~2gb.  Step #3
in Kylin is launching 19 mappers and 5 reducers.  5 reducers when there are
88 slots.

btw: kylin version is 1.6

On Wed, Mar 14, 2018 at 9:48 AM, Sonny Heer <sonnyheer@gmail.com> wrote:

> YARN is properly configured.  we use many other m/r and spark programs
> that utilize the full slots.  It's only when building cubes.
>
> On Wed, Mar 14, 2018 at 9:46 AM, Alberto Ramón <a.ramonportoles@gmail.com>
> wrote:
>
>> You need  check your yarn configuration first
>>
>> On Wed, 14 Mar 2018, 14:58 Sonny Heer, <sonnyheer@gmail.com> wrote:
>>
>>> Step 3 isn't using our full cluster.  How can i increase the
>>> mappers/reducers to use all the slots?  Any config to look at in kylin?
>>>
>>> Thanks
>>>
>>
>

Mime
View raw message