kylin-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shaofeng SHI (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (KYLIN-3421) Improve job scheduler fetch performance
Date Wed, 20 Jun 2018 08:45:00 GMT

     [ https://issues.apache.org/jira/browse/KYLIN-3421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Shaofeng SHI updated KYLIN-3421:
--------------------------------
    Fix Version/s:     (was: v2.4.0)

As 2.4.0 has been kicked off (almost done), so I would suggest move it to next version.

> Improve job scheduler fetch performance
> ---------------------------------------
>
>                 Key: KYLIN-3421
>                 URL: https://issues.apache.org/jira/browse/KYLIN-3421
>             Project: Kylin
>          Issue Type: Improvement
>          Components: Job Engine
>            Reporter: Zhong Yanghong
>            Assignee: Zhong Yanghong
>            Priority: Major
>
> Currently, there are several improvements we can do for job scheduler:
> * For {{DistributedScheduler}}, it does not check {{isReady()}} for checkpoint jobs
> * Since the state lots of executables fetched from HBase are not READY, it's better to
fetch their output first, which can reduce lots of get requests to hbase
> * {{FetcherRunnerWithPriority}} is not enabled in {{DistributedScheduler}}, which needs
to change the lockPath from *{color:#f79232}segmentId{color}* to *{color:#f79232}jobId{color}*



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message