drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kunal Khatua (JIRA)" <j...@apache.org>
Subject [jira] [Created] (DRILL-5270) Improve loading of profiles listing in the WebUI
Date Fri, 17 Feb 2017 00:34:41 GMT
Kunal Khatua created DRILL-5270:
-----------------------------------

             Summary: Improve loading of profiles listing in the WebUI
                 Key: DRILL-5270
                 URL: https://issues.apache.org/jira/browse/DRILL-5270
             Project: Apache Drill
          Issue Type: Improvement
          Components: Web Server
    Affects Versions: 1.9.0
            Reporter: Kunal Khatua
            Assignee: Kunal Khatua
             Fix For: 1.10.0


Currently, as the number of profiles increase, we reload the same list of profiles from the
FS.
An ideal improvement would be to detect if there are any new profiles and only reload from
the disk then. Otherwise, a cached list is sufficient.

For a directory of 280K profiles, the load time is close to 6 seconds on a 32 core server.
With the caching, we can get it down to as much as a few milliseconds.

To render the cache as invalid, we inspect the last modified time of the directory to confirm
whether a reload is needed. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message