phoenix-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <>
Subject [jira] [Commented] (PHOENIX-5069) Use asynchronous refresh to provide non-blocking Phoenix Stats Client Cache
Date Wed, 02 Jan 2019 21:14:00 GMT


ASF GitHub Bot commented on PHOENIX-5069:

Github user karanmehta93 commented on a diff in the pull request:
    --- Diff: phoenix-core/src/main/java/org/apache/phoenix/query/ ---
    @@ -80,38 +94,46 @@ public GuidePostsCache(ConnectionQueryServices queryServices, Configuration
                     // Log removals at TRACE for debugging
                     .removalListener(new PhoenixStatsCacheRemovalListener())
                     // Automatically load the cache when entries are missing
    -                .build(new StatsLoader());
    +                .build(new PhoenixStatsCacheLoader(new StatsLoaderImpl(), executor));
    -     * {@link CacheLoader} implementation for the Phoenix Table Stats cache.
    +     * {@link PhoenixStatsLoader} implementation for the Stats Loader.
    -    protected class StatsLoader extends CacheLoader<GuidePostsKey, GuidePostsInfo>
    +    protected class StatsLoaderImpl implements PhoenixStatsLoader {
    +        @Override
    +        public boolean needsLoad() {
    +            // Whenever it's called, we try to load stats from stats table
    +            // no matter it has been updated or not.
    +            return true;
    +        }
    -        public GuidePostsInfo load(GuidePostsKey statsKey) throws Exception {
    +        public GuidePostsInfo loadStats(GuidePostsKey statsKey, GuidePostsInfo prevGuidepostInfo)
throws Exception {
    -            Table statsHTable = queryServices.getTable(SchemaUtil.getPhysicalName(
    +            TableName tableName = SchemaUtil.getPhysicalName(
    -                            queryServices.getProps()).getName());
    +                    queryServices.getProps());
    +            Table statsHTable = queryServices.getTable(tableName.getName());
                 try {
                     GuidePostsInfo guidePostsInfo = StatisticsUtil.readStatistics(statsHTable,
                     traceStatsUpdate(statsKey, guidePostsInfo);
                     return guidePostsInfo;
                 } catch (TableNotFoundException e) {
                     // On a fresh install, stats might not yet be created, don't warn about
    -                logger.debug("Unable to locate Phoenix stats table", e);
    -                return GuidePostsInfo.NO_GUIDEPOST;
    +                logger.debug("Unable to locate Phoenix stats table: " + tableName.toString(),
    +                return prevGuidepostInfo;
                 } catch (IOException e) {
    -                logger.warn("Unable to read from stats table", e);
    +                logger.warn("Unable to read from stats table: " + tableName.toString(),
                     // Just cache empty stats. We'll try again after some time anyway.
    --- End diff --
    nit: remove this comment.

> Use asynchronous refresh to provide non-blocking Phoenix Stats Client Cache
> ---------------------------------------------------------------------------
>                 Key: PHOENIX-5069
>                 URL:
>             Project: Phoenix
>          Issue Type: Improvement
>            Reporter: Bin Shi
>            Assignee: Bin Shi
>            Priority: Major
> The current Phoenix Stats Cache uses TTL based eviction policy. A cached entry will expire
after a given amount of time (900s by default) passed since the entry's been created. This
will lead to cache miss when Compiler/Optimizer fetches stats from cache at the next time.
As you can see from the above graph, fetching stats from the cache is a blocking operation
— when there is cache miss, it has a round trip over the wire to scan the SYSTEM.STATS Table
and to get the latest stats info, rebuild the cache and finally return the stats to the Compiler/Optimizer.
Whenever there is a cache miss, this blocking call causes significant performance penalty and
see periodic spikes.
> *This Jira suggests to use asynchronous refresh mechanism to provide a non-blocking
cache. For details, please see the linked design document below.*
> [~karanmehta93] [] [~dbwong] [~elserj] [] [~sergey

This message was sent by Atlassian JIRA

View raw message