phoenix-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PHOENIX-5069) Use asynchronous refresh to provide non-blocking Phoenix Stats Client Cache
Date Wed, 02 Jan 2019 21:14:00 GMT

    [ https://issues.apache.org/jira/browse/PHOENIX-5069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16732430#comment-16732430
] 

ASF GitHub Bot commented on PHOENIX-5069:
-----------------------------------------

Github user karanmehta93 commented on a diff in the pull request:

    https://github.com/apache/phoenix/pull/416#discussion_r244856169
  
    --- Diff: phoenix-core/src/main/java/org/apache/phoenix/query/GuidePostsCache.java ---
    @@ -80,38 +94,46 @@ public GuidePostsCache(ConnectionQueryServices queryServices, Configuration
conf
                     // Log removals at TRACE for debugging
                     .removalListener(new PhoenixStatsCacheRemovalListener())
                     // Automatically load the cache when entries are missing
    -                .build(new StatsLoader());
    +                .build(new PhoenixStatsCacheLoader(new StatsLoaderImpl(), executor));
         }
     
         /**
    -     * {@link CacheLoader} implementation for the Phoenix Table Stats cache.
    +     * {@link PhoenixStatsLoader} implementation for the Stats Loader.
          */
    -    protected class StatsLoader extends CacheLoader<GuidePostsKey, GuidePostsInfo>
{
    +    protected class StatsLoaderImpl implements PhoenixStatsLoader {
    +        @Override
    +        public boolean needsLoad() {
    +            // Whenever it's called, we try to load stats from stats table
    +            // no matter it has been updated or not.
    +            return true;
    +        }
    +
             @Override
    -        public GuidePostsInfo load(GuidePostsKey statsKey) throws Exception {
    +        public GuidePostsInfo loadStats(GuidePostsKey statsKey, GuidePostsInfo prevGuidepostInfo)
throws Exception {
                 @SuppressWarnings("deprecation")
    -            Table statsHTable = queryServices.getTable(SchemaUtil.getPhysicalName(
    +            TableName tableName = SchemaUtil.getPhysicalName(
                         PhoenixDatabaseMetaData.SYSTEM_STATS_NAME_BYTES,
    -                            queryServices.getProps()).getName());
    +                    queryServices.getProps());
    +            Table statsHTable = queryServices.getTable(tableName.getName());
                 try {
                     GuidePostsInfo guidePostsInfo = StatisticsUtil.readStatistics(statsHTable,
statsKey,
                             HConstants.LATEST_TIMESTAMP);
                     traceStatsUpdate(statsKey, guidePostsInfo);
                     return guidePostsInfo;
                 } catch (TableNotFoundException e) {
                     // On a fresh install, stats might not yet be created, don't warn about
this.
    -                logger.debug("Unable to locate Phoenix stats table", e);
    -                return GuidePostsInfo.NO_GUIDEPOST;
    +                logger.debug("Unable to locate Phoenix stats table: " + tableName.toString(),
e);
    +                return prevGuidepostInfo;
                 } catch (IOException e) {
    -                logger.warn("Unable to read from stats table", e);
    +                logger.warn("Unable to read from stats table: " + tableName.toString(),
e);
                     // Just cache empty stats. We'll try again after some time anyway.
    --- End diff --
    
    nit: remove this comment.


> Use asynchronous refresh to provide non-blocking Phoenix Stats Client Cache
> ---------------------------------------------------------------------------
>
>                 Key: PHOENIX-5069
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-5069
>             Project: Phoenix
>          Issue Type: Improvement
>            Reporter: Bin Shi
>            Assignee: Bin Shi
>            Priority: Major
>
> The current Phoenix Stats Cache uses TTL based eviction policy. A cached entry will expire
after a given amount of time (900s by default) passed since the entry's been created. This
will lead to cache miss when Compiler/Optimizer fetches stats from cache at the next time.
As you can see from the above graph, fetching stats from the cache is a blocking operation
— when there is cache miss, it has a round trip over the wire to scan the SYSTEM.STATS Table
and to get the latest stats info, rebuild the cache and finally return the stats to the Compiler/Optimizer.
Whenever there is a cache miss, this blocking call causes significant performance penalty and
see periodic spikes.
> *This Jira suggests to use asynchronous refresh mechanism to provide a non-blocking
cache. For details, please see the linked design document below.*
> [~karanmehta93] [~twdsilva@gmail.com] [~dbwong] [~elserj] [~ankit@apache.org] [~sergey
soldatov] 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message