carbondata-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CARBONDATA-204) Query statistics issue
Date Sat, 03 Sep 2016 11:20:21 GMT

    [ https://issues.apache.org/jira/browse/CARBONDATA-204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15460917#comment-15460917
] 

ASF GitHub Bot commented on CARBONDATA-204:
-------------------------------------------

Github user Vimal-Das commented on a diff in the pull request:

    https://github.com/apache/incubator-carbondata/pull/123#discussion_r77435539
  
    --- Diff: core/src/main/java/org/apache/carbondata/core/carbon/querystatistics/DriverQueryStatisticsRecorder.java
---
    @@ -78,106 +82,142 @@ public synchronized void recordStatisticsForDriver(QueryStatistic
statistic, Str
        */
       public void logStatisticsAsTableDriver() {
         synchronized (lock) {
    -      String tableInfo = collectDriverStatistics();
    -      if (null != tableInfo) {
    -        LOGGER.statistic(tableInfo);
    +      for (String key: queryStatisticsMap.keySet()) {
    +        // print sql_parse_t,load_meta_t,block_allocation_t,block_identification_t
    +        // or just print block_allocation_t,block_identification_t
    +        if (queryStatisticsMap.get(key).size() >= 2) {
    --- End diff --
    
    The call can return null, because the keyset obtained reflects the state of map when the
.keySet() method was called, later changes in the map will not be reflected. So, if the element
is removed in the meantime, get() can return null.
    
    Solution: use an iterator over entrySet()


> Query statistics issue 
> -----------------------
>
>                 Key: CARBONDATA-204
>                 URL: https://issues.apache.org/jira/browse/CARBONDATA-204
>             Project: CarbonData
>          Issue Type: Bug
>            Reporter: Gin-zhj
>            Assignee: Gin-zhj
>
> Query statistics issue:
> 1. some query statistics that never be printed will be keeped into querystatisticsMap,
so it will cause "out of memory" for long time running
> 2. in some sceniaro, the driver can't record "sql_parse_time" , the driver statistics
logs will not be output



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message