hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Szehon Ho (JIRA)" <>
Subject [jira] [Commented] (HIVE-6500) Stats collection via filesystem
Date Thu, 09 Oct 2014 22:55:34 GMT


Szehon Ho commented on HIVE-6500:

[~leftylev] This looks good to me, although my knowledge is limited about stats.  Only comment
is there seems to be an un-needed dot on the configuration wiki page:  jdbc(:.<database>)

I think it makes sense to fix that as you suggested in HIVE-6586.  Thanks!

PS Yes , I think so.  On [|],
it's not listed  in the 'children' list on the left.  I misunderstood to think all supported
pages are listed there..

> Stats collection via filesystem
> -------------------------------
>                 Key: HIVE-6500
>                 URL:
>             Project: Hive
>          Issue Type: New Feature
>          Components: Statistics
>            Reporter: Ashutosh Chauhan
>            Assignee: Ashutosh Chauhan
>              Labels: TODOC13, TODOC14
>             Fix For: 0.13.0
>         Attachments: HIVE-6500.2.patch, HIVE-6500.3.patch, HIVE-6500.patch
> Recently, support for stats gathering via counter was [added |]
Although, its useful it has following issues:
> * [Length of counter group name is limited |]
> * [Length of counter name is limited |]
> * [Number of distinct counter groups are limited |]
> * [Number of distinct counters are limited |]
> Although, these limits are configurable, but setting them to higher value implies increased
memory load on AM and job history server.
> Now, whether these limits makes sense or not is [debatable |]
it is desirable that Hive doesn't make use of counters features of framework so that it we
can evolve this feature without relying on support from framework. Filesystem based counter
collection is a step in that direction.

This message was sent by Atlassian JIRA

View raw message