hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sergey Shelukhin (JIRA)" <>
Subject [jira] [Commented] (HIVE-5705) TopN might use better heuristic for disable
Date Fri, 01 Nov 2013 17:53:17 GMT


Sergey Shelukhin commented on HIVE-5705:

[~hagleitn] fyi this is the jira we were talking about yday

> TopN might use better heuristic for disable
> -------------------------------------------
>                 Key: HIVE-5705
>                 URL:
>             Project: Hive
>          Issue Type: Improvement
>            Reporter: Sergey Shelukhin
>            Priority: Minor
> Right now, if TopN overruns memory threshold it disables itself if it couldn't directly
exclude rows as they are sent; it doesn't count evictions that were initially put in the heap
and then superceded for this purpose. 
> It's reasonable in most cases, but if N is relatively small, and map output is large,
the cost could still be worth it even if rows don't get excluded immediately and are only
evicted after being stored for some time. So we'd pay some memory copies but emit much less

This message was sent by Atlassian JIRA

View raw message