hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Olga Natkovich (JIRA)" <j...@apache.org>
Subject [jira] Commented: (PIG-1544) proactive-spill bags should share the memory alloted for it
Date Tue, 17 Aug 2010 01:21:18 GMT

    [ https://issues.apache.org/jira/browse/PIG-1544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12899227#action_12899227

Olga Natkovich commented on PIG-1544:

We should not be using these bags for the cases like UDF for exactly the reason you are mentioning

> proactive-spill bags should share the memory alloted for it
> -----------------------------------------------------------
>                 Key: PIG-1544
>                 URL: https://issues.apache.org/jira/browse/PIG-1544
>             Project: Pig
>          Issue Type: Bug
>            Reporter: Thejas M Nair
> Initially proactive spill bags were designed for use in (co)group (InternalCacheBag)
and they knew the total number of proactive bags that were present, and shared the memory
limit specified using the property pig.cachedbag.memusage .
> But the two proactive bag implementations were added later - InternalDistinctBag and
InternalSortedBag are not aware of actual number of bags being used - their users always assume
total-numbags = 3. 
> This needs to be fixed and all proactive-spill bags should share the memory-limit .

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message