hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Olga Natkovich (JIRA)" <j...@apache.org>
Subject [jira] Commented: (PIG-1544) proactive-spill bags should share the memory alloted for it
Date Tue, 17 Aug 2010 19:07:16 GMT

    [ https://issues.apache.org/jira/browse/PIG-1544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12899529#action_12899529

Olga Natkovich commented on PIG-1544:

So we should not use them in this case either. We should only use internal bags for things
we no upfront

> proactive-spill bags should share the memory alloted for it
> -----------------------------------------------------------
>                 Key: PIG-1544
>                 URL: https://issues.apache.org/jira/browse/PIG-1544
>             Project: Pig
>          Issue Type: Bug
>            Reporter: Thejas M Nair
> Initially proactive spill bags were designed for use in (co)group (InternalCacheBag)
and they knew the total number of proactive bags that were present, and shared the memory
limit specified using the property pig.cachedbag.memusage .
> But the two proactive bag implementations were added later - InternalDistinctBag and
InternalSortedBag are not aware of actual number of bags being used - their users always assume
total-numbags = 3. 
> This needs to be fixed and all proactive-spill bags should share the memory-limit .

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message