pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daniel Dai (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PIG-4299) SpillableMemoryManager assumes tenured heap incorrectly
Date Fri, 07 Nov 2014 22:47:34 GMT

    [ https://issues.apache.org/jira/browse/PIG-4299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14202869#comment-14202869
] 

Daniel Dai commented on PIG-4299:
---------------------------------

+1

> SpillableMemoryManager assumes tenured heap incorrectly
> -------------------------------------------------------
>
>                 Key: PIG-4299
>                 URL: https://issues.apache.org/jira/browse/PIG-4299
>             Project: Pig
>          Issue Type: Bug
>    Affects Versions: 0.13.0
>            Reporter: Prashant Kommireddi
>            Assignee: Prashant Kommireddi
>             Fix For: 0.14.0
>
>         Attachments: PIG-4299_1.patch
>
>
> {code}
> for (MemoryPoolMXBean b: mpbeans) {
>             log.debug("Found heap (" + b.getName() +
>                 ") of type " + b.getType());
>             if (b.getType() == MemoryType.HEAP) {
>                 /* Here we are making the leap of faith that the biggest
>                  * heap is the tenured heap
>                  */
>                 long size = b.getUsage().getMax();
>                 totalSize += size;
>                 if (size > biggestSize) {
>                     biggestSize = size;
>                     biggestHeap = b;
>                 }
>             }
>         }
> {code}
> A memory pool being the biggest MemoryType.HEAP does not guarantee it being tenured.
Moreover, we must check whether usage threshold is supported by heap before trying to set
usage threshold on it.
> Here is the stacktrace that resulted from this bug
> java.lang.UnsupportedOperationException: Usage threshold is not supported
> at sun.management.MemoryPoolImpl.setUsageThreshold(MemoryPoolImpl.java:114)
> at org.apache.pig.impl.util.SpillableMemoryManager.<init>(SpillableMemoryManager.java:130)
> at org.apache.pig.impl.util.SpillableMemoryManager.getInstance(SpillableMemoryManager.java:135)
> at org.apache.pig.data.BagFactory.<init>(BagFactory.java:123)
> at org.apache.pig.data.DefaultBagFactory.<init>(DefaultBagFactory.java:69)
> at org.apache.pig.data.BagFactory.getInstance(BagFactory.java:81)
> at search.dashboard.VariableLengthTupleToBag.<clinit>(VariableLengthTupleToBag.java:27)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message