pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Prashant Kommireddi (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (PIG-4299) SpillableMemoryManager assumes tenured heap incorrectly
Date Fri, 07 Nov 2014 06:47:33 GMT

     [ https://issues.apache.org/jira/browse/PIG-4299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Prashant Kommireddi updated PIG-4299:
-------------------------------------
    Description: 
{code}
for (MemoryPoolMXBean b: mpbeans) {
            log.debug("Found heap (" + b.getName() +
                ") of type " + b.getType());
            if (b.getType() == MemoryType.HEAP) {
                /* Here we are making the leap of faith that the biggest
                 * heap is the tenured heap
                 */
                long size = b.getUsage().getMax();
                totalSize += size;
                if (size > biggestSize) {
                    biggestSize = size;
                    biggestHeap = b;
                }
            }
        }
{code}

A memory pool being the biggest MemoryType.HEAP does not guarantee it being tenured. Moreover,
we must check whether usage threshold is supported by heap before trying to set usage threshold
on it.

Here is the stacktrace that resulted from this bug

java.lang.UnsupportedOperationException: Usage threshold is not supported
at sun.management.MemoryPoolImpl.setUsageThreshold(MemoryPoolImpl.java:114)
at org.apache.pig.impl.util.SpillableMemoryManager.<init>(SpillableMemoryManager.java:130)
at org.apache.pig.impl.util.SpillableMemoryManager.getInstance(SpillableMemoryManager.java:135)
at org.apache.pig.data.BagFactory.<init>(BagFactory.java:123)
at org.apache.pig.data.DefaultBagFactory.<init>(DefaultBagFactory.java:69)
at org.apache.pig.data.BagFactory.getInstance(BagFactory.java:81)
at search.dashboard.VariableLengthTupleToBag.<clinit>(VariableLengthTupleToBag.java:27)

  was:
{code}
for (MemoryPoolMXBean b: mpbeans) {
            log.debug("Found heap (" + b.getName() +
                ") of type " + b.getType());
            if (b.getType() == MemoryType.HEAP) {
                /* Here we are making the leap of faith that the biggest
                 * heap is the tenured heap
                 */
                long size = b.getUsage().getMax();
                totalSize += size;
                if (size > biggestSize) {
                    biggestSize = size;
                    biggestHeap = b;
                }
            }
        }
{code}

A memory pool being the biggest MemoryType.HEAP does not guarantee it being tenured? Moreover,
we must check whether usage threshold is supported by heap before trying to set usage threshold
on it. 


> SpillableMemoryManager assumes tenured heap incorrectly
> -------------------------------------------------------
>
>                 Key: PIG-4299
>                 URL: https://issues.apache.org/jira/browse/PIG-4299
>             Project: Pig
>          Issue Type: Bug
>    Affects Versions: 0.13.0
>            Reporter: Prashant Kommireddi
>            Assignee: Prashant Kommireddi
>             Fix For: 0.14.0
>
>         Attachments: PIG-4299_1.patch
>
>
> {code}
> for (MemoryPoolMXBean b: mpbeans) {
>             log.debug("Found heap (" + b.getName() +
>                 ") of type " + b.getType());
>             if (b.getType() == MemoryType.HEAP) {
>                 /* Here we are making the leap of faith that the biggest
>                  * heap is the tenured heap
>                  */
>                 long size = b.getUsage().getMax();
>                 totalSize += size;
>                 if (size > biggestSize) {
>                     biggestSize = size;
>                     biggestHeap = b;
>                 }
>             }
>         }
> {code}
> A memory pool being the biggest MemoryType.HEAP does not guarantee it being tenured.
Moreover, we must check whether usage threshold is supported by heap before trying to set
usage threshold on it.
> Here is the stacktrace that resulted from this bug
> java.lang.UnsupportedOperationException: Usage threshold is not supported
> at sun.management.MemoryPoolImpl.setUsageThreshold(MemoryPoolImpl.java:114)
> at org.apache.pig.impl.util.SpillableMemoryManager.<init>(SpillableMemoryManager.java:130)
> at org.apache.pig.impl.util.SpillableMemoryManager.getInstance(SpillableMemoryManager.java:135)
> at org.apache.pig.data.BagFactory.<init>(BagFactory.java:123)
> at org.apache.pig.data.DefaultBagFactory.<init>(DefaultBagFactory.java:69)
> at org.apache.pig.data.BagFactory.getInstance(BagFactory.java:81)
> at search.dashboard.VariableLengthTupleToBag.<clinit>(VariableLengthTupleToBag.java:27)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message