pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Thejas M Nair (JIRA)" <j...@apache.org>
Subject [jira] Commented: (PIG-1524) 'Proactive spill count' is misleading
Date Wed, 28 Jul 2010 23:53:16 GMT

    [ https://issues.apache.org/jira/browse/PIG-1524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12893428#action_12893428
] 

Thejas M Nair commented on PIG-1524:
------------------------------------

It will be useful to have a measure of number of records being spilled as well as the number
of bags that spilled.
ie have two counters - 'proactive spill count' and 'proactively spilled record count' .


> 'Proactive spill count' is misleading
> -------------------------------------
>
>                 Key: PIG-1524
>                 URL: https://issues.apache.org/jira/browse/PIG-1524
>             Project: Pig
>          Issue Type: Bug
>            Reporter: Thejas M Nair
>            Assignee: Thejas M Nair
>             Fix For: 0.8.0
>
>
> InternalCacheBag, InternalSortedBag, InternalDistinctBag increment this counter for every
record that it writes to disk, once it exceeds the memory limit. This number is misleading.
> Instead, this counter should be increment it by 1 for each instance of these bags that
has spilled to disk.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message