hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sam Pullara (JIRA)" <j...@apache.org>
Subject [jira] Commented: (PIG-40) Memory management in BigDataBag is probably wrong
Date Sat, 01 Dec 2007 21:50:42 GMT

    [ https://issues.apache.org/jira/browse/PIG-40?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12547522
] 

Sam Pullara commented on PIG-40:
--------------------------------

One more thing on the finalize() call.  Both close() and delete() can throw runtime exceptions
which would cause files to be left on the disk. Similarly they will be left on disk if the
VM exits abnormally (and sometimes normally, VM dependent).  I would probably add a call to
store.deleteOnExit() when you create the temp files to ensure that they are not left over
no matter what happens.

> Memory management in BigDataBag is probably wrong
> -------------------------------------------------
>
>                 Key: PIG-40
>                 URL: https://issues.apache.org/jira/browse/PIG-40
>             Project: Pig
>          Issue Type: Bug
>          Components: impl
>            Reporter: Sam Pullara
>         Attachments: BigDataBag.java, MemoryUsage.java
>
>
> src/org/apache/pig/data/BigDataBag.java
> 1) You should not use finalizers for things other than external resources -- using them
here is very dangerous and could inadvertantly lead to deadlocks and object resurrection and
just decreases performance without any advantage.
> 2) Using .freeMemory() the way it is used in this class is broken.  freeMemory() is going
to return a mostly random number between 0 and the real amount.  Adding gc() in here is a
terrible performance burden.  If you really want to do something like this you should using
softreferences and finalization queues.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message