hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Thejas M Nair (JIRA)" <j...@apache.org>
Subject [jira] Created: (PIG-1519) Stop relying on finalize() to delete files, close filehandles in bag implementations
Date Mon, 26 Jul 2010 23:46:16 GMT
Stop relying on finalize() to delete files, close filehandles in bag implementations
------------------------------------------------------------------------------------

                 Key: PIG-1519
                 URL: https://issues.apache.org/jira/browse/PIG-1519
             Project: Pig
          Issue Type: Improvement
    Affects Versions: 0.8.0
            Reporter: Thejas M Nair
            Priority: Minor


In DefaultAbstractBag and its subclasses, the files used for spilling to disk are deleted
using finalize() . 
The iterators associated with these bags use DataInputStreams but don't call close on them,
and the underlying FileInputStream.close() is called only through FileInputStream.finalize().

The use of finalize has performance implications and also makes it hard to predict when the
resources will get freed. 

WeakReferences can be used to avoid the use of finalize().  See http://java.sun.com/developer/technicalArticles/javase/finalization/
(look for "An Alternative to Finalization") .

I have marked the priority has minor because the allocation of these resources objects that
have finalize happens only for large bags that spill to disk (see related jira - PIG-1516),
so the performance  impact of the use of finalize is not likely to be significant. Also, I
haven't come across any case where we have run out of these resources because finalize() thread
has not freed them yet.



-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message