hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Scott Sitar (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HIVE-5344) DeprecatedLzoTextInputFormat never purges its lzo index cache
Date Mon, 23 Sep 2013 17:53:03 GMT
Scott Sitar created HIVE-5344:
---------------------------------

             Summary: DeprecatedLzoTextInputFormat never purges its lzo index cache
                 Key: HIVE-5344
                 URL: https://issues.apache.org/jira/browse/HIVE-5344
             Project: Hive
          Issue Type: Bug
          Components: File Formats
    Affects Versions: 0.10.0
         Environment: Debian 6, cloudera cdh4.2.0
            Reporter: Scott Sitar


DeprecatedLzoTextInputFormat holds a cache of lzo indexes for every file that it ever reads
(so does LzoTextInputFormat), but this cache can grow in size without bound and is never pruned.

We are running hive queries against lzo-compressed logs, connecting through jdbc and hive-server2.
 HiveInputFormat stores a single instance of DeprecatedLzoTextInputFormat, and will eventually
run out of memory as this cache grows out of control.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message