hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Florin Diaconeasa (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-1884) Potential risk of resource leaks in Hive
Date Thu, 08 Sep 2011 12:10:14 GMT

    [ https://issues.apache.org/jira/browse/HIVE-1884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13100250#comment-13100250
] 

Florin Diaconeasa commented on HIVE-1884:
-----------------------------------------

Hello,

Could this lead to input files being ignored?

We have a query in which we are doing several "UNION ALL". Apparently, sometimes Hive ignores
one of the SELECTs. Actually not sure if it ignores the SELECT or it doesn't see the input
files for that select. There are 6 queries which are united using UNION ALL.

This happened several times with different SELECTs from that big query and the query i valid.
This leads me to think it's either related to this issue or a memory leak.

Setup: Hadoop 0.20.1, Hive 0.6, Debian 5.0 x64

Thank you,

Flo

> Potential risk of resource leaks in Hive
> ----------------------------------------
>
>                 Key: HIVE-1884
>                 URL: https://issues.apache.org/jira/browse/HIVE-1884
>             Project: Hive
>          Issue Type: Bug
>          Components: CLI, Metastore, Query Processor, Server Infrastructure
>    Affects Versions: 0.3.0, 0.4.0, 0.4.1, 0.5.0, 0.6.0
>         Environment: Hive 0.6.0, Hadoop 0.20.1
> SUSE Linux Enterprise Server 11 (i586)
>            Reporter: Mohit Sikri
>            Assignee: Chinna Rao Lalam
>             Fix For: 0.8.0
>
>         Attachments: HIVE-1884.1.PATCH, HIVE-1884.2.patch, HIVE-1884.3.patch, HIVE-1884.4.patch,
HIVE-1884.5.patch
>
>
> h3.There are couple of resource leaks.
> h4.For example,
> In CliDriver.java, Method :- processReader() the buffered reader is not closed.
> h3.Also there are risk(s) of  resource(s) getting leaked , in such cases we need to re
factor the code to move closing of resources in finally block.
> h4. For Example :- 
> In Throttle.java   Method:- checkJobTracker() , the following code snippet might cause
resource leak.
> {code}
> InputStream in = url.openStream();
> in.read(buffer);
> in.close();
> {code}
> Ideally and as per the best coding practices it should be like below
> {code}
> InputStream in=null;
> try   {
>         in = url.openStream();
>         int numRead = in.read(buffer);
> }
> finally {
>        IOUtils.closeStream(in);
> }
> {code}
> Similar cases, were found in ExplainTask.java, DDLTask.java etc.Need to re factor all
such occurrences.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message