hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mikhail Antonov (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-16117) Fix Connection leak in mapred.TableOutputFormat
Date Fri, 08 Jul 2016 22:25:11 GMT

    [ https://issues.apache.org/jira/browse/HBASE-16117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15368597#comment-15368597
] 

Mikhail Antonov commented on HBASE-16117:
-----------------------------------------

Looking at the patch I have few questions.

3167	      zkw.getRecoverableZooKeeper().getZooKeeper().exists(zkw.baseZNode, false);
3179	      connection.isMasterRunning();	3168	      connection.isMasterRunning();
3169	    } catch (IOException e) {
3170	      throw new ZooKeeperConnectionException("Can't connect to ZooKeeper", e);
3171	    } catch (InterruptedException e) {

So we moved up isMasterRunning() check in the try{} block, this one can throw MNRE which extends
IOE and we would masquerade master not running exception with ZK exception?

Also not sure it's good to add more IOE to Registry interface, those are too ubiquitous already..any
way to avoid that?

> Fix Connection leak in mapred.TableOutputFormat 
> ------------------------------------------------
>
>                 Key: HBASE-16117
>                 URL: https://issues.apache.org/jira/browse/HBASE-16117
>             Project: HBase
>          Issue Type: Bug
>          Components: mapreduce
>    Affects Versions: 2.0.0, 1.3.0, 1.2.2, 1.1.6
>            Reporter: Jonathan Hsieh
>            Assignee: Jonathan Hsieh
>             Fix For: 2.0.0, 1.3.0, 1.1.6, 1.2.3
>
>         Attachments: hbase-16117.branch-1.patch, hbase-16117.patch, hbase-16117.v2.branch-1.patch,
hbase-16117.v2.patch, hbase-16117.v3.branch-1.patch, hbase-16117.v3.patch, hbase-16117.v4.patch
>
>
> Spark seems to instantiate multiple instances of output formats within a single process.
 When mapred.TableOutputFormat (not mapreduce.TableOutputFormat) is used, this may cause connection
leaks that slowly exhaust the cluster's zk connections.  
> This patch fixes that.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message