hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jonathan Hsieh <...@cloudera.com>
Subject Re: [jira] [Created] (HBASE-16117) Fix Connection leak in mapred.TableOutputFormat when spark uses it.
Date Mon, 27 Jun 2016 14:10:56 GMT
Thanks Ted,

This issue came up from a user who upgraded from 0.98 based hbase that
didn't have the leak to a 1.x+ based hbase which does have the leak.  It is
a regression and code hygiene issue so it makes sense to fix this and allow
old code to continue to work.

Jon.

On Sun, Jun 26, 2016 at 5:01 PM, Ted Malaska <ted.malaska@cloudera.com>
wrote:

> If u use bulk put in the spark module it will do the same functionality and
> not have this problem.
>
>
>
> On Sunday, June 26, 2016, Jonathan Hsieh (JIRA) <jira@apache.org> wrote:
>
> > Jonathan Hsieh created HBASE-16117:
> > --------------------------------------
> >
> >              Summary: Fix Connection leak in mapred.TableOutputFormat
> when
> > spark uses it.
> >                  Key: HBASE-16117
> >                  URL: https://issues.apache.org/jira/browse/HBASE-16117
> >              Project: HBase
> >           Issue Type: Bug
> >           Components: mapreduce
> >     Affects Versions: 2.0.0, 1.3.0, 1.2.2, 1.1.6
> >             Reporter: Jonathan Hsieh
> >             Assignee: Jonathan Hsieh
> >              Fix For: 2.0.0, 1.3.0, 1.2.2, 1.1.6
> >
> >
> > Spark seems to instantiate multiple instances of output formats within a
> > single process.  When mapred.TableOutputFormat (not
> > mapreduce.TableOutputFormat) is used, this may cause connection leaks
> that
> > slowly exhaust the cluster's zk connections.
> >
> > This patch fixes that.
> >
> >
> >
> > --
> > This message was sent by Atlassian JIRA
> > (v6.3.4#6332)
> >
>
>
> --
> Sent from Gmail Mobile
>



-- 
// Jonathan Hsieh (shay)
// HBase Tech Lead, Software Engineer, Cloudera
// jon@cloudera.com // @jmhsieh

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message