hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From lars hofhansl <la...@apache.org>
Subject Re: Heads up, HTablePool will be deprecated in 0.94, 0.95/0.96, and removed in 0.98
Date Mon, 05 Aug 2013 02:23:38 GMT
Just look at HConnectionKey part, and hoops we go through to detect whether HConnections are
the same or not, when to cache them, when/how to release them.
In fact almost all HConnectionManager does is managing HConnections on behalf of HTable, when
it should be other way around.

Typically, when things get hard to explain (check out the comments in HConnectionManager)
there is either an abstraction missing, or the abstraction is not right.
The reverse (Connections "managing" HTables) has none of this.


-- Lars

_______________________________
From: Ted Yu <yuzhihong@gmail.com>
To: dev@hbase.apache.org; lars hofhansl <larsh@apache.org> 
Sent: Sunday, August 4, 2013 4:27 PM
Subject: Re: Heads up, HTablePool will be deprecated in 0.94, 0.95/0.96, and removed in 0.98



bq. no funny business with unique Configurations

Mind telling us what is funny about this part ?


On Sat, Aug 3, 2013 at 10:41 PM, lars hofhansl <larsh@apache.org> wrote:

Correct. The HConnection is naturally shared between the HTables.
>There is no longer any need to worry about this (no funny business with unique Configurations,
in fact most of the code in HConnectionManager can be removed in trunk).
>
>It is also correct that the code now has to hold on the created HConnection, rather asking
HConnectionManager for it.
>
>-- Lars
>
>
>
>________________________________
> From: Nick Dimiduk <ndimiduk@gmail.com>
>To: dev@hbase.apache.org
>Sent: Saturday, August 3, 2013 8:56 PM
>
>Subject: Re: Heads up, HTablePool will be deprecated in 0.94, 0.95/0.96, and removed in
0.98
>
>
>On Sat, Aug 3, 2013 at 8:52 PM, Ted Yu <yuzhihong@gmail.com> wrote:
>
>> Does this mean that user code wouldn't be able to depend
>> on HConnectionManager for connection sharing ?
>>
>
>My read of the above is that the HConnection instance is shared across
>consumers, is the shared connection. Am I reading that correctly?
>
>On Sat, Aug 3, 2013 at 7:20 AM, Ted Yu <yuzhihong@gmail.com> wrote:
>>
>> > Ah, I find the JIRA - HBASE-9117.
>> >
>> > Cheers
>> >
>> >
>> > On Fri, Aug 2, 2013 at 10:54 PM, lars hofhansl <larsh@apache.org> wrote:
>> >
>> >> Yeah, I filed a separate ticket for the API removal in trunk.
>> >>
>> >>
>> >>
>> >> ________________________________
>> >>  From: Ted Yu <yuzhihong@gmail.com>
>> >> To: dev@hbase.apache.org; lars hofhansl <larsh@apache.org>
>> >> Sent: Friday, August 2, 2013 10:31 PM
>> >> Subject: Re: Heads up, HTablePool will be deprecated in 0.94, 0.95/0.96,
>> >> and removed in 0.98
>> >>
>> >>
>> >> bq. HConnectionManager.getConnection() will be removed.
>> >>
>> >> I don't see the above change in 6580-trunk.txt
>> >> Would the above be done in next patch or in another JIRA ?
>> >>
>> >> Cheers
>> >>
>> >> On Fri, Aug 2, 2013 at 9:29 PM, lars hofhansl <larsh@apache.org> wrote:
>> >>
>> >> > See. https://issues.apache.org/jira/browse/HBASE-6580
>> >> >
>> >> > The new proposed API looks like this:
>> >> >
>> >> > Here's the proposed new API:
>> >> > * HConnectionManager:
>> >> >     public static HConnection createConnection(Configuration conf)
>> >> >     public static HConnection createConnection(Configuration conf,
>> >> > ExecutorService pool)
>> >> >
>> >> > * HConnection:
>> >> >     public HTableInterface getTable(byte[] tableName) throws
>> IOException
>> >> >     public HTableInterface getTable(byte[] tableName, ExecutorService
>> >> > pool) throws IOException
>> >> >     public HTableInterface getTable(String tableName) throws
>> IOException
>> >> >
>> >> > By default HConnectionImplementation will create an ExecutorService
>> when
>> >> > needed. The ExecutorService can optionally passed be passed in.
>> >> > HTableInterfaces are retrieved from the HConnection. By default the
>> >> > HConnection's ExecutorService is used, but optionally that can be
>> >> > overridden for each HTable.
>> >> >
>> >> > In 0.98/trunk:
>> >> >
>> >> > 1. HTablePool will be removed. It is not longer needed.
>> >> > 2. All constructors in HTable will be removed and changed to be
>> >> protected.
>> >> > All code use HTableInterface only.
>> >> > 3. HConnectionManager.getConnection() will be removed.
>> >> > 3. All HConnection caching (deleteConnection, etc,etc) will be
>> removed,
>> >> as
>> >> > it is no longer needed.
>> >> >
>> >> >
>> >> > The new flow of setting up a client would look like this:
>> >> >
>> >> > ----- Snip -----
>> >> > // connection to the cluster
>> >> > HConnection conn = HConnectionManager.createConnection(conf);
>> >> > ...
>> >> > // When the cluster connection is established get an HTableInterface
>> for
>> >> > each operation or thread.
>> >> > // HConnection.getTable(...) is lightweight. The table is really just
>> a
>> >> > convenient place to call table method and for a temporary batch cache.
>> >> > // It is in fact less overhead than HTablePool had when retrieving
a
>> >> > cached HTable.
>> >> > // The HTableInterface returned is not thread safe as before.
>> >> > // It's fine to get 1000's of these.
>> >> > // Don't cache the longer than the lifetime of the HConnection
>> >> > HTableInterface table = conn.getTable("MyTable");
>> >> > ...
>> >> > // just flushes outstanding commit, no futher cleanup needed, can be
>> >> > omitted.
>> >> > // HConnection holds no references to the returned HTable objects,
>> they
>> >> > can be GC'd as soon as they leave scope.
>> >> > table.close();
>> >> > ...
>> >> > conn.close(); // done with the cluster, release resources
>> >> > ----- Snip -----
>> >> >
>> >> > The HConnection will maintain and share its own ThreadPool for all
>> batch
>> >> > operations executed by the HTables.
>> >> > This can overridden per HConnection and/or per individual HTable
>> object.
>> >> >
>> >> > I will commit the new API to all branches early next week.
>> >> >
>> >> > Questions? Comments? Concerns? Praise?
>> >> >
>> >> > -- Lars
>> >>
>> >
>> >
>>

Mime
View raw message