commons-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sandy McArthur" <sandy...@apache.org>
Subject Re: [pool] why the composite pool implementation isn't plugable [was: picking descriptive class names]
Date Mon, 27 Mar 2006 09:38:59 GMT
On 3/27/06, Rahul Akolkar <rahul.akolkar@gmail.com> wrote:
> On 3/26/06, Sandy McArthur <sandymac@apache.org> wrote:
> > On 3/25/06, Rahul Akolkar <rahul.akolkar@gmail.com> wrote:
> > > On 3/25/06, Sandy McArthur <sandymac@apache.org> wrote:
> > > <snip/>
> > > > The enums above don't actually specify any implementation, they
> > > > describe desired features of a pool. The actual implementation isn't
> > > > broken down into four parts like that so try not to confuse how you
> > > > would implement that feature with how you would request that feature.
> > > >
> > > <snap/>
> > >
> > > Why isn't it broken down like that?
> >
> > Because there are fundamentally three parts to a pool's behavior.
> > 1. How objects are treated while they are in the idle object pool.
> > 2. How objects are added/removed from the idle object pool.
> > 3. How objects are treated while they are out of the pool, aka: active.
> >
> > I choose to map these three aspects to four types of behavior because
> <snip/>
> > The composite pool is already plugable, you must give it a
> > PoolableObjectFactory. :-)
> <snap/>
>
> Isn't the PoolableObjectFactory orthogonal to the four enum types you
> mention? Those tune the FactoryConfig?

Yes, hence the smiley.

> > Not everything is made better because it's made plugable (or
> <snip/>
> > I'm not against plugable APIs. They often make sense but not always,
> > and this is one time they don't. I also want the composite pool code
> > to be "future proof".
> <snap/>
> > With a fully plugable API the synchronization optimization above
> > wouldn't really be available.
> <snip/>
>
> No silver bullet, we can only guarantee what is provided out of the
> box. All the internal optimizations you list may be appealing, and
> what I take from your post is that the optimizations rely on fact that
> the four "behavior" impls will always be one of the enum'ed ones. The
> question that I still cannot address is -- what is the implication of
> needing a enum'ed behavior that is not provided? Maybe that is what
> really makes this "future proof". I don't think we know.

The implication for a programmer wanting a feature that isn't
expressible with the enums is one of:
* the programmer customizes one of the other existing ObjectPool
implementations available to him already.
* the programmer uses his ASL given right to customize and enhance the
source to meet his needs. (If we're lucky he'll submit a patch back to
us.)

Neither of those are terrible, end of the world implications.
Personally I think the second one is pretty good.

> > Peter Steijn who emailed a week ago is exploring
> > some new ways to improve the performance of Pool (and by extension
> > Dbcp) by using some more complex threading behaviors. We've discussed
> > some of his ideas off-list and provided his ideas pan out maybe the
> > composite pool code in Pool 2.1 will be faster and client code using
> > pool won't have to know or care how the improved performance came
> > about.
> >
> <snap/>
>
> 2.1 because we'd rather get 2.0 out first, instead of waiting to try
> out the ideas? If you guys think its appropriate, you might even move
> that conversation here? Maybe others are interested as well; I am :-)
> Plus it will give us better background while reading the commit
> messages.

Peter Steijn initiated the private email to me. It's up to him if he
wants to discuss it on list. I will let him talk about his ideas for
his thesis to whomever he chooses.

In regards to the the commit message mentioning Peter, I just wanted
to give him credit for triggering the chain of thought that led me to
take an idea I had for optimizing destroyObject (recently committed)
and applying it to makeObject (also recently committed).

> > > Since we're talking about Pool 2.0 and beyond, perhaps a focus on
> > > similar extensibility is justified, and maybe we should revisit the
> > > enumeration approach, even before we get to names.
> >
> > If you want to implement a more plugable pool implementation and put
> > the plugable pool and composite pool code in a steel cage match to the
> > death based on usability, flexibility, and performance I'm all for it.
> >
> <snip/>
>
> I think this has value. In terms of a major version release, we might
> want to use some of liberties we get the best we can, and to that end,
> playing with multiple options can be beneficial. If I can make some
> progress on the things that are already on my plate within the next
> few months, I'll play in a new branch (and in that case, I'll ping the
> list for objections first). However I am hardly talking about a
> complete rewrite, so it should have the same usability and performance
> (for out-of-the-box impls) since I'll base it off of your
> contribution, just to be more flexible on the enums front. Maybe that
> puts it in a higher weight class already? ;-)

Go for it, but when designing a good API it's just as important to
consider what you are leaving out.

For example Gary Gregory recently requested that isClosed() in
BaseObjectPool be made public but that isn't really what he wanted.
What he wanted was for the pool not to throw an exception when you
call close a second time or when you call returnObject after close has
been called and he'll get this with behavior with pool 2.

Making isClosed public like he requested seems like it would have met
his needs as a quick and easy fix but it wouldn't really work. Not
only is it harder to use because now he'd have to call isClosed to
guard each access to the pool to avoid exceptions but it wouldn't work
because between the call to isClosed and returnObject the pool could
have been closed by another thread. To fix that he'd have to
synchronize (hurting pool performance), check isClosed, and then call
returnObject.

GenericObjectPool really suffers from naively tacking on features
without an eye to usability or solving the real problem. A FIFO pool
doesn't need an idle object evictor, all idle objects get touched over
time during normal usage. GOP has an evictor because people noticed
old idle objects never being removed from the pool. This is because
GOP was actually a LIFO and during heavy usage the pool may grow large
and until the next heavy usage the deepest idle object won't be
touched at all.

When the composite pool is first release I fully expect the first
feature request to be a naive request for a minIdle configuration
option. It was a completely intentional decision that the composite
pool code does not have a minIdle feature. Adding minIdle only fixes
the symptom of making new objects being slow thus slowing the pool.
What should be addressed is how the pool can remain fast despite slow
operations. I didn't know how to solve this back in November 2005 when
I wrote the composite pool code but Peter has some ideas and he help
me get an idea. The end result is/will be a pool that needs one less
configuration option and performs optimally.

> P.S.- [pool] code is quite hard to read with all that horizontal
> scrolling. Irrespective of the code already in place, maybe we should
> stick to a reasonable (80?) character line width for new code?

The code I contribute to apache is code I wrote for pleasure. The code
I contribute is in the form that was most pleasurable for me to write
in. I impose no restrictions on how others choose to write their code.
If you wish to compensate me to write code differently or reject my
contributions because of such trivial issues, that is fine. The ASL
grants anyone the right reformat ASL licensed code however they see
fit. I only request that I am not stripped of attribution for my
contributions.
--
Sandy McArthur

"He who dares not offend cannot be honest."
- Thomas Paine

---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org


Mime
View raw message