incubator-crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matthias Friedrich <m...@mafr.de>
Subject Re: Moving PType and friends
Date Sat, 27 Oct 2012 21:31:06 GMT
On Saturday, 2012-10-27, Gabriel Reid wrote:
> On 27 Oct 2012, at 18:06, Matthias Friedrich <matt@mafr.de> wrote:
 
>> we have made some progress cleaning up javadocs and some packages, but
>> there are a few more things I'd like to do. Right now, not all
>> abstractions are in the base package, which means they won't be on the
>> same javadoc package page and users won't recognize them as core
>> concepts.
 
>> Thus, I propose to move the PType, PTypeFamily, PTableType, Converter,
>> and OutputHandler types to the base package. This would have the
>> advantage that the .types and .io packages would become pure
>> implementation packages, breaking package dependency cycles. We could
>> then hide the .types package from the javadoc overview page because
>> nothing there would be client-facing anymore.
 
> I'm on board with the moving of PType, PTypeFamily, and PTableType classes
> to the base package (although it seems a shame because the package name
> that they're in already got changed once).

Hmm, yes, but fortunately it's a change that is solved by "Organize
Imports". I think it's better to do API changes early in a project's
lifetime than to stay backwards compatible until everybody's sick of
the API and you have to make a huge incompatible change like it
happened with Hadoop.

> However, I'm less sure about moving Converter and OutputHandler, as
> I don't really see these as core (external) abstractions. It kind of
> feels like moving those will just clutter up the base package.

Yeah, Converter is collateral damage because after moving the other
types they will be part of the base package's referential transitive
closure. And OutputHandler causes the cycle between base and .io.

Things are coupled there, do you have an idea on how to untangle this?
If we don't move them we'll have partially cleaned up the javadocs but
I don't see how we can cut the dependency cycles.

BTW, I don't think the base package will be cluttered. Right now we're
at 22 top-level types, which is fewer than most Guava packages (my
benchmark in most things). If we're really serious about reducing
clutter then the CombineFn implementations would be the right point to
start [1]. Only the CombineFn type itself needs to be in base,
everything else could very well live in .fn, but I wasn't planning on
doing that just yet. Small steps, like we agreed.
 
Regards,
  Matthias

[1] http://tmp.mafr.de/crunch/apidocs/0.4.0-incubating-SNAPSHOT/org/apache/crunch/package-summary.html

Mime
View raw message