hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nick Dimiduk <ndimi...@gmail.com>
Subject Re: Declaring HBase Public API in 0.94
Date Tue, 16 Apr 2013 21:14:18 GMT
Aleks,

With the exception of the source paths [0], [1], I don't see anything that
needs changed for 0.94. In fact, those paths look correct for 0.94 but
wrong for trunk. I would have expected to see all of the modules broken
out. Are you sure this generates the reports you expect when run on trunk?

Thanks,
Nick

[0]:
https://github.com/apache/hbase/commit/379f88f0b55b817cfa395326d5da0e3d6552ef28#L0R38
[1]:
https://github.com/apache/hbase/commit/379f88f0b55b817cfa395326d5da0e3d6552ef28#L0R42

On Tue, Apr 16, 2013 at 1:56 PM, Aleksandr Shulman <aleks@cloudera.com>wrote:

> Hello,
>
> I put together a JDiff compatibility tool that was committed to trunk.
> Would someone be kind enough to backport it into 0.94.8?
>
> Here is the commit for the tool:
>
> https://github.com/apache/hbase/commit/379f88f0b55b817cfa395326d5da0e3d6552ef28
>
> Here is the review:
> https://reviews.apache.org/r/10361/
>
> Thanks in advance,
>
> Aleks S.
>
> On Tue, Apr 9, 2013 at 10:47 AM, Aleksandr Shulman <aleks@cloudera.com
> >wrote:
>
> > I put together a tool that leverages Tom White's work.
> > Here is a review for it: https://reviews.apache.org/r/10361/
> >
> > It will diff the public java api definitions of hbase in two git repos
> and
> > generate an HTML report. The tool will reside in the /dev-support folder.
> > The documentation is inline in the file.
> >
> > I'd appreciate your input in how I can make it more useful and usable for
> > us.
> >
> > Once we agree on the definitions of what classes are indeed public, we
> can
> > fine-tune this tool to ignore everything else.
> >
> > -Aleks S.
> >
> >
> > On Mon, Apr 8, 2013 at 3:55 PM, Todd Lipcon <todd@cloudera.com> wrote:
> >
> >> The advantage of the annotations is that Tom White already did the work
> >> for
> >> jdiff to ignore non-public classes over in Hadoop land. We could
> leverage
> >> that work, whether we re-use the o.a.h.classification annotations or add
> >> our own copies in org.apache.hbase.*.
> >>
> >> -Todd
> >>
> >> On Mon, Apr 8, 2013 at 3:08 PM, lars hofhansl <larsh@apache.org> wrote:
> >>
> >> > It seems we could just generally document that:
> >> > - no RPC incompatibilities
> >> > - no API breaking changes to any user facing classes (now we'll pay
> >> better
> >> > attention to this)
> >> > - best effort to keep coprocessor API changes backward compatible
> >> >
> >> > If - on the other hand - we wanted to automate API checks then we'd
> need
> >> > tagging (either in form of an annotation or Javadoc)
> >> >
> >> > +1 on the javadoc tagging if you're willing to take than on. As other
> >> have
> >> > said -1 on pulling Interface Audience in.
> >> > Your set of classes looks good.
> >> >
> >> > -- Lars
> >> >
> >> >
> >> >
> >> > ________________________________
> >> >  From: Elliott Clark <eclark@apache.org>
> >> > To: "dev@hbase.apache.org" <dev@hbase.apache.org>
> >> > Sent: Monday, April 8, 2013 1:49 PM
> >> > Subject: Re: Declaring HBase Public API in 0.94
> >> >
> >> > Please don't pull in @InterfaceAudience.  Keeping 0.2x compatibility
> was
> >> > something that was hard won in 0.94, it would be a real shame to loose
> >> that
> >> > now.
> >> >
> >> >
> >> > On Mon, Apr 8, 2013 at 11:07 AM, Aleksandr Shulman <
> aleks@cloudera.com
> >> > >wrote:
> >> >
> >> > > Hi everyone,
> >> > >
> >> > > In light of all the conversation on compatibility, I wanted to float
> >> the
> >> > > idea of documenting which Java packages, classes, and methods we
> want
> >> to
> >> > > declare as being API compatible in 0.94.x. I'd like your input on:
> >> > > 1. JavaDoc vs. using AudienceInterface
> >> > > 2. What the javadoc notation should look like
> >> > > 3. Which pieces of code should be tagged
> >> > >
> >> > > What do I mean by documenting API compatibility? That means that we
> >> > suggest
> >> > > the anyone building applications use specific methods because they
> >> would
> >> > > continue to be both binary and RPC-compatible going forward. Any
> >> > > application written, either running on a node of a cluster or on a
> >> remote
> >> > > machine, would continue to work properly without recompile for all
> >> > versions
> >> > > of 0.94.x running on the cluster.
> >> > >
> >> > > *Benefits:*
> >> > > It would prevent developers from using calls that are subject to
> >> change.
> >> > > This would give developers more confidence in using the platform,
> >> which
> >> > > will encourage more development on our platform.
> >> > > 0.94 will still be with us for some time and I think the
> >> > > better-late-than-never approach will save us pain down the road.
> >> Finally,
> >> > > it would allow us to more easily verify that we are in fact API
> >> > compatible.
> >> > >
> >> > > *Can we use AudienceInterface?*
> >> > > HBase 0.94 can be compiled against both hadoop 0.2x, 1.x, and 2.0.x.
> >> In
> >> > the
> >> > > case of 0.2x, the AudienceInterface classes were not bundled.
> >> Therefore,
> >> > we
> >> > > cannot expect HBase 0.94 to support it. For that reason, I think
> >> JavaDoc
> >> > > might be better.
> >> > > On the other hand, perhaps we might just want to bundle
> >> AudienceInterface
> >> > > with 0.94 going forward? Then we can have consistent annotations in
> >> 0.94,
> >> > > 0.95, and 0.96 without worrying about the hadoop version.
> >> > >
> >> > > Please correct me if I'm wrong about any of the above.
> >> > >
> >> > > *Clarification of RPC compatibility:*
> >> > > We care about RPC compatibility when we create clients that bundle
> >> their
> >> > > dependency jars with them. These jars are used to form a request
> that
> >> is
> >> > > executed on a remote machine (i.e. the cluster). If the cluster is
> >> > upgraded
> >> > > and no longer recognizes the command, then this will break RPC
> >> > > compatibility.
> >> > >
> >> > > *Clarification of Binary compatibility:*
> >> > > We care about binary compatibility when a client is created and
> >> compiled,
> >> > > and the jars on which is depends change. It should still be able to
> >> form
> >> > > requests using those jars. If the cluster is upgraded and the
> compiled
> >> > > client code cannot find a method it was depending on to be there,
we
> >> > break
> >> > > binary compatibility. A recent example is in 0.94.2, where the
> return
> >> > type
> >> > > of HColumnDescriptor.setMaximumVersions was changed and those who
> >> > upgraded
> >> > > received this error:
> >> > >
> >> > >     java.lang.NoSuchMethodError: org.apache.hadoop.hbase.**
> >> > > HColumnDescriptor.**setMaxVersions(I)V
> >> > >
> >> > > *What we currently have:*
> >> > > We have an @audience annotation set up in 0.95/0.96. In 0.94, I
> >> suggest
> >> > > either adding JavaDoc or pulling in the AudienceInterface
> annotation.
> >> > >
> >> > > *Suggested Javadoc language:*
> >> > > @custom.94_api
> >> > >
> >> > > *Granularity:*
> >> > > Just to the class level. The native java access level (e.g. public,
> >> > > protected, etc.) should indicate what should be kept compatible.
> >> > >
> >> > > *Suggested classes:*
> >> > > Here is a first-cut of things that should be declared and documented
> >> as
> >> > > public APIs. This list was obtained from looking at some MapReduce
> >> over
> >> > > HBase example code.
> >> > >
> >> > > *JAVA API:*
> >> > >
> >> > > *org.apache.hadoop.hbase (some selected classes, see below)
> >> > > org.apache.hadoop.hbase.client.*
> >> > > org.apache.hadoop.hbase.filter.*
> >> > > org.apache.hadoop.hbase.io.hfile.Compression.Algorithm
> >> > > org.apache.hadoop.hbase.util.*
> >> > > org.apache.hadoop.hbase.mapreduce.**
> >> > >
> >> > > *REST API:
> >> > > org.apache.hadoop.hbase.rest.client.**
> >> > >
> >> > > *Thrift API:
> >> > > All methods defined in:
> >> > >
> /hbase/src/main/resources/org/apache/hadoop/hbase/thrift/Hbase.thrift*
> >> > >
> >> > > *Selected classes in org.apache.hadoop.hbase:*
> >> > >
> >> > > *import org.apache.hadoop.hbase.ClusterStatus;
> >> > > import org.apache.hadoop.hbase.HBaseConfiguration;
> >> > > import org.apache.hadoop.hbase.HColumnDescriptor;
> >> > > import org.apache.hadoop.hbase.HRegionInfo;
> >> > > import org.apache.hadoop.hbase.HRegionLocation;
> >> > > import org.apache.hadoop.hbase.HServerAddress;
> >> > > import org.apache.hadoop.hbase.HTableDescriptor;
> >> > > import org.apache.hadoop.hbase.KeyValue;*
> >> > >
> >> > > --
> >> > > Best Regards,
> >> > >
> >> > > Aleks Shulman
> >> > > 847.814.5804
> >> > > Cloudera
> >> > >
> >> >
> >>
> >>
> >>
> >> --
> >> Todd Lipcon
> >> Software Engineer, Cloudera
> >>
> >
> >
> >
> > --
> > Best Regards,
> >
> > Aleks Shulman
> > 847.814.5804
> > Cloudera
> >
>
>
>
> --
> Best Regards,
>
> Aleks Shulman
> 847.814.5804
> Cloudera
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message