drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jacques Nadeau <jacq...@dremio.com>
Subject Re: Proposal: Create v2 branch to work on breaking changes
Date Thu, 14 Apr 2016 01:29:54 GMT
A number of the vector problems are an issue for the Java client but not
the c client since the c client doesn't support complex or union types yet.

Agree on your other points:

- We need to get to rolling upgrades. (Note that I suggest that we try to
get to "minor" compatibility for Drillbit <--> Drillbit by the end of the
2.x series in my notes)
- Also agree that we should always work to avoid changing any of the
interfaces described in the doc, no matter what the external commitment is.
The performance analog is a good one.



--
Jacques Nadeau
CTO and Co-Founder, Dremio

On Wed, Apr 13, 2016 at 5:54 PM, Parth Chandra <parthc@apache.org> wrote:

> Thanks for putting this doc together Jacques. This gives us a clear
> framework for discussion.
> Just as a clarification (I haven't yet been able to do more than glance at
> the doc), for 2.0, I was suggesting client-server compatibility not
> drillbit-drillbit compatibility. It seems some of the items you noted
> earlier (null lists, union vectors, etc.) may break drillbit-drillbit
> compatibility but not necessarily affect client-server compatibility. So we
> may be in agreement on some things here.
> In general, though, as the size of user's clusters grows, it will be
> required that we permit rolling upgrades. As I said in the hangout, it's
> like performance, we have to consider it at every instance; and take a
> decision to not support backward compatibility only after due
> consideration. At the moment, some of the functionality we are talking
> about might justify breaking drillbit-drillbit compatibility. Our design
> decisions for these implementations, though, must keep the requirement for
> future backward compatibility in mind.
> I'll add further comments in the JIRA.
>
> On Tue, Apr 12, 2016 at 6:47 PM, Jacques Nadeau <jacques@dremio.com>
> wrote:
>
> > A general policy shouldn't hold up a specific decision. Even after we
> > establish a guiding policy, there will be exceptions that we will
> consider.
> > I'm looking for concrete counterpoint to the cost of maintaining
> backwards
> > compatibility.
> >
> > That being said, I have put together an initial proposal of the
> > compatibility commitments we should make to the users. It is important to
> > note that my outline is about our public commitment. As a development
> > community, we should always work to avoid disruptive or backwards
> > incompatible changes on public apis even if the our public commitment
> > policy doesn't dictate it.
> >
> > The proposal is attached here:
> > https://issues.apache.org/jira/browse/DRILL-4600
> >
> >
> > --
> > Jacques Nadeau
> > CTO and Co-Founder, Dremio
> >
> > On Tue, Apr 12, 2016 at 5:54 PM, Neeraja Rentachintala <
> > nrentachintala@maprtech.com> wrote:
> >
> > > Makes sense to postpone the debate : )
> > > Will Look forward for the proposal.
> > >
> > > On Tuesday, April 12, 2016, Zelaine Fong <zfong@maprtech.com> wrote:
> > >
> > > > As we discussed at this morning's hangout, Jacques took the action to
> > put
> > > > together a strawman compatibility points document.  Would it be
> better
> > to
> > > > wait for that document before we debate this further?
> > > >
> > > > -- Zelaine
> > > >
> > > > On Tue, Apr 12, 2016 at 4:39 PM, Jacques Nadeau <jacques@dremio.com
> > > > <javascript:;>> wrote:
> > > >
> > > > > I agree with Paul, too. Perfect compatibility would be great. I
> > > recognize
> > > > > the issues that a version break could cause.  These are some of the
> > > > issues
> > > > > that I believe require a version break to address:
> > > > > - Support nulls in lists.
> > > > > - Distinguish null maps from empty maps.
> > > > > - Distinguish null arrays from empty arrays.
> > > > > - Support sparse maps (analogous to Parquet maps instead of our
> > current
> > > > > approach analogous to structs in Parquet lingo).
> > > > > - Clean up decimal and enable it by default.
> > > > > - Support full Avro <> Parquet roundtrip (and Parquet files
> generated
> > > by
> > > > > other tools).
> > > > > - Enable union type by default.
> > > > > - Improve performance execution performance of nullable values.
> > > > >
> > > > > I think these things need to be addressed in the 2.x line (let's
> say
> > > that
> > > > > is ~12 months). This is all about tradeoffs which is why I keep
> > asking
> > > > > people to provide concrete impact. If you think at least one of
> these
> > > > > should be resolved, you're arguing for breaking wire compatibility
> > > > between
> > > > > 1.x and 2.x.
> > > > >
> > > > > So let's get concrete:
> > > > >
> > > > > - How many users are running multiple clusters and using a single
> > > client
> > > > to
> > > > > connect them?
> > > > > - What BI tools are most users using? What is the primary driver
> they
> > > are
> > > > > using?
> > > > > - What BI tools are packaging a Drill driver? If any, what is the
> > > update
> > > > > process and lead time?
> > > > > - How many users are skipping multiple Drill versions (e.g. going
> > from
> > > > 1.2
> > > > > to 1.6)? (Beyond the MapR tick-tock pattern)
> > > > > - How many users are delaying driver upgrade substantially? Are
> there
> > > > > customers using the 1.0 driver?
> > > > > - What is the average number of deployed clients per Drillbit
> > cluster?
> > > > >
> > > > > These are some of the things that need to be evaluated to determine
> > > > whether
> > > > > we choose to implement a compatibility layer or simply make a full
> > > break.
> > > > > (And in reality, I'm not sure we have the resources to build and
> > carry
> > > a
> > > > > complex compatibility layer for these changes.)
> > > > >
> > > > > Whatever the policy we agree upon for future commitments to the
> user
> > > > base,
> > > > > we're in a situation where there are very important reasons to move
> > the
> > > > > codebase forward and change the wire protocol for 2.x.
> > > > >
> > > > > I think it is noble to strive towards backwards compatibility. We
> > > should
> > > > > always do this. However, I also think that--especially early in a
> > > > product's
> > > > > life--it is better to resolve technical debt issues and break a few
> > > eggs
> > > > > than defer and carry a bunch of extra code around.
> > > > >
> > > > > Yes, it can suck for users. Luckily, we should also be giving
> users a
> > > > bunch
> > > > > of positive reasons that it is worth upgrading and dealing with
> this
> > > > > version break. These include better perf, better compatibility with
> > > other
> > > > > tools, union type support, faster bi tool behaviors and a number
of
> > > other
> > > > > things.
> > > > >
> > > > > I for one vote for moving forward and making sure that the 2.x
> branch
> > > is
> > > > > the highest quality and best version of Drill yet rather than
> > focusing
> > > on
> > > > > minimizing the upgrade cost. All upgrades are a cost/benefit
> > analysis.
> > > > > Drill is too young to focus on only minimizing the cost. We should
> be
> > > > > working to make sure the other part of the equation (benefit) is
> > where
> > > > > we're spending the vast majority of our time.
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Jacques Nadeau
> > > > > CTO and Co-Founder, Dremio
> > > > >
> > > > > On Tue, Apr 12, 2016 at 3:38 PM, Neeraja Rentachintala <
> > > > > nrentachintala@maprtech.com <javascript:;>> wrote:
> > > > >
> > > > > > I agree with Paul. Great points.
> > > > > > I would also add the partners aspect to it. Majority of Drill
> users
> > > use
> > > > > it
> > > > > > in conjunction with a BI tool.
> > > > > >
> > > > > >
> > > > > > -Neeraja
> > > > > >
> > > > > > On Tue, Apr 12, 2016 at 3:34 PM, Paul Rogers <
> progers@maprtech.com
> > > > <javascript:;>>
> > > > > wrote:
> > > > > >
> > > > > > > Hi Jacques,
> > > > > > >
> > > > > > > My two cents…
> > > > > > >
> > > > > > > The unfortunate reality is that enterprise customers move
> slowly.
> > > > There
> > > > > > is
> > > > > > > a delay in the time it takes for end users to upgrade to
a new
> > > > release.
> > > > > > > When a third-party tool must also upgrade, the delay becomes
> even
> > > > > longer.
> > > > > > >
> > > > > > > At a high level, we need to provide a window of time in
which
> > > old/new
> > > > > > > clients work with old/new servers. I may have a 1.6 client.
The
> > > > cluster
> > > > > > > upgrades to 1.8. I need time to upgrade my client to 1.8
—
> > > especially
> > > > > if
> > > > > > I
> > > > > > > have to wait for the vendor to provide a new package.
> > > > > > >
> > > > > > > If I connect to two clusters, I may upgrade my client to
1.8
> for
> > > one,
> > > > > but
> > > > > > > I still need to connect to 1.6 for the other if they upgrade
on
> > > > > different
> > > > > > > schedules.
> > > > > > >
> > > > > > > This is exactly why we need to figure out a policy: how
do we
> > give
> > > > > users
> > > > > > a
> > > > > > > sufficient window of time to complete upgrades, even across
the
> > > > 1.x/2.x
> > > > > > > boundary?
> > > > > > >
> > > > > > > The cost of not providing such a window? Broken production
> > systems,
> > > > > > > unpleasant escalations and unhappy customers.
> > > > > > >
> > > > > > > Thanks,
> > > > > > >
> > > > > > > - Paul
> > > > > > >
> > > > > > > > On Apr 12, 2016, at 3:14 PM, Jacques Nadeau <
> > jacques@dremio.com
> > > > <javascript:;>>
> > > > > > wrote:
> > > > > > > >
> > > > > > > >>> What I am suggesting is that we need to maintain
backward
> > > > > > > compatibility with
> > > > > > > > a defined set of 1.x version clients when Drill 2.0
version
> is
> > > out.
> > > > > > > >
> > > > > > > > I'm asking you to be concrete on why. There is definitely
a
> > cost
> > > to
> > > > > > > > maintaining this compatibility. What are the real
costs if we
> > > > don't?
> > > > > > > >
> > > > > > > > --
> > > > > > > > Jacques Nadeau
> > > > > > > > CTO and Co-Founder, Dremio
> > > > > > > >
> > > > > > > > On Wed, Apr 6, 2016 at 9:21 AM, Neeraja Rentachintala
<
> > > > > > > > nrentachintala@maprtech.com <javascript:;>>
wrote:
> > > > > > > >
> > > > > > > >> Jacques
> > > > > > > >> can you elaborate on what you mean by 'internal'
> > implementation
> > > > > > changes
> > > > > > > but
> > > > > > > >> maintain external API.
> > > > > > > >> I thought that changes that are being discussed
here are the
> > > Drill
> > > > > > > client
> > > > > > > >> library changes.
> > > > > > > >> What I am suggesting is that we need to maintain
backward
> > > > > > compatibility
> > > > > > > >> with a defined set of 1.x version clients when
Drill 2.0
> > version
> > > > is
> > > > > > out.
> > > > > > > >>
> > > > > > > >> Neeraja
> > > > > > > >>
> > > > > > > >> On Tue, Apr 5, 2016 at 12:12 PM, Jacques Nadeau
<
> > > > jacques@dremio.com <javascript:;>
> > > > > >
> > > > > > > >> wrote:
> > > > > > > >>
> > > > > > > >>> Thanks for bringing this up. BI compatibility
is super
> > > important.
> > > > > > > >>>
> > > > > > > >>> The discussions here are primarily about internal
> > > implementation
> > > > > > > changes
> > > > > > > >> as
> > > > > > > >>> opposed to external API changes. From a BI
perspective, I
> > think
> > > > > > (hope)
> > > > > > > >>> everyone shares the goal of having zero (to
minimal)
> changes
> > in
> > > > > terms
> > > > > > > of
> > > > > > > >>> ODBC and JDBC behaviors in v2. The items outlined
in
> > DRILL-4417
> > > > are
> > > > > > > also
> > > > > > > >>> critical to strong BI adoption as numerous
patterns right
> now
> > > are
> > > > > > > >>> suboptimal and we need to get them improved.
> > > > > > > >>>
> > > > > > > >>> In terms of your request of the community,
it makes sense
> to
> > > > have a
> > > > > > > >>> strategy around this. It sounds like you have
a bunch of
> > > > > > considerations
> > > > > > > >>> that should be weighed but your presentation
doesn't
> actually
> > > > share
> > > > > > > what
> > > > > > > >>> the concrete details. To date, there has been
no formal
> > > consensus
> > > > > or
> > > > > > > >>> commitment to any particular compatibility
behavior. We've
> > had
> > > an
> > > > > > > >> informal
> > > > > > > >>> "don't change wire compatibility within a
major version".
> If
> > we
> > > > are
> > > > > > > going
> > > > > > > >>> to have a rich dialog about pros and cons
of different
> > > > approaches,
> > > > > we
> > > > > > > >> need
> > > > > > > >>> to make sure that everybody has the same understanding
of
> the
> > > > > > dynamics.
> > > > > > > >> For
> > > > > > > >>> example:
> > > > > > > >>>
> > > > > > > >>> Are you saying that someone has packaged the
Apache Drill
> > > drivers
> > > > > in
> > > > > > > >> their
> > > > > > > >>> BI solution? If so, what version? Is this
the Apache
> release
> > > > > artifact
> > > > > > > or
> > > > > > > >> a
> > > > > > > >>> custom version? Has someone certified them?
Did anyone
> > commit a
> > > > > > > >> particular
> > > > > > > >>> compatibility pattern to a BI vendor on behalf
of the
> > > community?
> > > > > > > >>>
> > > > > > > >>> To date, I'm not aware of any of these types
of decisions
> > being
> > > > > > > discussed
> > > > > > > >>> in the community so it is hard to evaluate
how important
> they
> > > are
> > > > > > > versus
> > > > > > > >>> other things. Knowing that DRILL-4417 is outstanding
and
> > > critical
> > > > > to
> > > > > > > the
> > > > > > > >>> best BI experience, I think we should be very
cautious of
> > > > requiring
> > > > > > > >>> long-term support of the existing (internal)
> implementation.
> > > > > > > Guaranteeing
> > > > > > > >>> ODBC and JDBC behaviors should be satisfactory
for the vast
> > > > > majority
> > > > > > of
> > > > > > > >>> situations. Anything beyond this needs to
have a very
> public
> > > > > > > cost/benefit
> > > > > > > >>> tradeoff. In other words, please expose your
thinking 100x
> > more
> > > > so
> > > > > > that
> > > > > > > >> we
> > > > > > > >>> can all understand the ramifications of different
> strategies.
> > > > > > > >>>
> > > > > > > >>> thanks!
> > > > > > > >>> Jacques
> > > > > > > >>>
> > > > > > > >>>
> > > > > > > >>>
> > > > > > > >>>
> > > > > > > >>>
> > > > > > > >>> --
> > > > > > > >>> Jacques Nadeau
> > > > > > > >>> CTO and Co-Founder, Dremio
> > > > > > > >>>
> > > > > > > >>> On Tue, Apr 5, 2016 at 10:01 AM, Neeraja Rentachintala
<
> > > > > > > >>> nrentachintala@maprtech.com <javascript:;>>
wrote:
> > > > > > > >>>
> > > > > > > >>>> Sorry for coming back to this thread late.
> > > > > > > >>>> I have some feedback on the compatibility
aspects of 2.0.
> > > > > > > >>>>
> > > > > > > >>>> We are working with a variety of BI vendors
to certify
> Drill
> > > and
> > > > > > > >> provide
> > > > > > > >>>> native connectors for Drill. Having native
access from BI
> > > tools
> > > > > > helps
> > > > > > > >>> with
> > > > > > > >>>> seamless experience for the users with
performance and
> > > > > > functionality.
> > > > > > > >>> This
> > > > > > > >>>> work is in progress and they are (and
will be) working
> with
> > > 1.x
> > > > > > > >> versions
> > > > > > > >>> of
> > > > > > > >>>> Drill as part of the development because
thats what we
> have
> > > now.
> > > > > > Some
> > > > > > > >> of
> > > > > > > >>>> these connectors will be available before
2.0 and some of
> > them
> > > > can
> > > > > > > come
> > > > > > > >>> in
> > > > > > > >>>> post 2.0 as certification is a long process.
We don't want
> > to
> > > be
> > > > > in
> > > > > > a
> > > > > > > >>>> situation where the native connectors
are just released by
> > > > certain
> > > > > > BI
> > > > > > > >>>> vendor and the connector is immediately
obsolete or
> doesn't
> > > work
> > > > > > > >> because
> > > > > > > >>> we
> > > > > > > >>>> have 2.0 release out now.
> > > > > > > >>>> So the general requirement should be that
we maintain
> > backward
> > > > > > > >>>> compatibility with certain number of prior
releases. This
> is
> > > > very
> > > > > > > >>> important
> > > > > > > >>>> for the success of the project and adoption
by eco
> system. I
> > > am
> > > > > > happy
> > > > > > > >> to
> > > > > > > >>>> discuss further.
> > > > > > > >>>>
> > > > > > > >>>> -Neeraja
> > > > > > > >>>>
> > > > > > > >>>> On Tue, Apr 5, 2016 at 8:44 AM, Jacques
Nadeau <
> > > > > jacques@dremio.com <javascript:;>>
> > > > > > > >>> wrote:
> > > > > > > >>>>
> > > > > > > >>>>> I'm going to take this as lazy consensus.
I'll create the
> > > > branch.
> > > > > > > >>>>>
> > > > > > > >>>>> Once created, all merges to the master
(1.x branch)
> should
> > > also
> > > > > go
> > > > > > to
> > > > > > > >>> the
> > > > > > > >>>>> v2 branch unless we have a discussion
here that they
> aren't
> > > > > > > >> applicable.
> > > > > > > >>>>> When committing, please make sure
to commit to both
> > > locations.
> > > > > > > >>>>>
> > > > > > > >>>>> thanks,
> > > > > > > >>>>> Jacques
> > > > > > > >>>>>
> > > > > > > >>>>>
> > > > > > > >>>>> --
> > > > > > > >>>>> Jacques Nadeau
> > > > > > > >>>>> CTO and Co-Founder, Dremio
> > > > > > > >>>>>
> > > > > > > >>>>> On Sat, Mar 26, 2016 at 7:26 PM, Jacques
Nadeau <
> > > > > > jacques@dremio.com <javascript:;>>
> > > > > > > >>>>> wrote:
> > > > > > > >>>>>
> > > > > > > >>>>>> Re Compatibility:
> > > > > > > >>>>>>
> > > > > > > >>>>>> I actually don't even think 1.0
clients work with 1.6
> > > server,
> > > > do
> > > > > > > >>> they?
> > > > > > > >>>>>>
> > > > > > > >>>>>> I would probably decrease the
cross-compatibility
> > > requirement
> > > > > > > >>> burden. A
> > > > > > > >>>>>> nice goal would be cross compatibility
across an
> extended
> > > > series
> > > > > > of
> > > > > > > >>>>>> releases. However, given all the
things we've learned in
> > the
> > > > > last
> > > > > > > >>> year,
> > > > > > > >>>>> we
> > > > > > > >>>>>> shouldn't try to maintain more
legacy than is necessary.
> > As
> > > > > such,
> > > > > > I
> > > > > > > >>>>> propose
> > > > > > > >>>>>> that we consider the requirement
of 2.0 to be:
> > > > > > > >>>>>>
> > > > > > > >>>>>> 1.lastX works with 2.firstX. (For
example, if 1.8 is the
> > > last
> > > > > > minor
> > > > > > > >>>>>> release of the 1.x series, 1.8
would work with 2.0.)
> > > > > > > >>>>>>
> > > > > > > >>>>>> This simplifies testing (we don't
have to worry about
> > things
> > > > > like
> > > > > > > >>> does
> > > > > > > >>>>> 1.1
> > > > > > > >>>>>> work with 2.3, etc) and gives
people an upgrade path as
> > they
> > > > > > > >> desire.
> > > > > > > >>>> This
> > > > > > > >>>>>> also allows us to decide what
pieces of the
> compatibility
> > > shim
> > > > > go
> > > > > > > >> in
> > > > > > > >>>> the
> > > > > > > >>>>>> 2.0 server versus the 1.lastX
client. (I actually lean
> > > towards
> > > > > > > >>>> allowing a
> > > > > > > >>>>>> full break between v1 and v2 server/client
but
> understand
> > > that
> > > > > > that
> > > > > > > >>>> level
> > > > > > > >>>>>> or coordination is hard in many
organizations since
> > analysts
> > > > are
> > > > > > > >>>> separate
> > > > > > > >>>>>> from IT). Hopefully, what I'm
proposing can be a good
> > > > compromise
> > > > > > > >>>> between
> > > > > > > >>>>>> progress and deployment ease.
> > > > > > > >>>>>>
> > > > > > > >>>>>> Thoughts?
> > > > > > > >>>>>>
> > > > > > > >>>>>> Re: Branches/Dangers
> > > > > > > >>>>>>
> > > > > > > >>>>>> Good points on this Julian.
> > > > > > > >>>>>>
> > > > > > > >>>>>> How about this:
> > > > > > > >>>>>>
> > > > > > > >>>>>> - small fixes and enhancements
PRs should be made
> against
> > v1
> > > > > > > >>>>>> - new feature PRs should be made
against v2
> > > > > > > >>>>>> - v2 should continue to always
pass all precommit tests
> > > during
> > > > > its
> > > > > > > >>> life
> > > > > > > >>>>>> - v2 becomes master in two months
> > > > > > > >>>>>>
> > > > > > > >>>>>> I definitely don't want to create
instability in the v2
> > > > branch.
> > > > > > > >>>>>>
> > > > > > > >>>>>> The other option I see is we can
only do bug fix
> releases
> > > and
> > > > > > > >> branch
> > > > > > > >>>> the
> > > > > > > >>>>>> current master into a maintenance
branch and treat
> master
> > as
> > > > v2.
> > > > > > > >>>>>>
> > > > > > > >>>>>> Other ideas?
> > > > > > > >>>>>>
> > > > > > > >>>>>>
> > > > > > > >>>>>> --
> > > > > > > >>>>>> Jacques Nadeau
> > > > > > > >>>>>> CTO and Co-Founder, Dremio
> > > > > > > >>>>>>
> > > > > > > >>>>>> On Sat, Mar 26, 2016 at 6:07 PM,
Julian Hyde <
> > > > jhyde@apache.org <javascript:;>>
> > > > > > > >>> wrote:
> > > > > > > >>>>>>
> > > > > > > >>>>>>> Do you plan to be doing significant
development on both
> > the
> > > > v1
> > > > > > and
> > > > > > > >>> v2
> > > > > > > >>>>>>> branches, and if so, for how
long? I have been bitten
> > badly
> > > > by
> > > > > > > >> that
> > > > > > > >>>>> pattern
> > > > > > > >>>>>>> in the past. Developers put
lots of unrelated,
> > > destabilizing
> > > > > > > >> changes
> > > > > > > >>>>> into
> > > > > > > >>>>>>> v2, it look longer than expected
to stabilize v2,
> product
> > > > > > > >> management
> > > > > > > >>>>> lost
> > > > > > > >>>>>>> confidence in v2 and shifted
resources back to v1, and
> v2
> > > > never
> > > > > > > >>> caught
> > > > > > > >>>>> up
> > > > > > > >>>>>>> with v1.
> > > > > > > >>>>>>>
> > > > > > > >>>>>>> One important question: Which
branch will you ask
> people
> > to
> > > > > > target
> > > > > > > >>> for
> > > > > > > >>>>>>> pull requests? v1, v2 or both?
If they submit to v2,
> and
> > v2
> > > > is
> > > > > > > >>> broken,
> > > > > > > >>>>> how
> > > > > > > >>>>>>> will you know whether the
patches are good?
> > > > > > > >>>>>>>
> > > > > > > >>>>>>> My recommendation is to choose
one of the following:
> (1)
> > > put
> > > > a
> > > > > > > >>> strict
> > > > > > > >>>>>>> time limit of say 2 months
after which v2 would become
> > the
> > > > > master
> > > > > > > >>>> branch
> > > > > > > >>>>>>> (and v1 master would become
a maintenance branch), or
> (2)
> > > > make
> > > > > v2
> > > > > > > >>>>> focused
> > > > > > > >>>>>>> on a particular architectural
feature; create multiple
> > > > > > independent
> > > > > > > >>>>> feature
> > > > > > > >>>>>>> branches with breaking API
changes if you need to.
> > > > > > > >>>>>>>
> > > > > > > >>>>>>> Julian
> > > > > > > >>>>>>>
> > > > > > > >>>>>>>
> > > > > > > >>>>>>>> On Mar 26, 2016, at 1:41
PM, Paul Rogers <
> > > > > progers@maprtech.com <javascript:;>>
> > > > > > > >>>>> wrote:
> > > > > > > >>>>>>>>
> > > > > > > >>>>>>>> Hi All,
> > > > > > > >>>>>>>>
> > > > > > > >>>>>>>> 2.0 is a good opportunity
to enhance our ZK
> information.
> > > See
> > > > > > > >>>>>>> DRILL-4543: Advertise Drill-bit
ports, status,
> > capabilities
> > > > in
> > > > > > > >>>>> ZooKeeper.
> > > > > > > >>>>>>> This change will simplify
YARN integration.
> > > > > > > >>>>>>>>
> > > > > > > >>>>>>>> This enhancement will
change the “public API” in ZK.
> To
> > > > > Parth’s
> > > > > > > >>>> point,
> > > > > > > >>>>>>> we can do so in a way that
old clients work - as long
> as
> > a
> > > > > > > >> Drill-bit
> > > > > > > >>>>> uses
> > > > > > > >>>>>>> default ports.
> > > > > > > >>>>>>>>
> > > > > > > >>>>>>>> I’ve marked this JIRA
as a candidate for 2.0.
> > > > > > > >>>>>>>>
> > > > > > > >>>>>>>> Thanks,
> > > > > > > >>>>>>>>
> > > > > > > >>>>>>>> - Paul
> > > > > > > >>>>>>>>
> > > > > > > >>>>>>>>> On Mar 24, 2016, at
4:11 PM, Parth Chandra <
> > > > > parthc@apache.org <javascript:;>>
> > > > > > > >>>>> wrote:
> > > > > > > >>>>>>>>>
> > > > > > > >>>>>>>>> What's our proposal
for backward compatibility
> between
> > > 1.x
> > > > > and
> > > > > > > >>> 2.x?
> > > > > > > >>>>>>>>> My thoughts:
> > > > > > > >>>>>>>>> Optional  -  Allow
a mixture of 1.x and 2.x drillbits
> > in
> > > a
> > > > > > > >>> cluster.
> > > > > > > >>>>>>>>> Required - 1.x clients
should be able to talk to 2.x
> > > > > drillbits.
> > > > > > > >>>>>>>>>
> > > > > > > >>>>>>>>>
> > > > > > > >>>>>>>>>
> > > > > > > >>>>>>>>> On Thu, Mar 24, 2016
at 8:55 AM, Jacques Nadeau <
> > > > > > > >>>> jacques@dremio.com <javascript:;>>
> > > > > > > >>>>>>> wrote:
> > > > > > > >>>>>>>>>
> > > > > > > >>>>>>>>>> There are some
changes that either have reviews
> > pending
> > > or
> > > > > are
> > > > > > > >>> in
> > > > > > > >>>>>>> progress
> > > > > > > >>>>>>>>>> that would require
breaking changes to Drill core.
> > > > > > > >>>>>>>>>>
> > > > > > > >>>>>>>>>> Examples Include:
> > > > > > > >>>>>>>>>> DRILL-4455 (arrow
integration)
> > > > > > > >>>>>>>>>> DRILL-4417 (jdbc/odbc/rpc
changes)
> > > > > > > >>>>>>>>>> DRILL-4534 (improve
null performance)
> > > > > > > >>>>>>>>>>
> > > > > > > >>>>>>>>>> I've created a
new 2.0.0 release version in JIRA and
> > > moved
> > > > > > > >> these
> > > > > > > >>>>>>> tasks to
> > > > > > > >>>>>>>>>> that umbrella.
> > > > > > > >>>>>>>>>>
> > > > > > > >>>>>>>>>> I'd like to propose
a new v2 release branch where we
> > can
> > > > > start
> > > > > > > >>>>>>>>>> incorporating
these changes without disrupting v1
> > > > stability
> > > > > > > >> and
> > > > > > > >>>>>>>>>> compatibility.
> > > > > > > >>>>>>>>>>
> > > > > > > >>>>>>>>>>
> > > > > > > >>>>>>>>>> --
> > > > > > > >>>>>>>>>> Jacques Nadeau
> > > > > > > >>>>>>>>>> CTO and Co-Founder,
Dremio
> > > > > > > >>>>>>>>>>
> > > > > > > >>>>>>>>
> > > > > > > >>>>>>>
> > > > > > > >>>>>>>
> > > > > > > >>>>>>
> > > > > > > >>>>>
> > > > > > > >>>>
> > > > > > > >>>
> > > > > > > >>
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message