drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Steven Phillips <ste...@dremio.com>
Subject Re: [DISCUSS] Proposal to turn ValueVectors into separate reusable library & project
Date Tue, 10 Nov 2015 04:00:56 GMT
+1 on merging this soon.

Going forward, I agree it makes sense to break the RPC module into a
stand-alone module that is not specific to drill. But whether it is better
for it live in the Drill project or in the new Vector project, I am not
sure.

On Sun, Nov 8, 2015 at 6:42 PM, Jacques Nadeau <jacques@dremio.com> wrote:

> FYI, the patch also just successfully completed the extended regression
> suite.
>
> --
> Jacques Nadeau
> CTO and Co-Founder, Dremio
>
> On Sun, Nov 8, 2015 at 5:09 PM, Jacques Nadeau <jacques@dremio.com> wrote:
>
> > Ok guys,
> >
> > I took the quiet time directly after the release candidate went out to do
> > the first phase of componentization. You can see my work at [1].
> >
> > This set of commits has little functional impact. I've also done my best
> > to avoid package or file renaming, rather keeping things in their same
> > packages but in different modules (so that other patches are more easily
> > applied). There are nine commits in the branch. They break down into
> three
> > categories: MOVE, REFACTOR & CLEANUP.
> >
> > I've separated the changes out so that it should be reasonably
> > straightforward to review. The MOVE patches are constrained primarily to
> > moving files from module to another.
> >
> > DRILL-3987: (MOVE) Extract key vector, field reader, complex/field wr… …
> > 21cbd84
> > DRILL-3987: (REFACTOR) Common and Vector modules building. … e390db9
> > DRILL-3987: (REFACTOR) Working TPCH unit tests … 2cc1d30
> > DRILL-3987: (MOVE) Extract RPC, memory-base and memory-impl as separa… …
> > d5f3211
> > DRILL-3987: (REFACTOR) Extract BoundsChecking check from AssertionUti… …
> > 83c53d8
> > DRILL-3987: (CLEANUP) Delete unused files 5d596d5
> > DRILL-3987: (REFACTOR) Remove any parent Drill dependencies for drill… …
> > 76f578c
> > DRILL-3987: (MOVE) Move logical expressions and operators out of comm… …
> > f908b8b
> > DRILL-3987: (CLEANUP) Final cleanups to get complete working build/di… …
> > d09aa3b
> >
> > The main goal was to extract a number of separate java-exec submodules.
> > I've also outlined the modularization in a couple slides at [2]. In those
> > slides you'll see that there are some orange dependencies that will need
> to
> > be removed in a second phase of effort. We also need to decide which
> > portions of the third slide at [2] would be appropriate as a separate
> > project versus maintained inside of Drill.
> >
> > Some of the dependencies will need a finer grained hand to separate. The
> > biggest remaining is cleaning up VectorDescriptor, MaterializedField,
> > SerializedField, SchemaPath and FieldReference so that vector can stop
> > depending on the new drill-logical module.
> >
> > My preference would be to merge this straight away as the functional
> > impact is limited and it would be exceedingly difficult to maintain this
> > patch. This patch set provides a complete set of changes for
> modularization
> > and passes all unit tests. I'm running the extended regression suite now
> to
> > confirm no impact on those issues. I don't expect any since the only bugs
> > I've had to track down thus far are drill-module or pom dependency
> issues.
> >
> > Let me know your thoughts.
> >
> > [1] https://github.com/apache/drill/pull/250
> > [2]
> >
> https://docs.google.com/presentation/d/1HD-EzAgNe4EJvoP91ILFLFJdFjT2T5yfM9MEv79BaiM/edit
> >
> >
> > --
> > Jacques Nadeau
> > CTO and Co-Founder, Dremio
> >
> > On Tue, Oct 27, 2015 at 5:59 PM, Jacques Nadeau <jacques@dremio.com>
> > wrote:
> >
> >> Yes, I've started the umbrella @
> >> https://issues.apache.org/jira/browse/DRILL-3986
> >>
> >> And the first sub task: extraction poc @
> >> https://issues.apache.org/jira/browse/DRILL-3987
> >>
> >> I posted some existing materials. I'll start looking at how we can
> >> extract. Would love others thoughts about how we might slice things.
> I'll
> >> post some initial thoughts on the jiras in this regard.
> >>
> >> --
> >> Jacques Nadeau
> >> CTO and Co-Founder, Dremio
> >>
> >> On Tue, Oct 27, 2015 at 5:39 PM, Julian Hyde <jhyde@apache.org> wrote:
> >>
> >>> Jacques, Can you please log the JIRA case you mentioned, and also
> attach
> >>> any documentation (e.g. javadoc) you already have.
> >>>
> >>>
> >>
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message