drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jacques Nadeau <jacq...@dremio.com>
Subject Re: [DISCUSS] Proposal to turn ValueVectors into separate reusable library & project
Date Mon, 09 Nov 2015 02:42:01 GMT
FYI, the patch also just successfully completed the extended regression
suite.

--
Jacques Nadeau
CTO and Co-Founder, Dremio

On Sun, Nov 8, 2015 at 5:09 PM, Jacques Nadeau <jacques@dremio.com> wrote:

> Ok guys,
>
> I took the quiet time directly after the release candidate went out to do
> the first phase of componentization. You can see my work at [1].
>
> This set of commits has little functional impact. I've also done my best
> to avoid package or file renaming, rather keeping things in their same
> packages but in different modules (so that other patches are more easily
> applied). There are nine commits in the branch. They break down into three
> categories: MOVE, REFACTOR & CLEANUP.
>
> I've separated the changes out so that it should be reasonably
> straightforward to review. The MOVE patches are constrained primarily to
> moving files from module to another.
>
> DRILL-3987: (MOVE) Extract key vector, field reader, complex/field wr… …
> 21cbd84
> DRILL-3987: (REFACTOR) Common and Vector modules building. … e390db9
> DRILL-3987: (REFACTOR) Working TPCH unit tests … 2cc1d30
> DRILL-3987: (MOVE) Extract RPC, memory-base and memory-impl as separa… …
> d5f3211
> DRILL-3987: (REFACTOR) Extract BoundsChecking check from AssertionUti… …
> 83c53d8
> DRILL-3987: (CLEANUP) Delete unused files 5d596d5
> DRILL-3987: (REFACTOR) Remove any parent Drill dependencies for drill… …
> 76f578c
> DRILL-3987: (MOVE) Move logical expressions and operators out of comm… …
> f908b8b
> DRILL-3987: (CLEANUP) Final cleanups to get complete working build/di… …
> d09aa3b
>
> The main goal was to extract a number of separate java-exec submodules.
> I've also outlined the modularization in a couple slides at [2]. In those
> slides you'll see that there are some orange dependencies that will need to
> be removed in a second phase of effort. We also need to decide which
> portions of the third slide at [2] would be appropriate as a separate
> project versus maintained inside of Drill.
>
> Some of the dependencies will need a finer grained hand to separate. The
> biggest remaining is cleaning up VectorDescriptor, MaterializedField,
> SerializedField, SchemaPath and FieldReference so that vector can stop
> depending on the new drill-logical module.
>
> My preference would be to merge this straight away as the functional
> impact is limited and it would be exceedingly difficult to maintain this
> patch. This patch set provides a complete set of changes for modularization
> and passes all unit tests. I'm running the extended regression suite now to
> confirm no impact on those issues. I don't expect any since the only bugs
> I've had to track down thus far are drill-module or pom dependency issues.
>
> Let me know your thoughts.
>
> [1] https://github.com/apache/drill/pull/250
> [2]
> https://docs.google.com/presentation/d/1HD-EzAgNe4EJvoP91ILFLFJdFjT2T5yfM9MEv79BaiM/edit
>
>
> --
> Jacques Nadeau
> CTO and Co-Founder, Dremio
>
> On Tue, Oct 27, 2015 at 5:59 PM, Jacques Nadeau <jacques@dremio.com>
> wrote:
>
>> Yes, I've started the umbrella @
>> https://issues.apache.org/jira/browse/DRILL-3986
>>
>> And the first sub task: extraction poc @
>> https://issues.apache.org/jira/browse/DRILL-3987
>>
>> I posted some existing materials. I'll start looking at how we can
>> extract. Would love others thoughts about how we might slice things. I'll
>> post some initial thoughts on the jiras in this regard.
>>
>> --
>> Jacques Nadeau
>> CTO and Co-Founder, Dremio
>>
>> On Tue, Oct 27, 2015 at 5:39 PM, Julian Hyde <jhyde@apache.org> wrote:
>>
>>> Jacques, Can you please log the JIRA case you mentioned, and also attach
>>> any documentation (e.g. javadoc) you already have.
>>>
>>>
>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message