impala-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Henry Robinson <he...@cloudera.com>
Subject Re: IMPALA-3223
Date Thu, 26 May 2016 00:03:08 GMT
Ok, thanks that makes sense. Sounds good to me!

Henry

> On May 25, 2016, at 4:59 PM, Tim Armstrong <tarmstrong@cloudera.com> wrote:
> 
> I think it's definitely easier to move everything out of /thirdparty for logistical reasons,
rather than trying to trim it more surgically. Most of the intermediate solutions to building
Apache Impala are harder if we have to special-case some parts of thirdparty. E.g. I was able
to build Apache Impala by using 'svn checkout' to grab thirdparty from Cloudera github, but
that would be trickier if thirdparty is still present in modified form.
> 
> We could have a thirdparty tree under be/src/ as well.
> 
>> On Wed, May 25, 2016 at 4:46 PM, Henry Robinson <henry@apache.org> wrote:
>> On 25 May 2016 at 16:33, Michael Ho <kwho@cloudera.com> wrote:
>> 
>> > Yes. The biggest dependency in thirdparty now is the reliance of the CDH
>> > components, namely
>> > hadoop, hbase, hive, sentry and some llama stuff. The short term goal as
>> > discussed in another email
>> > thread is to initially put a publicly available released version of those
>> > CDH components in S3 so a user
>> > can get all the components they need to build and run tests by just
>> > running the script to bootstrap the
>> > toolchain. Internally, we can replace those CDH components with our own
>> > builds of those components
>> > by somehow instructing the toolchain boostrapping script to pull from our
>> > internal repo if needed. The
>> > users can in theory specify their own repo for those components too.
>> > Whether we will converge to
>> > the Apache version of things in the long run is not the goal of this bug.
>> >
>> > The majority of the stuff left in thirdparty are mostly duplicates of what
>> > exist already in
>> > the toolchain directory: avro-c, sasl, gflags, glog, gperftools, gtest,
>> > lz4, openldap, rapidjson, re2, snappy,
>> > thrift so we may as well remove them.
>> >
>> > After all the changes above, thirdparty will be pretty much empty except
>> > for mustache and squeasel.
>> > If I recalled correctly from the meeting we had a month ago before you
>> > went on leave, you pointed out
>> > that they can reside in util directory as they are actually compiled
>> > together with the webserver.cc. I am
>> > also open to keeping them in thirdparty directory as this means we can now
>> > have a thirdparty directory
>> > in the ASF repo upstream after all the trimming above.
>> >
>> 
>> Thanks for the explanation. I still don't see from your response why you
>> think Squeasel and Mustache *should* move to util/, even if, as we
>> discussed, they *can* (there are good reasons to consider moving them -
>> maybe we don't want thirdparty/ any more, maybe we don't want to treat
>> external compile dependencies any differently).
>> 
>> The argument I can see in favour of not moving them is that any external
>> compile-time dependencies are in an easy-to-find place, which probably
>> makes getting an Apache release together just a little easier since we can
>> split the code into "developed as part of Impala" and "developed
>> externally".
>> 
>> I'm not going to stand in the way of a small change like this, so feel free
>> to GVM if you think best.
>> 
>> 
>> >
>> > Michael
>> >
>> > On Wed, May 25, 2016 at 3:30 PM, Henry Robinson <henry@apache.org> wrote:
>> >
>> >> I thought the long-term goal was to remove any heavyweight dependencies
>> >> that are checked into the repository. It's not the existence of thirdparty/
>> >> that's the problem there, it's what we were keeping in it.
>> >>
>> >> Mustache and Squeasel change very infrequently, and are only a few K in
>> >> total. Do you think we should remove thirdparty/ for other reasons?
>> >>
>> >> On 25 May 2016 at 14:20, Michael Ho <kwho@cloudera.com> wrote:
>> >>
>> >>> The long term goal is to remove thirdparty. I thought we discussed about
>> >>> it.
>> >>>
>> >>> On Wed, May 25, 2016 at 2:13 PM, Henry Robinson <henry@cloudera.com>
>> >>> wrote:
>> >>>
>> >>>> (Sorry, not near Gerrit) why not keep them in thirdparty/? That
makes
>> >>>> their provenance clear.
>> >>>>
>> >>>> Sent from my iPhone
>> >>>>
>> >>>> > On May 25, 2016, at 2:06 PM, Dan Hecht (Code Review) <
>> >>>> gerrit@cloudera.org> wrote:
>> >>>> >
>> >>>> > Dan Hecht has posted comments on this change.
>> >>>> >
>> >>>> > Change subject: IMPALA-3223: Move squeasel and mustache to
the util
>> >>>> directory
>> >>>> > ......................................................................
>> >>>> >
>> >>>> >
>> >>>> > Patch Set 1: Code-Review+2
>> >>>> >
>> >>>> > Okay.
>> >>>> >
>> >>>> > --
>> >>>> > To view, visit http://gerrit.cloudera.org:8080/3206
>> >>>> > To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
>> >>>> >
>> >>>> > Gerrit-MessageType: comment
>> >>>> > Gerrit-Change-Id: I782e493bc28086a1587274b3c474ea6b6f201855
>> >>>> > Gerrit-PatchSet: 1
>> >>>> > Gerrit-Project: Impala
>> >>>> > Gerrit-Branch: cdh5-trunk
>> >>>> > Gerrit-Owner: Michael Ho <kwho@cloudera.com>
>> >>>> > Gerrit-Reviewer: Dan Hecht <dhecht@cloudera.com>
>> >>>> > Gerrit-Reviewer: Jim Apple <jbapple@cloudera.com>
>> >>>> > Gerrit-Reviewer: Michael Ho <kwho@cloudera.com>
>> >>>> > Gerrit-HasComments: No
>> >>>> >
>> >>>> > --
>> >>>> > You received this message because you are subscribed to the
Google
>> >>>> Groups "impala-cr" group.
>> >>>> > To unsubscribe from this group and stop receiving emails from
it,
>> >>>> send an email to impala-cr+unsubscribe@cloudera.com.
>> >>>> > For more options, visit
>> >>>> https://groups.google.com/a/cloudera.com/d/optout.
>> >>>>
>> >>>
>> >>>
>> >>>
>> >>> --
>> >>> Thanks,
>> >>> Michael
>> >>>
>> >>
>> >>
>> >
>> >
>> > --
>> > Thanks,
>> > Michael
>> >
>> > --
>> > You received this message because you are subscribed to the Google Groups
>> > "impala-cr" group.
>> > To unsubscribe from this group and stop receiving emails from it, send an
>> > email to impala-cr+unsubscribe@cloudera.com.
>> > For more options, visit https://groups.google.com/a/cloudera.com/d/optout.
>> >
> 
> -- 
> You received this message because you are subscribed to the Google Groups "impala-cr"
group.
> To unsubscribe from this group and stop receiving emails from it, send an email to impala-cr+unsubscribe@cloudera.com.
> For more options, visit https://groups.google.com/a/cloudera.com/d/optout.

Mime
  • Unnamed multipart/alternative (inline, 7-Bit, 0 bytes)
View raw message