impala-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Ho <k...@cloudera.com>
Subject Re: IMPALA-3223
Date Thu, 26 May 2016 00:22:32 GMT
Other than the logistics reasons Tim mentioned, my thinking is that thirdparty and toolchain
overlap a lot so it would be easier to have just one location for everything.

Thanks,
Michael

> On May 25, 2016, at 16:46, Henry Robinson <henry@apache.org> wrote:
> 
>> On 25 May 2016 at 16:33, Michael Ho <kwho@cloudera.com> wrote:
>> Yes. The biggest dependency in thirdparty now is the reliance of the CDH components,
namely
>> hadoop, hbase, hive, sentry and some llama stuff. The short term goal as discussed
in another email
>> thread is to initially put a publicly available released version of those CDH components
in S3 so a user
>> can get all the components they need to build and run tests by just running the script
to bootstrap the
>> toolchain. Internally, we can replace those CDH components with our own builds of
those components
>> by somehow instructing the toolchain boostrapping script to pull from our internal
repo if needed. The
>> users can in theory specify their own repo for those components too. Whether we will
converge to 
>> the Apache version of things in the long run is not the goal of this bug.
>> 
>> The majority of the stuff left in thirdparty are mostly duplicates of what exist
already in
>> the toolchain directory: avro-c, sasl, gflags, glog, gperftools, gtest, lz4, openldap,
rapidjson, re2, snappy,
>> thrift so we may as well remove them.
>> 
>> After all the changes above, thirdparty will be pretty much empty except for mustache
and squeasel.
>> If I recalled correctly from the meeting we had a month ago before you went on leave,
you pointed out
>> that they can reside in util directory as they are actually compiled together with
the webserver.cc. I am
>> also open to keeping them in thirdparty directory as this means we can now have a
thirdparty directory
>> in the ASF repo upstream after all the trimming above.
> 
> Thanks for the explanation. I still don't see from your response why you think Squeasel
and Mustache should move to util/, even if, as we discussed, they can (there are good reasons
to consider moving them - maybe we don't want thirdparty/ any more, maybe we don't want to
treat external compile dependencies any differently). 
> 
> The argument I can see in favour of not moving them is that any external compile-time
dependencies are in an easy-to-find place, which probably makes getting an Apache release
together just a little easier since we can split the code into "developed as part of Impala"
and "developed externally".
> 
> I'm not going to stand in the way of a small change like this, so feel free to GVM if
you think best.
>  
>> 
>> Michael
>> 
>>> On Wed, May 25, 2016 at 3:30 PM, Henry Robinson <henry@apache.org> wrote:
>>> I thought the long-term goal was to remove any heavyweight dependencies that
are checked into the repository. It's not the existence of thirdparty/ that's the problem
there, it's what we were keeping in it. 
>>> 
>>> Mustache and Squeasel change very infrequently, and are only a few K in total.
Do you think we should remove thirdparty/ for other reasons? 
>>> 
>>>> On 25 May 2016 at 14:20, Michael Ho <kwho@cloudera.com> wrote:
>>>> The long term goal is to remove thirdparty. I thought we discussed about
it.
>>>> 
>>>>> On Wed, May 25, 2016 at 2:13 PM, Henry Robinson <henry@cloudera.com>
wrote:
>>>>> (Sorry, not near Gerrit) why not keep them in thirdparty/? That makes
their provenance clear.
>>>>> 
>>>>> Sent from my iPhone
>>>>> 
>>>>> > On May 25, 2016, at 2:06 PM, Dan Hecht (Code Review) <gerrit@cloudera.org>
wrote:
>>>>> >
>>>>> > Dan Hecht has posted comments on this change.
>>>>> >
>>>>> > Change subject: IMPALA-3223: Move squeasel and mustache to the util
directory
>>>>> > ......................................................................
>>>>> >
>>>>> >
>>>>> > Patch Set 1: Code-Review+2
>>>>> >
>>>>> > Okay.
>>>>> >
>>>>> > --
>>>>> > To view, visit http://gerrit.cloudera.org:8080/3206
>>>>> > To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
>>>>> >
>>>>> > Gerrit-MessageType: comment
>>>>> > Gerrit-Change-Id: I782e493bc28086a1587274b3c474ea6b6f201855
>>>>> > Gerrit-PatchSet: 1
>>>>> > Gerrit-Project: Impala
>>>>> > Gerrit-Branch: cdh5-trunk
>>>>> > Gerrit-Owner: Michael Ho <kwho@cloudera.com>
>>>>> > Gerrit-Reviewer: Dan Hecht <dhecht@cloudera.com>
>>>>> > Gerrit-Reviewer: Jim Apple <jbapple@cloudera.com>
>>>>> > Gerrit-Reviewer: Michael Ho <kwho@cloudera.com>
>>>>> > Gerrit-HasComments: No
>>>>> >
>>>>> > --
>>>>> > You received this message because you are subscribed to the Google
Groups "impala-cr" group.
>>>>> > To unsubscribe from this group and stop receiving emails from it,
send an email to impala-cr+unsubscribe@cloudera.com.
>>>>> > For more options, visit https://groups.google.com/a/cloudera.com/d/optout.
>>>> 
>>>> 
>>>> 
>>>> -- 
>>>> Thanks,
>>>> Michael
>> 
>> 
>> 
>> -- 
>> Thanks,
>> Michael
>> -- 
>> You received this message because you are subscribed to the Google Groups "impala-cr"
group.
>> To unsubscribe from this group and stop receiving emails from it, send an email to
impala-cr+unsubscribe@cloudera.com.
>> For more options, visit https://groups.google.com/a/cloudera.com/d/optout.
> 

Mime
  • Unnamed multipart/alternative (inline, 7-Bit, 0 bytes)
View raw message