bigtop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Evans Ye <evan...@apache.org>
Subject Re: Bundling Snappy with Hadoop
Date Wed, 10 Jun 2015 01:10:13 GMT
Thanks for the comment Olaf.
Whether or not to include snappy codec is optional in upstream hadoop
projrct, therefore I'm not quite sure to put it in to toolchain is the
right way to go. Since this implies that we'll build hadoop with snappy by
default.
Do you see most of your users or customer require this feature?
2015/6/10 上午4:13 於 "Olaf Flebbe" <of@oflebbe.de> 寫道:

> Hi Kelly,
>
> the bigtop-toolchain needs more love for amazon-linux. There is already a
> JIRA for it : BIGTOP-1883 , but it should be improved slightly.
>
> Olaf
>
>
>
> > Am 09.06.2015 um 21:35 schrieb Kelly, Jonathan <jonathak@amazon.com>:
> >
> > Oh, OK, do you guys want it in the toolchain for all distros by default?
> I
> > wasn't sure if you did (I probably should have asked), so for now I only
> > added it to AmazonLinux in our own repo. If you'd like to add it to all
> > distros, that would be great.
> >
> > Thanks,
> > Jonathan Kelly
> >
> >
> >
> >
> > On 6/9/15, 11:56 AM, "Olaf Flebbe" <of@oflebbe.de> wrote:
> >
> >> Hi Evans,
> >>
> >> Just discovered that snappy-devel is missing from bigtop_toolchain. Will
> >> open a JIRA for it.
> >>
> >> Olaf
> >>
> >>
> >>> Am 09.06.2015 um 05:13 schrieb Evans Ye <evansye@apache.org>:
> >>>
> >>> Yup, you need snappy-devel to be installed. Our new distribution
> >>> targeting bigtop 1.0 is building hadoop with snappy, so the snappy
> >>> compression should work at your side as well. :)
> >>>
> >>> 2015-06-09 6:57 GMT+08:00 Kelly, Jonathan <jonathak@amazon.com>:
> >>> Nvm, answered my own question. Apparently I want snappy-devel rather
> >>> than snappy.
> >>>
> >>> ~ Jonathan Kelly
> >>>
> >>> From: <Kelly>, Jonathan Kelly <jonathak@amazon.com>
> >>> Date: Monday, June 8, 2015 at 3:31 PM
> >>> To: "user@bigtop.apache.org" <user@bigtop.apache.org>
> >>> Subject: Bundling Snappy with Hadoop
> >>>
> >>> Is anybody bundling Hadoop with Snappy support on Bigtop? I noticed
> >>> that
> >>>
> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/h
> >>> adoop/do-component-build checks for the presence of
> >>> /usr/lib/libsnappy.so or /usr/lib64/libsnappy.so and sets
> $BUNDLE_SNAPPY
> >>> accordingly, so it seems that I should just need to install the snappy
> >>> package, and Hadoop will automatically be built with Snappy support.
> >>> However, when I install the snappy package ("sudo yum install -y
> snappy"
> >>> on AmazonLinux 2015.03), the library file that gets installed is
> >>> /usr/lib64/libsnappy.so, not /usr/lib64/libsnappy.so.1, so installing
> >>> the snappy package has no effect on the Hadoop build.
> >>>
> >>> Thanks,
> >>> Jonathan Kelly
> >>>
> >>
> >
>
>

Mime
View raw message