bigtop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Olaf Flebbe ...@oflebbe.de>
Subject Re: Bundling Snappy with Hadoop
Date Tue, 09 Jun 2015 20:13:19 GMT
Hi Kelly,

the bigtop-toolchain needs more love for amazon-linux. There is already a JIRA for it : BIGTOP-1883
, but it should be improved slightly.

Olaf



> Am 09.06.2015 um 21:35 schrieb Kelly, Jonathan <jonathak@amazon.com>:
> 
> Oh, OK, do you guys want it in the toolchain for all distros by default? I
> wasn't sure if you did (I probably should have asked), so for now I only
> added it to AmazonLinux in our own repo. If you'd like to add it to all
> distros, that would be great.
> 
> Thanks,
> Jonathan Kelly
> 
> 
> 
> 
> On 6/9/15, 11:56 AM, "Olaf Flebbe" <of@oflebbe.de> wrote:
> 
>> Hi Evans,
>> 
>> Just discovered that snappy-devel is missing from bigtop_toolchain. Will
>> open a JIRA for it.
>> 
>> Olaf
>> 
>> 
>>> Am 09.06.2015 um 05:13 schrieb Evans Ye <evansye@apache.org>:
>>> 
>>> Yup, you need snappy-devel to be installed. Our new distribution
>>> targeting bigtop 1.0 is building hadoop with snappy, so the snappy
>>> compression should work at your side as well. :)
>>> 
>>> 2015-06-09 6:57 GMT+08:00 Kelly, Jonathan <jonathak@amazon.com>:
>>> Nvm, answered my own question. Apparently I want snappy-devel rather
>>> than snappy.
>>> 
>>> ~ Jonathan Kelly
>>> 
>>> From: <Kelly>, Jonathan Kelly <jonathak@amazon.com>
>>> Date: Monday, June 8, 2015 at 3:31 PM
>>> To: "user@bigtop.apache.org" <user@bigtop.apache.org>
>>> Subject: Bundling Snappy with Hadoop
>>> 
>>> Is anybody bundling Hadoop with Snappy support on Bigtop? I noticed
>>> that
>>> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/h
>>> adoop/do-component-build checks for the presence of
>>> /usr/lib/libsnappy.so or /usr/lib64/libsnappy.so and sets $BUNDLE_SNAPPY
>>> accordingly, so it seems that I should just need to install the snappy
>>> package, and Hadoop will automatically be built with Snappy support.
>>> However, when I install the snappy package ("sudo yum install -y snappy"
>>> on AmazonLinux 2015.03), the library file that gets installed is
>>> /usr/lib64/libsnappy.so, not /usr/lib64/libsnappy.so.1, so installing
>>> the snappy package has no effect on the Hadoop build.
>>> 
>>> Thanks,
>>> Jonathan Kelly
>>> 
>> 
> 


Mime
View raw message