Mailing-List: contact common-user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: common-user@hadoop.apache.org
Received-SPF: pass (athena.apache.org: domain of phil.wills.young@gmail.com
 designates 209.85.214.176 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type;
        b=A9VSzE6DRVbbyrxOqCFUWAOJ63NjMHF1HZgixbp2xr4kCz3/CCAiimyEhAKcGnSCn+
         JncBOTPv73DYBNRekXlfUuIJzlfy8S5/0gIil7xqKxk4uwkuW2kf8XFS8Z+/lG87whnd
         +n0eykOTDYzAvFTAOiKdVDPNNNM9pgnACyz20=
MIME-Version: 1.0
In-Reply-To: <op.vrybohed7o0gc3@mail.cox.net>
References: <AANLkTin2TPzsJOXfbT8VVTCuyteRhOOcRo2mXBA4v+nT@mail.gmail.com>
	<0A8236F7A8A1364F9075835BD2DCEF3401CC1C@mail1.impetus.co.in>
	<op.vrybohed7o0gc3@mail.cox.net>
Date: Mon, 7 Mar 2011 00:37:22 -0500
Message-ID: <AANLkTinRWdnmByuun+LAcz6ZvBZSrC9s8CRY8sRd2sUR@mail.gmail.com>
Subject: Re: LZO Packager on Centos 5.5 fails
From: phil young <phil.wills.young@gmail.com>
To: common-user@hadoop.apache.org
Content-Type: multipart/alternative; boundary=20cf303f6284d7f02a049ddde388

--20cf303f6284d7f02a049ddde388
Content-Type: text/plain; charset=ISO-8859-1

Thanks for the responses.

Saurabh,
  Thanks for the response. I got LZO installed without the packager, but I
do appreciate the help as the rpm method seems like a better way to go.

Pete,
  The clusters are all 64-bit machines. That doesn't matter at this point
because I was able to get this to work without the packager by reading many,
many web pages. I had wanted to use the packager because I assumed it would
resolve open questions like: if the code evolved from
GoogleCode->KevinWeil->ToddLipconn, and the GoogleCode says to use branch
0.1 because the trunk is for the new Hadoop APIs (which Hive doesn't use
yet), then what's the right thing to install.

I do have LZO working now with Hive. Tables can now be compressed as LZO,
with Indexes, with data validated via lzop directly, and I can use
intermediate LZO compression. That took more work than I anticipated, so I'm
still interested in the packager.


To sum up what I've done:
I replaced "wget" with "wget --no-check-certificate" in run.sh.

I did replace the string in the file you mentioned, but the only file with
that name is in the build directory
[root@adhocmaster01an hadoop-lzo-packager]# find ./ -name
"*toddlipcon-hadoop-lzo.spec*"
./build/topdir/SPECS/toddlipcon-hadoop-lzo.spec

Unsurprisingly, executing "run.sh" again yielded the original error message.
It seems I should be able to change the version tag "upstream" somewhere,
but I don't have enough time now to research SPEC files and how deep this
rabbit hole might go.

I do appreciate the body of work that this is all built on and your
responses, but I am surprised that there isn't a more comprehensive set of
How-Tos. I'll assist in writing up something like that when I get a chance.
Are other people using CDH3B3 with 64-bit CentOS/RedHat, or is there
something I'm missing?

Thanks for the assistance.


On Sun, Mar 6, 2011 at 9:25 PM, Pete Haidinyak <javamann@cox.net> wrote:

> Are you doing a 64bit build or doesn't it matter?
>
> Thanks
>
> -Pete
>
>
> On Sat, 05 Mar 2011 21:35:30 -0800, Saurabh Dutta <
> saurabh.dutta@impetus.co.in> wrote:
>
>  Hi,
>>
>> Try and open the file "toddlipcon-hadoop-lzo.spec" in your SPECS directory
>> and replace the '-' with '_' in line number 3. See if this works for you.
>>
>> Thanks,
>> Saurabh
>> ________________________________________
>> From: phil young [phil.wills.young@gmail.com]
>> Sent: Sunday, March 06, 2011 4:27 AM
>> To: hadoop-common-user
>> Subject: LZO Packager on Centos 5.5 fails
>>
>> I'm trying to install LZO on CDH3B3, using the LZO Packager tool here (
>> https://github.com/toddlipcon/hadoop-lzo-packager)
>> I'm running the following on Centos 5.5.
>>
>> I'm not an expert on RPM building. Does anyone have any suggestions about
>> how to debug this?
>>
>> hadoop-lzo-packager]# ./run.sh
>> + ANT_VERSION=1.8.1
>> + ANT_TARBALL=apache-ant-1.8.1-bin.tar.gz
>> + ANT_TARBALL_URL=
>>
>> http://www.gtlib.gatech.edu/pub/apache/ant/binaries/apache-ant-1.8.1-bin.tar.gz
>> + '[' -n '' ']'
>> + '[' -n '' ']'
>> + SRC_PROJECT=github
>> + RELEASE=1
>> ++ getent passwd root
>> ++ cut -d: -f5
>> ++ cut -d, -f1
>> + PACKAGER=root
>> ++ hostname -f
>> + HOST=pyoung-dev.tripadvisor.com
>> + PACKAGER_EMAIL=root@pyoung-dev.tripadvisor.com
>> + HADOOP_HOME=/usr/lib/hadoop
>> +++ dirname ./run.sh
>> ++ readlink -f .
>> + BINDIR=/root/pwy/lzo_research/toddlipcon/hadoop-lzo-packager
>> + mkdir -p build
>> + setup_github
>> + GITHUB_ACCOUNT=toddlipcon
>> + GITHUB_BRANCH=master
>> + PACKAGE_HOMEPAGE=http://github.com/toddlipcon/hadoop-lzo
>> + TARURL=http://github.com/toddlipcon/hadoop-lzo/tarball/master
>> ++ ls
>>
>> /root/pwy/lzo_research/toddlipcon/hadoop-lzo-packager/build/toddlipcon-hadoop-lzo-0.4.9-3-g2bd0d5b.tar.gz
>>
>> /root/pwy/lzo_research/toddlipcon/hadoop-lzo-packager/build/toddlipcon-hadoop-lzo-20110305174723.0.4.9-3-g2bd0d5b.tar.gz
>> + '[' -z
>>
>> '/root/pwy/lzo_research/toddlipcon/hadoop-lzo-packager/build/toddlipcon-hadoop-lzo-0.4.9-3-g2bd0d5b.tar.gz
>>
>> /root/pwy/lzo_research/toddlipcon/hadoop-lzo-packager/build/toddlipcon-hadoop-lzo-20110305174723.0.4.9-3-g2bd0d5b.tar.gz'
>> ']'
>> ++ ls -1
>>
>> /root/pwy/lzo_research/toddlipcon/hadoop-lzo-packager/build/toddlipcon-hadoop-lzo-0.4.9-3-g2bd0d5b.tar.gz
>>
>> /root/pwy/lzo_research/toddlipcon/hadoop-lzo-packager/build/toddlipcon-hadoop-lzo-20110305174723.0.4.9-3-g2bd0d5b.tar.gz
>> ++ head -1
>> +
>>
>> ORIG_TAR=/root/pwy/lzo_research/toddlipcon/hadoop-lzo-packager/build/toddlipcon-hadoop-lzo-0.4.9-3-g2bd0d5b.tar.gz
>> ++ expr match
>>
>> /root/pwy/lzo_research/toddlipcon/hadoop-lzo-packager/build/toddlipcon-hadoop-lzo-0.4.9-3-g2bd0d5b.tar.gz
>> '.*hadoop-lzo-\(.*\).tar.gz'
>> + GIT_HASH=0.4.9-3-g2bd0d5b
>> + echo 'Git hash: 0.4.9-3-g2bd0d5b'
>> Git hash: 0.4.9-3-g2bd0d5b
>> + NAME=toddlipcon-hadoop-lzo
>> ++ date +%Y%m%d%H%M%S
>> + VERSION=20110305174749.0.4.9-3-g2bd0d5b
>> + pushd /root/pwy/lzo_research/toddlipcon/hadoop-lzo-packager/build/
>> + mkdir toddlipcon-hadoop-lzo-20110305174749.0.4.9-3-g2bd0d5b/
>> + tar -C toddlipcon-hadoop-lzo-20110305174749.0.4.9-3-g2bd0d5b/
>> --strip-components=1 -xzf
>>
>> /root/pwy/lzo_research/toddlipcon/hadoop-lzo-packager/build/toddlipcon-hadoop-lzo-0.4.9-3-g2bd0d5b.tar.gz
>> + tar czf toddlipcon-hadoop-lzo-20110305174749.0.4.9-3-g2bd0d5b.tar.gz
>> toddlipcon-hadoop-lzo-20110305174749.0.4.9-3-g2bd0d5b/
>> + popd
>> +
>>
>> CHECKOUT_TAR=/root/pwy/lzo_research/toddlipcon/hadoop-lzo-packager/build/toddlipcon-hadoop-lzo-20110305174749.0.4.9-3-g2bd0d5b.tar.gz
>> +
>> TOPDIR=/root/pwy/lzo_research/toddlipcon/hadoop-lzo-packager/build/topdir
>> +
>>
>> CHECKOUT=/root/pwy/lzo_research/toddlipcon/hadoop-lzo-packager/toddlipcon-hadoop-lzo-20110305174749.0.4.9-3-g2bd0d5b
>> + checkout_github
>> + echo -n
>> + '[' '!' -e
>>
>> /root/pwy/lzo_research/toddlipcon/hadoop-lzo-packager/build/toddlipcon-hadoop-lzo-20110305174749.0.4.9-3-g2bd0d5b.tar.gz
>> ']'
>> + '[' -n '' ']'
>> + '[' -z '' ']'
>> + rm -Rf
>> /root/pwy/lzo_research/toddlipcon/hadoop-lzo-packager/build/topdir
>> + mkdir -p
>> /root/pwy/lzo_research/toddlipcon/hadoop-lzo-packager/build/topdir
>> + cd /root/pwy/lzo_research/toddlipcon/hadoop-lzo-packager/build/topdir/
>> + mkdir SOURCES BUILD SPECS SRPMS RPMS BUILDROOT
>> + cat /root/pwy/lzo_research/toddlipcon/hadoop-lzo-packager/template.spec
>> + do_substs
>> + sed '
>>  s,@PACKAGE_NAME@,toddlipcon-hadoop-lzo,g;
>>  s,@PACKAGE_HOMEPAGE@,http://github.com/toddlipcon/hadoop-lzo,g;
>>  s,@VERSION@,20110305174749.0.4.9-3-g2bd0d5b,g;
>>  s,@RELEASE@,1,g;
>>  s,@PACKAGER@,root,g;
>>  s,@PACKAGER_EMAIL@,root@pyoung-dev.tripadvisor.com,g;
>>  s,@HADOOP_HOME@,/usr/lib/hadoop,g;
>> '
>> + cp
>>
>> /root/pwy/lzo_research/toddlipcon/hadoop-lzo-packager/build/toddlipcon-hadoop-lzo-20110305174749.0.4.9-3-g2bd0d5b.tar.gz
>> /root/pwy/lzo_research/toddlipcon/hadoop-lzo-packager/build/topdir/SOURCES
>> + pushd
>> /root/pwy/lzo_research/toddlipcon/hadoop-lzo-packager/build/topdir/SPECS
>> ++ pwd
>> ++ pwd
>> + rpmbuild --buildroot
>>
>> /root/pwy/lzo_research/toddlipcon/hadoop-lzo-packager/build/topdir/SPECS/../BUILDROOT
>> --define '_topdir
>>
>> /root/pwy/lzo_research/toddlipcon/hadoop-lzo-packager/build/topdir/SPECS/..'
>> -ba toddlipcon-hadoop-lzo.spec
>> error: line 3: Illegal char '-' in version: Version:
>> 20110305174749.0.4.9-3-g2bd0d5b
>>
>> ________________________________
>>
>> Are you exploring a Big Data Strategy ? Listen to this recorded webinar on
>> Planning your Hadoop/ NoSQL projects for 2011 at
>> www.impetus.com/featured_webinar?eventid=37
>>
>> Follow us on www.twitter.com/impetuscalling or visit www.impetus.com to
>> know more.
>>
>>
>> NOTE: This message may contain information that is confidential,
>> proprietary, privileged or otherwise protected by law. The message is
>> intended solely for the named addressee. If received in error, please
>> destroy and notify the sender. Any use of this email is prohibited when
>> received in error. Impetus does not represent, warrant and/or guarantee,
>> that the integrity of this communication has been maintained nor that the
>> communication is free of errors, virus, interception or interference.
>>
>
>

--20cf303f6284d7f02a049ddde388--