hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Todd Lipcon <t...@cloudera.com>
Subject Re: Building Hive - cannot resolve dependencies
Date Thu, 12 Nov 2009 20:49:46 GMT
On Thu, Nov 12, 2009 at 12:45 PM, Sasha Ovsankin <sasha@codebistro.com>wrote:

> Todd, Zheng --
>
> The build process relies on downloading 4 different (!) hadoop versions
> from http://archive.apache.org/dist/hadoop/core/hadoop-0.20.0/ , a total
> of 160MB and fails if something goes wrong.
>
> This doesn't make a lot of sense -- why do you need 4 versions?
>

Hive includes shims so that it can run against all of these versions. In
truth it really only needs the jars from these releases, but since Hadoop
isn't currently publishing its jars to a maven repo or anything, we have to
get the tarballs. I agree that this is a little annoying, but once you've
done it once it stays cached and you shouldn't have to worry about it.


>
> Also, could you provide error message that explains how to manually
> download and install the dependencies, not only "try again". Or point to the
> doc that does.
>

This is a good idea. Patches welcome :)


>
> BTW there might be a problem with http://archive.apache.org server -- it
> may be timing out connections which got me into an infinite loop of "try
> again".
>
>
I haven't run into that before. This problem with 0.19 is new (didn't happen
when we committed this built process). I've written up a sticky note on my
monitor to take a look when I have a moment.

-Todd


> Thanks,
> -- Sasha
>
> On Thu, Nov 12, 2009 at 2:30 AM, Zheng Shao <zshao9@gmail.com> wrote:
>
>> Hi Todd,
>>
>> I don't have a clear idea but it seems the ivy download code has a bug?
>> People get the files by doing "wget" on the same machine and it succeeds.
>>
>> If you have time, maybe we can debug into ivy's download code?
>>
>> Zheng
>>
>> On Thu, Nov 12, 2009 at 9:23 AM, Todd Lipcon <todd@cloudera.com> wrote:
>> > Hey Zheng,
>> >
>> > What do we need to do to fix this? It seems to have bitten a number of
>> > people by now.
>> >
>> > -Todd
>> >
>> > On Tue, Nov 10, 2009 at 3:50 PM, Zheng Shao <zshao9@gmail.com> wrote:
>> >>
>> >> I am forwarding an earlier email from the same mailing list by search
>> >> for "Downloaded file size doesn't match expected Content Length":
>> >>
>> >> "
>> >> Hi Rahul,
>> >>
>> >> Please follow these steps:
>> >>
>> >> 1) In your hive source directory run 'ant clean'.
>> >> 2) remove the contents of ~/.ant/cache/hadoop/core/sources
>> >> 3) Download the following files to ~/.ant/cache/hadoop/core/sources:
>> >>
>> >>      hadoop-0.17.2.1.tar.gz
>> >>      hadoop-0.17.2.1.tar.gz.asc
>> >>      hadoop-0.18.3.tar.gz
>> >>      hadoop-0.18.3.tar.gz.asc
>> >>
>> >>      hadoop-0.19.0.tar.gz
>> >>      hadoop-0.19.0.tar.gz.asc
>> >>      hadoop-0.20.0.tar.gz
>> >>      hadoop-0.20.0.tar.gz.asc
>> >>
>> >> 4) For each hadoop-xxx.tar.gz file, compute the sha1 checksum using
>> >> sha1sum, and verify that it matches the sha1 checksum in the
>> >> corresponding .asc file.
>> >>
>> >> If it does not match then the file is corrupt and you need to try
>> >> downloading it again.
>> >>
>> >> 5) Try building Hive again following the instructions on the wiki. You
>> >> shouldn't have any problems if you verified the checksums.
>> >>
>> >> As an additional note, if you don't care about support for Hadoop
>> >> 0.17.2.1, or 0.18, etc, you can disable support for these versions
>> >> (and skip the download) by removing the references to these versions
>> >> that shims/build.xml and shims/ivy.xml
>> >>
>> >>
>> >>
>> >>    Also I want to use hive on top of current working hadoop cluster.
>> >>    Please provide some inputs.
>> >>
>> >>
>> >> You need to set HADOOP_HOME and make sure that HADOOP_HOME/conf (or
>> >> HADOOP_CONF_DIR if you have this variable set) contains valid
>> >> configuration files for your current working hadoop cluster. See the
>> >> following page for more information:
>> >> http://wiki.apache.org/hadoop/GettingStartedWithHadoop
>> >>
>> >> Thanks.
>> >>
>> >> Carl
>> >> "
>> >>
>> >> On Tue, Nov 10, 2009 at 3:39 PM, Sasha Ovsankin <sasha@codebistro.com>
>> >> wrote:
>> >> > Hello, all --
>> >> >
>> >> > I am trying to build as per GettingStarted and am getting this error
>> >> > from
>> >> > Ivy:
>> >> >
>> >> > Downloaded file size doesn't match expected Content Length for
>> >> > ...hadoop-0.19.0.tar.gz. Please retry.
>> >> >
>> >> > This error repeats 4 times for 4 different versions of Hadoop and the
>> >> > build
>> >> > stops.
>> >> >
>> >> > I have tried on Mac and Windows with the same results. How can I
>> build
>> >> > Hive?
>> >> >
>> >> > Thanks for your help,
>> >> > -- Sasha
>> >> >
>> >> > P.S. This is try#3 to get over the spam filter.
>> >> >
>> >> >
>> >>
>> >>
>> >>
>> >> --
>> >> Yours,
>> >> Zheng
>> >
>> >
>>
>>
>>
>> --
>> Yours,
>> Zheng
>>
>
>

Mime
View raw message