hadoop-hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Carl Steinbach (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HIVE-984) Building Hive occasionally fails with Ivy error: hadoop#core;0.20.1!hadoop.tar.gz(source): invalid md5:
Date Tue, 16 Feb 2010 23:19:28 GMT

    [ https://issues.apache.org/jira/browse/HIVE-984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12834544#action_12834544

Carl Steinbach commented on HIVE-984:

bq.  If at all possible, we want to keep building all supported shims as part of ant package
to make sure that when a change breaks one, the developer finds out early (before even submitting
a bad patch)

Unless your change specifically mucks with the shim code I think it's unlikely that you're
going to introduce a compile time error. It seems more likely that you would cause a test
error, and that's something you will only catch if you run the full test suite against all
supported versions -- something that we only expect Hudson to do.

Which brings up another point. How do we configure JIRA/Hudson to automatically test submitted
patches? The Hadoop and Pig projects are both setup to do this, but I can't find any references
to how it was done. Do either of you know how to set this up, or have objections to doing

bq. Before doing anything else, I'm going to see if a more reliable source than archive.apache.org
would address the problem. I'll test this with my home network tomorrow, which usually fails
with archive.apache.org.

Over the weekend I figured out that there are actually two different reasons why people are
encountering errors during the download process, and wanted to make sure that everyone else
is aware of this as well:

# Unable to connect to archive.apache.org: We can fix this by adding additional apache mirrors
(see http://www.apache.org/mirrors/) to the hadoop-source resolver in ivysettings, and also
by letting people know that they can explicitly set the mirror location using the hadoop.mirror

# -Dhadoop.version=0.20.1: When people set hadoop.version to 0.20.1 it causes ant to download
both 0.20.0 *and* 0.20.1, which is unnecessary since the API does not change between patch
releases. But the bigger problem is that 0.20.1's md5 checksum file on archive.apache.org
contains an md5 hash along with a bunch of other garbage that breaks ivy. We can fix this
either by disabling checksums for archive.apache.org (set ivy.checksums="" on that resolver),
or by enhancing the build script so that it ignores patch release numbers and maps 0.20.1
to 0.20.0.

> Building Hive occasionally fails with Ivy error: hadoop#core;0.20.1!hadoop.tar.gz(source):
invalid md5:
> -------------------------------------------------------------------------------------------------------
>                 Key: HIVE-984
>                 URL: https://issues.apache.org/jira/browse/HIVE-984
>             Project: Hadoop Hive
>          Issue Type: Bug
>          Components: Build Infrastructure
>            Reporter: Carl Steinbach
>            Assignee: Carl Steinbach
>         Attachments: HIVE-984.2.patch, HIVE-984.patch
> Folks keep running into this problem when building Hive from source:
> {noformat}
> [ivy:retrieve]
> [ivy:retrieve] :: problems summary ::
> [ivy:retrieve] :::: WARNINGS
> [ivy:retrieve]          [FAILED     ]
> hadoop#core;0.20.1!hadoop.tar.gz(source): invalid md5:
> expected=hadoop-0.20.1.tar.gz: computed=719e169b7760c168441b49f405855b72
> (138662ms)
> [ivy:retrieve]          [FAILED     ]
> hadoop#core;0.20.1!hadoop.tar.gz(source): invalid md5:
> expected=hadoop-0.20.1.tar.gz: computed=719e169b7760c168441b49f405855b72
> (138662ms)
> [ivy:retrieve]  ==== hadoop-resolver: tried
> [ivy:retrieve]
> http://archive.apache.org/dist/hadoop/core/hadoop-0.20.1/hadoop-0.20.1.tar.gz
> [ivy:retrieve]          ::::::::::::::::::::::::::::::::::::::::::::::
> [ivy:retrieve]          ::              FAILED DOWNLOADS            ::
> [ivy:retrieve]          :: ^ see resolution messages for details  ^ ::
> [ivy:retrieve]          ::::::::::::::::::::::::::::::::::::::::::::::
> [ivy:retrieve]          :: hadoop#core;0.20.1!hadoop.tar.gz(source)
> [ivy:retrieve]          ::::::::::::::::::::::::::::::::::::::::::::::
> [ivy:retrieve]
> {noformat}
> The problem appears to be either with a) the Hive build scripts, b) ivy, or c) archive.apache.org
> Besides fixing the actual bug, one other option worth considering is to add the Hadoop
jars to the
> Hive source repository.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message